ESP32-S3 as a USB Audio Device

That’s right, folks – it’s time for an audio project.

I’ve been testing out my new boards - so far everything is going well.

First Tests – LEDs, Sensors, and Battery Charging

So far, so good:

• The RGB LED needed a small ninja fix. I put in a common cathode LED when it should have been common anode (or was it the other way round? Either way… fixed!).
• The accelerometer and gyroscope worked nicely, letting me build a digital twin of the board.
• The battery charger worked perfectly – no wires required while moving the board around for testing.

With that out of the way, it was time for the fun part: microphone and speaker tests.

Testing the Microphone

The board has:

• An I²S PDM microphone
• An I²S PCM amplifier
• A tiny speaker

My first thought: make it into a Bluetooth audio device. Unfortunately, the ESP32-S3 only supports Bluetooth Low Energy, not Bluetooth Classic – and BLE doesn’t do audio streaming.

So instead, I pushed audio over USB serial. Turns out you can send a lot of data that way – around 7 Mbps in previous tests. For audio, I only needed ~32.2 KB/s (16 kHz, 16-bit samples).

And it worked beautifully.

I even had AI generate a little Web Serial Audio Studio for me:

• Oscilloscope
• Gain control
• Spectrogram (looks pretty, not sure it’s correct…)
• A glorious 1980s-style bar chart
• Pitch tuner

Hooking up a piano confirmed the pitch tuner was picking up notes correctly. Microphone test: ✅

Moving to USB Audio (UAC)

Sending audio over serial worked fine, but then I remembered – the ESP32-S3 has native USB.

So why not make it into a USB Audio Class (UAC) device, like a USB headset or speaker?

And yes, it turns out you can.

The API gives you callbacks for:

• Output
• Input
• Mute
• Volume

The only catch: I couldn’t get the Arduino libraries to work. I had to switch over to ESP-IDF.

Thankfully, the ESP-IDF VS Code plugin makes that a lot easier with its explorer, build, and flash tools.

Limitations: Mac vs Windows

One big drawback:

• Tick “Support macOS” → Works on Mac, but not Windows.
• Untick it → Works on Windows, but not Mac.

Pretty annoying, but let’s see how it performs anyway.

Microphone Over USB

Switching back to my audio studio, the ESP32-S3 appeared as a new USB UAC device.

Connected up, the audio streamed in just fine:

I recorded a test, pulled it into Audacity, and amplified it a bit (these microphones are sensitive, so raw audio often looks quiet).

Result: very clean! 🎤

Speaker Over USB

The speaker was a different story.

At first, it worked – but the sound was very crackly when streaming via USB UAC. After a lot of head scratching, I still couldn’t fix it.

So I went back to basics: playing a WAV file directly from SPIFFS over I²S.

That sounded perfectly clean:

So clearly the hardware is fine – it’s the USB UAC speaker code that’s the problem. Microphone = great. Speaker = not quite there yet.

Final Verdict

So where are we?

• ✅ LED fixed
• ✅ Sensors working
• ✅ Battery charging fine
• ✅ Microphone excellent (both serial + USB)
• ⚠️ Speaker fine via I²S, but problematic via USB UAC

That’s good enough for me – this board has passed quality assurance.

There’s a full video here where you can listen to the audio: YouTube Link