Text-to-speech that runs on a microcontroller
vtts turns text into speech entirely on an ESP32: text normalization, pronunciation, an int8 neural voice and the vocoder all run on the chip. No cloud, no network, no Linux. Every clip on this page was generated by the chip and sent back over its serial port.
Listen
Each column is a model with its size and real-time factor (RTF) per chip; below 1 is faster than real time. Both models run on both chips. Every clip is the chip's own output, only compressed to MP3.
How it works
Everything below runs on the microcontroller, from the raw text to the samples sent to the speaker.
- UTF-8 text
- Language tag per word
- Normalizer (numbers, dates, units)
- G2P
- Duration + acoustic model (int8)
- Vocos decoder + iSTFT
- PCM 22.05 kHz → I2S speaker
Runtime
C99, no malloc: the caller passes one arena whose size is known up front. Int8 weights with per-channel scales, int8 activations with one scale per frame. Streaming keeps per-layer state, so every frame is computed once.
On the chip
Int8 SIMD kernels for the ESP32-P4 (PIE) and the ESP32-S3 (128-bit SIMD), work split over both cores with FreeRTOS. Chip audio matches the reference C runtime on a PC.
Models
A VITS teacher (~24M params) is distilled into 0.8–1.7M-param students: duration predictor, acoustic model with pitch and energy, Vocos-style ConvNeXt decoder with an iSTFT head.
Front-ends
One per language, in Python for training and in C for the device, tested to give identical token ids. Vietnamese: spelling-based G2P with 6 tones. English: CMUdict plus letter-to-sound rules learned from it.
Data
The Vietnamese prototype uses a public single-speaker corpus. Bilingual voices use about 98 h each of speech synthesized with VoxCPM2 (Apache-2.0) from designed voices.
Tooling
PyTorch for training, ESP-IDF v6.1 for the firmware, PhoWhisper and Whisper for intelligibility, UTMOS and DNSMOS for naturalness and cleanliness.
Get started
gh release download v0.3.2 -R tuanpmt/vtts -p '*.vtts' -D models cd firmware/esp32 BOARD=esp32-p4-nano ./build.sh ../../models/vtts-vi-s-v0.1.q8.vtts \ ../../models/vtts-vi-m-v0.2.q8.vtts -p PORT flash monitor
| BOARD= | Board | Speaker path | Default volume |
|---|---|---|---|
esp32-p4-86-panel-eth-2ro | ESP32-P4-86-Panel-ETH-2RO | ES8311 | vol 70, gain 0 |
esp32-p4-nano | ESP32-P4-NANO | ES8311 | vol 90, gain 12 |
esp32-s3-touch-lcd-2.8 | ESP32-S3-Touch-LCD-2.8 | PCM5101 | vol 80, gain 6 |
echoear | EchoEar 2.0 (ESP32-S3) | ES8311 | vol 75, gain 6 |
Type a sentence on the serial console and the board speaks it. Full instructions are in the README.