Mel Spectrogram Generation From Discrete Audio Tokens

A generative speech model for daily dialogue.

Unique: Uses a DVAE (Discrete Variational Autoencoder) rather than a simple lookup table or continuous decoder, enabling learned, high-quality reconstruction of spectrograms from discrete tokens. The DVAE is trained end-to-end with the audio codec, ensuring that discrete tokens capture all information needed for high-fidelity spectrogram reconstruction.

vs others: More flexible than fixed codebooks because the DVAE decoder learns to reconstruct spectrograms from tokens, enabling better quality and smoother transitions between tokens. More efficient than storing spectrograms directly because discrete tokens are more compact and enable better generalization across speakers and content.

via “mel-spectrogram to waveform vocoding”

text-to-speech model by undefined. 4,69,583 downloads.

Unique: Uses a non-autoregressive vocoder (likely HiFi-GAN variant) that generates entire waveforms in a single forward pass, achieving 50-100x speedup compared to autoregressive alternatives like WaveNet. The vocoder is optimized for MLX inference, leveraging GPU acceleration to produce 22050 Hz audio at real-time or faster-than-real-time speeds.

vs others: Faster than WaveGlow or WaveNet vocoders while maintaining comparable audio quality; more efficient than traditional signal processing vocoders (WORLD, STRAIGHT) because neural vocoding requires no explicit pitch extraction or spectral envelope modeling.