Audiocaps Based Audio Synthesis Training

via “sound generation and audio synthesis from prompts”

AI image upscaler that hallucinates detail guided by text prompts.

Unique: Offers prompt-based sound generation integrated into a creative platform, rather than standalone audio synthesis tools. The approach allows fast sound effect creation but sacrifices control and precision.

vs others: Faster than searching and licensing stock audio; comparable to dedicated audio synthesis tools but integrated into a broader creative suite.

via “audiocaps-based audio synthesis training”

* ⭐ 03/2023: [Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages (USM)](https://arxiv.org/abs/2303.01037)

Unique: Achieves state-of-the-art text-to-audio synthesis with single-GPU training on AudioCaps by operating in CLAP embedding latent space, avoiding the multi-GPU requirements of prior TTA systems that train in raw audio space

vs others: Requires significantly less computational resources than prior text-to-audio systems (single GPU vs. multi-GPU) while achieving better quality by leveraging pretrained CLAP embeddings and operating in latent space rather than raw audio