Cross Lingual Speech Synthesis With Multilingual Speaker Adaptation

via “cross-lingual-transfer-and-zero-shot-translation”

automatic-speech-recognition model by undefined. 48,72,389 downloads.

Unique: Performs zero-shot translation directly within the speech recognition pipeline by using language tokens to specify target language, eliminating the need for separate translation models. Leverages shared multilingual encoder representations to enable translation to languages not explicitly trained on.

vs others: Simpler than cascading transcription + translation because it uses a single model; however, lower quality than dedicated translation models (2-5% BLEU degradation) and more prone to hallucination because translation is performed on transcribed text rather than acoustic features.