CS224S: Spoken Language Processing - Stanford University

Teaches students to analyze speech signals using spectrograms, formant tracking, and pitch extraction through hands-on assignments. The course covers signal processing fundamentals including Fourier analysis, windowing techniques, and feature extraction methods that form the foundation for understanding how acoustic properties map to linguistic units. Students work with real speech data to identify phonetic distinctions through acoustic measurements.

Covers the complete pipeline of automatic speech recognition (ASR) systems including acoustic modeling, language modeling, and decoding strategies. The course teaches how to design and evaluate ASR systems, including the role of hidden Markov models (HMMs), neural acoustic models, and n-gram or neural language models. Students learn both classical GMM-HMM architectures and modern end-to-end approaches like attention-based sequence-to-sequence models.

Covers the extraction and modeling of emotional and sentiment information from speech, including acoustic feature analysis, emotion classification, and emotion prediction. The course teaches how prosodic, spectral, and voice quality features correlate with emotional states. Students learn both rule-based emotion detection and neural approaches for emotion classification from speech.

Covers the design, collection, and annotation of speech corpora for research and system development. The course teaches annotation schemes for phonetic, prosodic, and semantic information, quality control procedures, and best practices for corpus documentation. Students learn how to design corpora that are representative, well-annotated, and suitable for training and evaluating speech systems.

Covers techniques for transforming speech from one speaker to another (voice conversion) and adapting acoustic models to new speakers with limited data. The course teaches feature mapping approaches, neural voice conversion models, and speaker adaptation techniques for ASR. Students learn how to handle speaker variability while preserving linguistic content.

Teaches the design and implementation of language models (LMs) specifically for speech recognition and spoken language understanding tasks. The course covers n-gram models, neural language models (RNNs, Transformers), and their integration into ASR decoding. Students learn how LM probability estimates constrain the acoustic decoder's search space and how to evaluate LM quality using perplexity and downstream ASR metrics.

Teaches methods for extracting meaning from spoken input, including intent detection, slot filling, and semantic frame parsing. The course covers how to map spoken utterances to structured semantic representations (e.g., dialogue acts, semantic frames) using both rule-based and neural approaches. Students learn to handle speech-specific challenges like disfluencies, repairs, and acoustic ambiguities in semantic understanding.

Covers the architecture and implementation of dialogue systems that interact through spoken language, including dialogue state tracking, dialogue management, and response generation. The course teaches how to design dialogue flows, manage conversation context, and integrate ASR, NLU, and natural language generation (NLG) components. Students learn both task-oriented dialogue (slot-filling) and more open-ended conversational approaches.

Covers the design and implementation of text-to-speech systems that convert written text to natural-sounding speech. The course teaches both classical concatenative synthesis (unit selection) and modern neural approaches (Tacotron, WaveNet, FastSpeech). Students learn how to handle linguistic analysis (text normalization, phoneme conversion, prosody prediction) and acoustic synthesis, including the role of vocoders in converting acoustic features to waveforms.

Teaches the analysis and modeling of prosodic features (pitch, duration, intensity) in speech, including their role in phonology, pragmatics, and emotion expression. The course covers prosody extraction methods, prosodic annotation schemes, and neural models for prosody prediction. Students learn how prosodic variation conveys linguistic information (stress, intonation) and paralinguistic information (emotion, attitude).

Covers speaker identification and verification systems that authenticate or identify speakers based on voice characteristics. The course teaches feature extraction for speaker recognition (i-vectors, x-vectors, speaker embeddings), model training approaches, and scoring methods. Students learn how to handle speaker variability due to channel effects, background noise, and linguistic content variation.

Covers the challenges and techniques for processing speech in multiple languages and code-switching contexts (where speakers mix languages within utterances). The course teaches language identification, multilingual acoustic and language modeling, and handling of language-specific phonetic and prosodic phenomena. Students learn how to build systems that gracefully handle language mixing and switching.

Covers techniques for making speech processing systems robust to background noise, reverberation, and other acoustic degradations. The course teaches speech enhancement, robust feature extraction, and model adaptation techniques. Students learn how to handle real-world acoustic conditions where speech is corrupted by environmental noise, room acoustics, and microphone variability.