AssemblyAI API

Converts pre-recorded audio files to text using a single foundational model trained on 12.5M+ hours of audio data, supporting 99 languages with automatic language detection. Processes audio asynchronously via HTTP POST, returning word-level transcripts with optional auto-punctuation and capitalization. The model handles diverse audio conditions and accents without requiring language-specific model selection.

Specialized transcription model optimized for 6 languages (English, Spanish, German, French, Italian, Portuguese) with higher accuracy than Universal-2, trained on domain-specific data. Supports advanced features including keyterms prompting (up to 1000 custom words/phrases) and plain-language prompting (Beta) to inject contextual instructions that control transcription behavior, formatting, and audio event tagging. Pricing includes keyterms prompting at no additional cost.

Analyzes transcript content to detect overall sentiment (positive, negative, neutral) and emotional tone across the conversation. Returns sentiment scores and optional per-segment sentiment breakdown, enabling applications to understand customer satisfaction, agent performance, or conversation dynamics without manual annotation.

Generates abstractive summaries of transcripts using LeMUR (AssemblyAI's LLM integration layer), which routes requests to Claude, GPT-4, or other LLMs. Supports custom summarization instructions and context injection, enabling applications to generate meeting notes, call summaries, or custom extracts without managing separate LLM APIs. Pricing includes LLM inference cost.

Enables arbitrary LLM prompting on transcripts through LeMUR, allowing developers to ask questions, extract information, or perform custom analysis on audio content. Routes prompts to Claude, GPT-4, or other LLMs with transcript context automatically injected, supporting multi-turn conversations and custom instructions without managing separate LLM APIs.

Provides pre-built integrations with LiveKit (real-time communication platform) and Pipecat (voice agent framework) to enable developers to build conversational voice agents. Handles real-time transcription, LLM integration via LeMUR, and text-to-speech synthesis in a unified pipeline, reducing boilerplate for voice agent development.

Exposes AssemblyAI transcription and LeMUR capabilities as a Claude MCP server, enabling Claude to directly analyze audio files and transcripts through MCP protocol. Allows Claude users and applications to transcribe audio, generate summaries, and ask questions about audio content without leaving Claude interface or managing separate API calls.

Returns precise word-level timing information for each word in the transcript, enabling applications to synchronize text with audio playback, highlight words as they're spoken, or extract segments by time range. Timestamps are returned in milliseconds with start and end times per word.

Processes live audio streams with ultra-low latency transcription, delivering partial results as audio arrives and final results on speech completion. Supports Universal-3 Pro Streaming model with auto-punctuation, casing, and next-gen end-of-turn detection to identify when a speaker finishes. Integrates with WebSocket protocol for bidirectional real-time communication, enabling voice agent and live transcription applications.

Automatically detects and segments audio by speaker, labeling distinct speakers as 'Speaker A', 'Speaker B', etc. without requiring pre-labeled speaker data. Works with both pre-recorded and streaming audio, returning speaker labels aligned to word-level timestamps in the transcript. Supports optional speaker identification to map generic labels to actual names or roles (e.g., 'Speaker A' → 'John Smith').

Specialized transcription mode optimized for medical and healthcare conversations, with improved accuracy for medical terminology, drug names, anatomical terms, and clinical abbreviations. Applies domain-specific language models and vocabulary optimization to reduce transcription errors on healthcare-specific content. Available as add-on feature for both Universal-2 and Universal-3 Pro models.

Identifies and extracts named entities from transcripts including person names, company names, email addresses, dates, and locations. Operates on the transcript output, tagging entities with their type and position in the text. Enables downstream applications to automatically extract structured data (contacts, organizations, temporal references) without manual annotation or external NER models.

Automatically identifies and tags filler words (um, uh, er, erm, ah, hmm, mhm, like, you know, I mean) and disfluencies (repetitions, restarts, stutters, informal contractions like gonna, wanna, gotta) in transcripts. Preserves these elements in output with optional tagging, enabling analysis of speech patterns, confidence, or speaker characteristics without manual annotation.

Automatically detects and tags non-speech audio events (background noise, music, silence, beeps, etc.) in transcripts, marking them with labels like [beep], [silence], [music]. Enables applications to distinguish speech from environmental sounds and preserve audio context in transcripts without manual annotation.

Automatically detects and redacts personally identifiable information (PII) in transcripts including names, email addresses, phone numbers, credit card numbers, and other sensitive data. Replaces detected PII with placeholder tokens or masks, enabling HIPAA/GDPR-compliant transcript storage and sharing without manual review.

Analyzes transcript content for policy violations including profanity, hate speech, harassment, and other harmful content. Flags segments with violation types and severity levels, enabling applications to filter, flag, or quarantine transcripts that violate content policies without manual review.