Verbaly

Processes live audio input during user speech to extract and measure acoustic features including speech rate (words per minute), pause duration, filler word frequency (um, uh, like), and clarity markers. Uses signal processing pipelines to detect prosodic patterns and phonetic clarity in real-time, likely leveraging WebRTC for browser-based audio capture and streaming to backend speech analysis models that compute metrics against configurable thresholds for immediate feedback delivery.

Implements a multi-turn dialogue system where the AI takes on specific conversation roles (interviewer, audience member, client, etc.) and responds contextually to user speech input, creating realistic practice scenarios without requiring human partners. The system likely uses a large language model (GPT-based or similar) with prompt engineering to maintain character consistency, respond to speech content (transcribed via speech-to-text), and generate follow-up questions or objections that simulate real conversation dynamics.

Converts user audio input into text transcripts in real-time or post-recording, likely using a speech-to-text engine (Whisper, Google Cloud Speech-to-Text, or Azure Speech Services) with speaker segmentation to distinguish between user speech and any background audio. The transcription is timestamped and formatted to enable downstream analysis, feedback generation, and user review of what was actually said versus intended.

Synthesizes real-time metrics (speech rate, filler words, clarity) and conversation context into natural language feedback and specific, actionable recommendations. Uses rule-based logic and/or LLM-based generation to translate raw metrics into coaching advice (e.g., 'You used 12 filler words in 3 minutes — try pausing instead of saying um' or 'Your pace was 180 WPM, which is 20% faster than recommended for presentations — slow down by 10-15%'). Feedback is delivered immediately after speech or at session end.

Records user audio during practice sessions and stores it with associated metadata (metrics, timestamps, transcript). Enables playback of the recording with real-time metric visualization overlaid on the timeline (e.g., visual indicators of filler words, pace changes, clarity dips at specific timestamps). Users can scrub through the recording, see exactly when they used a filler word or spoke too fast, and correlate audio with metrics for self-directed learning.

Maintains a persistent record of user practice sessions over time, storing metrics, transcripts, and feedback for each session. Enables users to view trends (e.g., 'Your average filler word count has decreased from 15 to 8 over the last 10 sessions') and compare specific metrics across sessions to visualize improvement. Likely uses a user database with session indexing and basic analytics (average, trend, percentile) to surface progress without requiring manual analysis.

Provides pre-built practice scenarios (job interview, sales pitch, presentation, negotiation, etc.) that configure the AI conversation partner's role, expected questions, and difficulty level. Users select a scenario, optionally customize context (industry, role, audience type), and the system initializes the AI with appropriate prompts and constraints. This reduces setup friction and ensures users practice realistic, relevant conversations rather than generic dialogue.

Implements core speech analysis (filler word detection, pace calculation, clarity metrics) using client-side JavaScript libraries and WebRTC audio processing, reducing latency and server load. While some features (LLM-based feedback, STT) likely require cloud APIs, the real-time metric computation happens in-browser, enabling low-latency feedback even with network delays. This architecture choice prioritizes responsiveness and user privacy (audio processing happens locally before transmission).