Mcptube – Karpathy's LLM Wiki idea applied to YouTube videos

Automatically downloads and extracts transcripts from YouTube videos using the YouTube API or subtitle parsing, then indexes the raw transcript text into a searchable format. The system handles both auto-generated and manually-created captions, normalizing timestamps and speaker information for downstream processing. This enables full-text search and semantic retrieval across video content without requiring manual transcription.

Implements vector-based semantic search by embedding transcript segments using an LLM embedding model (likely OpenAI embeddings or local alternatives), storing embeddings in a vector database, and retrieving contextually relevant transcript chunks based on natural language queries. The system ranks results by semantic similarity rather than keyword matching, allowing users to find content by meaning even when exact terminology differs.

Chains semantic search with an LLM to answer user questions by retrieving relevant transcript segments and generating answers grounded in video content. The system uses retrieved transcript chunks as context (RAG pattern), ensuring answers cite specific videos and timestamps. This enables conversational interaction with video libraries where the LLM synthesizes information across multiple videos while maintaining source attribution.

Enables the LLM to retrieve and synthesize information from multiple videos simultaneously, identifying connections and relationships across content. The system retrieves relevant segments from different videos for a single query, allowing the LLM to generate comprehensive answers that integrate insights from multiple sources. This is implemented via batch semantic search across the entire corpus followed by LLM synthesis, with explicit tracking of which videos contributed to each answer.

Provides command-line interface for bulk operations on video collections: downloading transcripts from multiple YouTube URLs, building indexes, updating embeddings, and managing the vector database. The CLI abstracts away API complexity and enables scripting for automated workflows like scheduled re-indexing of channel uploads or batch processing of video playlists. Supports configuration files for managing API credentials and indexing parameters.

Exposes video search and QA capabilities as MCP tools that LLMs can invoke directly, enabling seamless integration with LLM agents and multi-tool workflows. The system implements MCP server endpoints for semantic search, QA, and transcript retrieval, allowing Claude, GPT-4, or other MCP-compatible LLMs to query video content as part of broader reasoning tasks. This enables agents to autonomously decide when to consult video knowledge bases during multi-step problem solving.

Intelligently chunks transcripts into segments that preserve semantic boundaries (sentence or paragraph breaks) while maintaining timestamp alignment, enabling precise retrieval and citation of specific video moments. The system implements sliding-window chunking with overlap to ensure context is preserved across chunk boundaries, and tracks start/end timestamps for each chunk. This enables answers to cite exact video timestamps (e.g., 'at 12:34 in the video') rather than approximate locations.

Handles transcripts in multiple languages by detecting language, optionally translating to a common language (English), and enabling search across multilingual content. The system uses language detection models and translation APIs (Google Translate, DeepL, or local models) to normalize transcripts, then embeds translated content for unified semantic search. This enables users to search in one language and retrieve results from videos in other languages.