LibreChat

LibreChat implements a BaseClient architecture that abstracts away provider-specific API differences (OpenAI, Anthropic, Google Vertex AI, AWS Bedrock, Azure OpenAI, Groq, Mistral, OpenRouter, DeepSeek, local Ollama/LM Studio) behind a single normalized interface. Requests are routed through provider-specific implementations that handle authentication, request formatting, streaming, and response normalization, allowing seamless model switching within the same conversation without client-side logic changes.

LibreChat integrates the @modelcontextprotocol/sdk to connect external tools, data sources, and context providers as MCP servers. The system manages MCP server lifecycle (connection, reconnection with exponential backoff, graceful degradation), exposes MCP resources and tools to the AI model, and handles tool invocation with automatic serialization/deserialization. This enables agents to access real-time data, execute external commands, and interact with third-party systems without hardcoding integrations.

LibreChat includes a token pricing system that tracks API costs for each model and provider. The system maintains a configurable pricing table (tokens per input/output, cost per token) for each model, calculates token usage for each message, and aggregates costs per user or conversation. The pricing configuration is stored in YAML or database, allowing administrators to update rates without code changes. The system supports both OpenAI's token counting library and provider-specific token estimation. Cost data is stored with messages and can be queried for billing or analytics.

LibreChat is structured as a monorepo using Turbo for build orchestration and caching. The codebase is organized into modular packages: @librechat/api (backend), @librechat/client (frontend), @librechat/data-provider (data layer), @librechat/data-schemas (shared types). This architecture enables code sharing, independent package versioning, and efficient builds through Turbo's incremental compilation and caching. Developers can work on individual packages without rebuilding the entire project. The monorepo structure facilitates contribution and maintenance by isolating concerns.

LibreChat provides production-ready Docker images with multi-stage builds (Dockerfile.multi) that minimize image size by separating build and runtime stages. The project includes docker-compose configurations for local development and production deployment. For Kubernetes, Helm charts are provided for declarative deployment with configurable values for replicas, resources, storage, and networking. The deployment system supports environment-based configuration, secrets management, and health checks. This enables both simple Docker Compose deployments and enterprise Kubernetes setups.

LibreChat uses a YAML-based configuration system (librechat.yaml) that allows administrators to configure providers, models, authentication, storage, and features without code changes. The configuration is validated against a JSON schema at startup, catching configuration errors early. Environment variables can override YAML settings, enabling deployment-specific customization. The configuration system supports nested structures for complex settings (e.g., provider-specific options, RAG settings). This enables flexible deployment across different environments without code changes.

LibreChat integrates text-to-speech (TTS) and speech-to-text (STT) capabilities supporting multiple providers (OpenAI, Google, Azure, etc.). Users can listen to AI responses via TTS or provide input via voice. The system handles audio encoding/decoding, streaming, and provider-specific API calls. TTS output can be played in the browser or downloaded. STT input is transcribed and inserted into the chat. This enables multimodal interaction beyond text, improving accessibility and user experience.

LibreChat provides a sandboxed code execution environment supporting Python, Node.js, Go, C/C++, Java, PHP, Rust, and Fortran. Code is executed in isolated containers or processes with resource limits, preventing malicious or runaway code from affecting the host system. The interpreter captures stdout/stderr, execution time, and return values, streaming results back to the chat interface. This enables agents and users to execute code directly within conversations for data analysis, visualization, and prototyping.

LibreChat includes an Agents system that enables users to define AI agents through a no-code UI, specifying system prompts, tool access, model selection, and execution parameters. Agents are persisted in the database and can be shared via a marketplace. The backend implements an agent execution loop that handles tool calling, result interpretation, and multi-step reasoning. Agents can be invoked from conversations or via API, with full message history and state management. The system supports both simple tool-calling agents and complex multi-step reasoning workflows.

LibreChat integrates web search capabilities that perform semantic queries, scrape and parse web content, and rerank results based on relevance to the user's query. The system uses configurable search providers (e.g., SerpAPI, Bing, Google) and implements content extraction to pull relevant text from search results. Results are reranked using embedding-based similarity to the original query, ensuring the most relevant information is prioritized. This enables agents and users to access current information beyond the model's training data cutoff.

LibreChat supports Artifacts — a feature where the AI generates interactive UI components (React, HTML, Mermaid diagrams) that are rendered in a dedicated panel alongside the chat. The system detects when a model response contains artifact markers, extracts the code, and renders it in a sandboxed iframe or React component. Users can edit artifacts, download them, or copy the code. This enables AI to generate interactive visualizations, prototypes, and diagrams without requiring users to copy-paste code into external tools.

LibreChat supports multimodal conversations where users can upload images, PDFs, and other files, and the AI can analyze them. The system handles image encoding (base64 or URL-based), file parsing (PDF text extraction, image OCR), and passes multimodal context to models that support vision (GPT-4V, Claude 3, Gemini Pro Vision, etc.). File uploads are stored in a configurable backend (local filesystem, S3, etc.) and associated with conversations. The vision capability enables use cases like document analysis, image annotation, and visual problem-solving.

LibreChat implements a RAG system that indexes documents into a vector database, performs semantic search on user queries, and augments AI responses with relevant document excerpts. The system supports multiple embedding models (OpenAI, local models via Ollama) and vector stores (Pinecone, Weaviate, Milvus, local SQLite with vector extensions). Documents are chunked, embedded, and stored with metadata. When a user asks a question, the system retrieves semantically similar chunks and passes them as context to the AI model. This enables knowledge base integration and document-grounded responses.

LibreChat implements a comprehensive authentication system supporting multiple protocols: OAuth2 (Google, GitHub, Discord, etc.), OpenID Connect, LDAP (for directory integration), and SAML (for enterprise SSO). The system manages user sessions, API keys, and role-based access control. Authentication is abstracted through a pluggable provider system, allowing organizations to integrate with their existing identity infrastructure. User data is stored in the database with encrypted credentials, and sessions are managed via secure cookies or JWT tokens.

LibreChat stores all conversations in a database with full message history, metadata (timestamps, model used, tokens consumed), and user associations. The system implements full-text search across conversation content, enabling users to find past messages and conversations. Messages can be filtered by date, model, or conversation. The database schema supports efficient querying and indexing. Conversations can be exported, shared, or deleted. This enables users to maintain a searchable archive of their interactions and retrieve context from past conversations.