letta

Letta manages agent instantiation, configuration, and lifecycle through a structured system that persists agent state across sessions via memory blocks (persona, human info, custom context). The Agent Lifecycle and Management subsystem handles agent creation, updates, and deletion while maintaining referential integrity with associated conversations and memory blocks. Unlike stateless chatbots, agents retain structured context that survives server restarts through ORM-backed database persistence.

Letta abstracts multiple LLM providers (OpenAI, Anthropic, Google Gemini, Ollama, and 10+ others) through a unified LLM Client Architecture that handles provider-specific message format transformations, model configuration, and error handling. The Provider System maps agent requests to provider-specific APIs while normalizing responses into a consistent schema. Message Format Transformation pipelines convert between Letta's internal message representation and each provider's native format (e.g., OpenAI's function_call vs Anthropic's tool_use).

Letta's Voice Agents subsystem enables agents to process audio input and generate audio responses, supporting real-time voice conversations. The system integrates speech-to-text (STT) and text-to-speech (TTS) providers, handling audio encoding/decoding and streaming. Voice agents maintain the same memory and tool capabilities as text agents, enabling voice-based access to all agent features. This enables use cases like voice assistants, phone-based customer support, and hands-free interaction.

Letta's Python SDK provides a type-safe client library for programmatic agent management and interaction. The SDK uses Pydantic models for request/response validation, enabling IDE autocomplete and type checking. The Client Libraries subsystem abstracts REST API calls and provides Pythonic interfaces for common operations (create agent, send message, update memory). The SDK supports both synchronous and asynchronous execution, enabling integration into async applications and frameworks.

Letta's Agent Import and Export subsystem enables agents to be exported as configuration files (JSON/YAML) and imported into other Letta instances. This enables version control of agent definitions, sharing agents across teams, and migrating agents between environments. The export includes agent configuration, memory blocks, and tool definitions, but not conversation history. Agents can be exported at any point in their lifecycle and imported with the same configuration, enabling reproducible agent deployments.

Letta's Multi-Tenancy and Security subsystem enables multiple organizations or users to share a single Letta instance with isolated data and access controls. The system implements role-based access control (RBAC) with roles (admin, agent_creator, user) and permissions (create_agent, read_agent, update_agent, delete_agent). Database-level isolation ensures tenants cannot access each other's agents, conversations, or memory. Authentication is handled via API keys or OAuth, with token-based authorization for REST API calls.

Letta's Observability subsystem provides comprehensive telemetry, logging, and error tracking for monitoring agent behavior and debugging issues. Telemetry and Monitoring collects metrics (token usage, latency, error rates) and exports them to monitoring systems (Prometheus, DataDog). Logging and Error Tracking captures detailed logs of agent execution, LLM calls, and tool execution with configurable log levels. The system integrates with error tracking services (Sentry) for automatic error reporting and alerting.

Letta's Memory System provides structured memory blocks (persona, human info, custom context) that agents can read and modify during conversations. The Memory Block Management subsystem stores blocks as ORM entities with optional git-backed versioning, enabling agents to track memory changes over time and revert to previous states. Agents access memory through core memory tools (read_memory, write_memory) that integrate with the message execution pipeline, allowing LLMs to explicitly modify their own context.

Letta's Tool System enables agents to execute custom Python tools with sandboxed execution environments and integrates with Model Context Protocol (MCP) for standardized tool definitions. The Tool Management subsystem registers tools, validates schemas, and enforces Tool Rules (execution constraints, rate limits, access controls). Tool Execution and Sandboxing handles function invocation with error isolation, preventing tool failures from crashing the agent. MCP Integration allows agents to discover and use tools defined via the MCP standard, enabling interoperability with external tool ecosystems.

Letta's Archival Memory and Passages subsystem enables agents to store and search over large document collections using semantic search. The File Processing Pipeline handles OCR, chunking, and embedding generation for documents and codebases. Vector Database Integration (Qdrant, Pinecone, or in-memory) stores embeddings and enables similarity search. Agents can retrieve relevant passages from archival memory during conversations, enabling RAG-style knowledge augmentation without loading entire documents into context.

Letta's Conversation History and Search subsystem persists all agent-user interactions in a structured message format with full-text and semantic search capabilities. The Message System stores messages with metadata (timestamp, sender, message type) in the ORM database. Message Persistence and Retrieval enables agents to access conversation history for context, while Message Conversion Pipeline normalizes messages between internal representation and provider-specific formats. Agents can search conversation history to find relevant past interactions without loading entire conversations into context.

Letta's Context Window Management and Summarization subsystem automatically manages token limits by summarizing conversation history when agents approach context window limits. The system monitors token usage across messages, memory blocks, and tool schemas, and triggers LLM-based summarization to compress conversation history while preserving key information. This enables agents to maintain long conversations without manual context management or conversation truncation.

Letta's REST API Structure provides full-featured endpoints for agent management, messaging, and streaming. The Streaming Architecture enables real-time message streaming using Server-Sent Events (SSE) or WebSockets, allowing clients to receive agent responses as they are generated. Job and Run Management handles asynchronous task execution with background job queues, enabling long-running operations (batch processing, file indexing) without blocking API responses. The SyncServer and Service Layer abstracts database operations and provides a consistent interface for both REST API and Python SDK clients.

Letta's Multi-Agent Systems and Groups subsystem enables coordination of multiple agents with different roles and capabilities. Agents can be organized into groups with defined coordination patterns (e.g., sequential, parallel, hierarchical). The system manages message routing between agents, enables inter-agent communication, and provides mechanisms for agents to delegate tasks to specialized agents. This enables complex workflows where different agents handle different aspects of a problem.

Letta's Batch Processing subsystem enables agents to process large datasets asynchronously, with results stored for later retrieval. Human-in-the-Loop Workflows allow agents to pause execution and request human feedback before proceeding, enabling collaborative AI systems where humans and agents work together. The Job and Run Lifecycle manages batch job execution, tracking progress and handling failures. This enables use cases like document processing, data labeling, and decision workflows that require human oversight.