MemGPT

Manages LLM context through a tiered memory system that separates core system context, conversation history, and retrieved memories into distinct layers. The system dynamically prioritizes which memories to include in the context window based on relevance scoring and token budgets, allowing conversations to extend far beyond native LLM context limits by intelligently swapping memories in and out of the active context.

Stores conversation turns and agent state as embeddings in a vector database, enabling semantic similarity search to retrieve relevant past interactions without keyword matching. The system converts conversation messages into dense vector representations and indexes them for fast approximate nearest-neighbor lookup, allowing the agent to find contextually relevant memories even when exact keywords don't match.

Automatically summarizes long conversation segments into condensed summaries that preserve key information while reducing token count, allowing older conversations to be compressed and stored efficiently. The system uses LLM-based summarization to extract important facts, decisions, and context from conversation turns, replacing verbose exchanges with concise summaries that can be retrieved and expanded if needed.

Combines semantic (embedding-based) and keyword-based search to retrieve memories, using a hybrid approach that balances semantic understanding with exact-match precision. The system performs both vector similarity search and BM25/keyword search in parallel, then merges results using configurable weighting to find memories that are either semantically similar or contain relevant keywords.

Maintains a protected core context layer that contains the agent's system prompt, personality definition, and core instructions, ensuring these foundational directives remain stable and prioritized in every LLM call regardless of memory eviction or context assembly decisions. This layer is never evicted and always occupies the first tokens of the context window, preventing the agent from losing its identity or core behavioral constraints.

Provides a unified interface for calling different LLM providers (OpenAI, Anthropic, local Ollama) with automatic request/response translation and provider-specific parameter mapping. The system abstracts away provider differences in API formats, token counting, and response structures, allowing agents to switch backends without code changes while handling provider-specific quirks like different max token limits or function-calling formats.

Automatically segments conversations into discrete turns (user message + agent response pairs) and indexes each turn with metadata including timestamps, speaker roles, and semantic content. The system maintains a structured conversation graph where each turn is a node with relationships to previous turns, enabling efficient traversal and selective retrieval of conversation segments rather than treating history as a flat transcript.

Dynamically assembles the context window by calculating token counts for each memory layer (core context, conversation buffer, retrieved memories) and prioritizing content to fit within a specified token budget. The system uses provider-specific token counters and iteratively adds memories in relevance order until the budget is exhausted, ensuring the context window never exceeds LLM limits while maximizing information density.

Serializes and persists the complete agent state (memory index, conversation history, core context, metadata) to disk or database, enabling agents to be paused, resumed, or migrated across processes without losing context or coherence. The system maintains versioned snapshots of agent state and supports atomic writes to prevent corruption during failures, allowing agents to survive process restarts and be cloned for parallel execution.

Enables agents to call external functions (tools) while maintaining memory context, automatically logging function calls and results back into the memory system. The system translates function definitions into LLM-compatible schemas, executes called functions, and stores both the call and result as memory turns, allowing the agent to learn from tool interactions and reference past tool usage.

Provides pluggable memory eviction strategies that determine which memories are removed when storage limits are reached, supporting policies like least-recently-used (LRU), least-frequently-used (LFU), and custom relevance-based eviction. The system allows developers to define eviction rules based on memory age, access patterns, or semantic importance, enabling fine-grained control over which information is retained versus discarded.

Manages separate memory and context for multiple concurrent users, ensuring conversations remain isolated and user-specific memories don't leak across sessions. The system maintains per-user memory indices, conversation histories, and state, with configurable sharing policies for shared knowledge (e.g., system facts) while keeping personal interactions private.