memvid

Memvid packages all agent memory—embeddings, search indexes, metadata, and multi-modal content—into a single immutable .mv2 file format with embedded write-ahead logging (WAL) for crash safety. Smart Frames are append-only memory units that are never modified, only added, ensuring durability and portability without external databases. The .mv2 file contains a table-of-contents (TOC), indexed search structures, and a WAL for recovery, enabling agents to carry their entire memory context as a single portable artifact.

Memvid implements unified semantic search across text, images, audio, and video by storing embeddings in a single index structure within the .mv2 file. The system supports pluggable embedding models (via feature flags like 'vec') and uses FAISS-compatible indexing for fast approximate nearest-neighbor retrieval. All modalities are embedded into a shared vector space, enabling cross-modal queries where a text query can retrieve relevant images or video frames, and vice versa.

Memvid includes a doctor utility that scans .mv2 files for corruption, inconsistencies, or incomplete transactions. The repair system can fix detected issues by rebuilding indexes, recovering orphaned Smart Frames, or truncating corrupted sections. The doctor operates offline (without requiring a running agent) and provides detailed diagnostics of file health and recovery options.

Memvid's parallel ingestion system processes multiple documents concurrently using a builder pattern. The builder accepts documents, extracts content in parallel, generates embeddings asynchronously, and batches Smart Frame commits to the .mv2 file. This design decouples I/O (document reading), CPU (embedding generation), and disk (frame writing) operations, maximizing throughput for large-scale ingestion. Errors in individual documents do not block the batch; failed documents are logged and skipped.

Memvid supports pluggable embedding models through a provider abstraction layer. Developers can use local embedding models (via ONNX or similar), cloud providers (OpenAI, Anthropic, Hugging Face), or custom models. The system caches embeddings in the .mv2 file to avoid recomputation and supports batch embedding generation for efficiency. Embedding model selection is configurable per ingestion operation, allowing different models for different content types.

Memvid provides full-text search via an inverted index (enabled with the 'lex' feature flag) that tokenizes and indexes text content within Smart Frames. The lexical index is stored alongside vector indexes in the .mv2 file and supports boolean queries, phrase matching, and term frequency-based ranking. This complements semantic search for exact-match and keyword-based retrieval scenarios where lexical precision is required.

Memvid ingests diverse content types (PDFs, images, audio, video) through pluggable document readers and multi-modal processors. PDFs are extracted via the 'pdf_extract' feature, images are processed with OpenCV, audio is transcribed via Whisper integration, and video is decomposed into frames. The parallel ingestion and builder system processes content concurrently, extracting text, generating embeddings, and creating Smart Frames that are atomically committed to the .mv2 file.

Memvid's RAG (Retrieval-Augmented Generation) system retrieves relevant Smart Frames based on a query, constructs a context window, and passes it to an LLM for generation. The 'ask' operation combines semantic search, optional lexical filtering, and context ranking to surface the most relevant memories. The system supports configurable context window sizes, ranking strategies, and LLM provider integration (OpenAI, Anthropic, etc.) via standard function-calling APIs.

Memvid supports optional encryption of sensitive data within the .mv2 file using industry-standard cryptographic algorithms. Encryption is applied at the Smart Frame level, allowing selective encryption of sensitive memories while keeping others in plaintext. The system manages encryption keys and provides secure serialization/deserialization of encrypted frames without exposing plaintext to the application layer.

Memvid embeds a write-ahead log (WAL) within the .mv2 file to ensure crash safety and durability. All writes (adding Smart Frames, updating indexes) are logged before being applied to the main data structures. In case of process termination or system failure, the WAL is replayed on next open to recover uncommitted transactions and restore the memory to a consistent state. The doctor and repair system can detect and fix corrupted indexes or incomplete transactions.

Memvid provides native SDKs for Rust, Node.js, and Python, all operating on the same .mv2 file format. The Rust core is the canonical implementation; Node.js and Python SDKs are wrappers around the compiled Rust library. This ensures consistency across languages: a .mv2 file created in Python can be read and modified in Node.js or Rust without format conversion. The CLI provides a command-line interface for shell scripts and automation.

Memvid uses Rust feature flags to enable optional capabilities (vec for vector search, lex for full-text search, pdf_extract for PDF processing, clip for image embeddings, whisper for audio transcription). Features are enabled at compile time, reducing binary size and dependencies for users who don't need all capabilities. Pre-built binaries (npm, PyPI, Docker) include commonly-used features; custom builds can select specific features.

Memvid provides a Docker image (memvid/cli) that packages the Memvid CLI with all dependencies, enabling containerized execution without local installation. The Docker image supports volume mounting for .mv2 files and environment variable configuration for API keys and settings. The image is built with multi-stage compilation to minimize size and includes health checks for container orchestration.