chroma

Chroma provides a unified API across three deployment modes (embedded SQLite, single-node FastAPI server, and Kubernetes-distributed) using a client factory pattern that abstracts underlying storage and compute layers. The architecture uses a Rust frontend service for performance-critical operations and Python FastAPI for HTTP access, with a gRPC-based log service for distributed coordination. This allows developers to start with in-process SQLite and scale to multi-node clusters without changing application code.

Chroma implements approximate nearest neighbor search using Hierarchical Navigable Small World (HNSW) graphs built in Rust, with a query execution pipeline that fetches candidate records from the log service, applies metadata filters via a query expression system, and ranks results by cosine/L2 distance. The knn_hnsw operator in the worker service performs graph traversal with configurable ef (exploration factor) parameters for accuracy/latency trade-offs. Results are merged across multiple segments and returned with similarity scores.

Chroma uses a system database (SysDB) to store metadata about collections, tenants, databases, and version history. The SysDB supports two backends: SQLite for embedded/single-node deployments and PostgreSQL for distributed Kubernetes deployments. The SysDB schema tracks collection ownership, segment references, version pointers, and compaction state. In distributed mode, a Go coordinator service manages SysDB access and ensures consistency across worker nodes. The SysDB is queried during collection creation, deletion, and version management operations.

Chroma's compaction service (rust/worker/src/compactor/) periodically consolidates log entries into immutable Arrow-formatted segments and constructs HNSW indices for efficient similarity search. The compaction workflow is triggered when log size exceeds a threshold or on a schedule, and it merges multiple segments into a single larger segment while deduplicating records and removing deleted entries. HNSW index construction is single-threaded and CPU-intensive, taking O(n log n) time for n vectors. The garbage collection service removes unreferenced segments and log entries after compaction completes. Compaction is asynchronous and may cause temporary query latency spikes.

Chroma supports Kubernetes deployment via Helm charts and Docker images, with separate services for frontend (gRPC), worker (query execution), and log service (write coordination). The deployment uses a PostgreSQL SysDB for metadata consistency, a shared blockstore (S3) for segment storage, and a log service for write ordering. Kubernetes manifests define resource requests/limits, health checks, and service discovery, enabling automatic scaling via Horizontal Pod Autoscaler (HPA). The architecture is stateless at the frontend/worker level, allowing pods to be added/removed without data loss.

Chroma implements a query expression system (where clauses) that supports logical operators ($and, $or, $not) and comparison operators ($eq, $ne, $gt, $gte, $lt, $lte, $in) on typed metadata fields (string, int, float, bool). The system validates filter expressions against collection schemas defined at creation time, catching type mismatches before query execution. Filters are compiled into predicates evaluated during the query execution pipeline, applied after kNN retrieval but before result ranking.

Chroma supports creating isolated collections within a database, each with independent schemas, embeddings, and metadata. Collections are versioned using a segment-based architecture where each write operation creates a new log entry, and compaction consolidates segments into immutable snapshots. The system supports collection forking (creating a copy at a specific version) without duplicating underlying data through copy-on-write semantics. The SysDB (system database) tracks collection metadata, ownership, and version history using SQLite (embedded) or PostgreSQL (distributed).

Chroma implements a write-ahead log (WAL) architecture where add/update/delete operations are appended to an immutable log service (gRPC-based in distributed mode, in-memory in embedded mode) before being applied to the in-memory index. A background compaction service periodically consolidates log entries into immutable Arrow-formatted segments stored in the blockstore (S3 or local filesystem). This design decouples write latency from indexing latency and enables efficient batch operations. The log service guarantees ordering and durability, while the compaction workflow handles segment merging and HNSW index construction.

Chroma's query execution pipeline (in rust/worker/src/execution/operators/) processes queries across multiple segments in parallel, with each segment performing independent kNN search using its HNSW index. A knn_merge operator combines results from all segments using a priority queue, deduplicating results and ranking by similarity score. The execution is orchestrated by a DAG-based operator system where fetch_log retrieves candidate records, knn_hnsw performs similarity search, and metadata filters are applied before final result ranking. This architecture enables horizontal scaling by adding more segments without query latency degradation.

Chroma abstracts storage through a blockstore interface supporting both S3 (with admission control for rate limiting) and local filesystem backends. Compacted segments are serialized to Arrow format and stored as immutable blocks, with a block cache layer providing in-memory caching of frequently accessed blocks. The storage layer is decoupled from the query execution layer, allowing segments to be fetched on-demand from S3 without loading entire collections into memory. The admission control mechanism prevents S3 request throttling by queuing requests and enforcing rate limits.

Chroma provides language-specific client libraries (Python and JavaScript) that abstract the underlying deployment mode (embedded, HTTP server, or Rust service). The Python client uses a factory pattern (chromadb.Client()) to instantiate the appropriate backend based on configuration, supporting both synchronous (blocking) and asynchronous (async/await) APIs. The JavaScript client provides similar abstractions for Node.js and browser environments. Both clients handle serialization/deserialization of embeddings, metadata, and query results, and provide type hints (Python) or TypeScript definitions (JavaScript) for IDE support.

Chroma's FastAPI server layer implements authentication via API keys and rate limiting via token bucket algorithms to enforce per-user/per-tenant quotas. The authentication middleware validates API keys against a configured key store (in-memory or external), and the rate limiter tracks request counts per key and enforces configurable limits (requests per second, queries per minute, etc.). These mechanisms are applied at the HTTP layer before requests reach the core query execution pipeline, protecting against abuse and ensuring fair resource allocation in shared deployments.

Chroma provides a Rust frontend service (rust/frontend/src/) that implements the core Chroma API via gRPC, offering lower latency and higher throughput than the Python FastAPI server. The Rust frontend handles collection management, query routing, and result serialization, delegating compute-intensive operations (kNN search, compaction) to worker services. The gRPC API uses Protocol Buffers for efficient serialization and supports streaming responses for large result sets. This service is the default backend for distributed Kubernetes deployments and can be used directly by Rust clients or via gRPC proxies.