Meilisearch

Implements keyword search using LMDB-backed inverted indexes (word_docids and word_pair_proximity_docids databases) built during parallel document extraction. The charabia tokenization layer automatically handles typos and misspellings with configurable Levenshtein distance thresholds, enabling users to find documents even with spelling errors. Search queries are parsed and matched against pre-computed word and word-pair proximity structures for sub-50ms response times.

Implements semantic search by storing document embeddings in the arroy vector index (HNSW-based approximate nearest neighbor search). Supports embeddings from OpenAI, HuggingFace, or Ollama via configurable embedding providers. Hybrid search combines keyword and semantic results using a weighted semanticRatio parameter (0.0-1.0) that blends BM25 keyword scores with vector similarity scores, enabling semantic understanding without abandoning keyword precision.

Implements document processing through a parallel extraction architecture in the milli crate that tokenizes, embeds, and indexes documents concurrently. The pipeline processes documents in batches, extracting text fields, generating embeddings (if configured), building inverted indexes, and creating vector indexes in parallel. Parallelization is transparent to the user; document batches are automatically distributed across CPU cores.

Implements data export through dump and export endpoints that serialize the entire index (documents, settings, indexes) to a portable format. Dumps can be imported into another Meilisearch instance, enabling backup, migration, and disaster recovery. Exports are performed asynchronously via the task queue, with progress tracking.

Uses LMDB (Lightning Memory-Mapped Database) as the underlying storage engine for all indexes, providing durability, ACID transactions, and memory-mapped file access. LMDB enables fast random access to index data without loading entire indexes into memory. Storage is organized into multiple databases (word_docids, word_pair_proximity_docids, facet_id_*_docids, vector indexes) for efficient querying.

Implements document ingestion through the IndexScheduler task queue, which batches write operations (document additions, deletions, index creation, settings changes) and processes them asynchronously in the background. The parallel extraction pipeline in the milli crate processes documents through tokenization, embedding generation, and index construction in parallel, with automatic batching for efficiency. Task status is tracked and exposed via the Task Management API, enabling non-blocking document uploads.

Implements filtering through a filter-parser that converts complex filter expressions into a FilterCondition abstract syntax tree (AST). Supports boolean operators (AND, OR, NOT), comparison operators (=, !=, <, >, <=, >=), range queries, and nested conditions. Filters are evaluated during search execution against indexed document fields, enabling precise result narrowing without separate filtering passes.

Implements faceted navigation by pre-computing facet distributions during indexing using facet_id_*_docids databases. When a search is executed, facet counts are computed from the filtered result set without scanning all documents. Supports hierarchical facets and configurable facet ordering (alphabetical, count-based). Facet results are returned alongside search results, enabling drill-down navigation.

Implements location-based search through the _geo attribute, which stores latitude/longitude coordinates for documents. Supports geographic filtering (e.g., 'find restaurants within 5km of coordinates') and sorting by distance. Coordinates are indexed and evaluated during search execution, enabling proximity-based ranking without external geospatial libraries.

Exposes Meilisearch functionality through a RESTful API built on actix-web, with endpoints for document management (POST/PUT/DELETE), search queries (GET /search), index administration, and task tracking. The API is fully specified in OpenAPI format, enabling client code generation and API documentation. All operations return JSON responses with consistent error handling and HTTP status codes.

Implements federated search (multi-search) by executing a single query across multiple indexes and merging results with configurable ranking strategies. Results from different indexes are combined using a weighted scoring approach, enabling cross-index semantic search. The federated search API accepts a list of index names and a shared query, returning merged results ranked by relevance.

Implements task management through the IndexScheduler, which tracks all write operations (document additions, deletions, index creation, settings changes) as tasks with status (enqueued, processing, succeeded, failed). The Task Management API exposes task status, progress, and error details. Webhooks can be configured to notify external systems when tasks complete, enabling event-driven indexing workflows.

Allows configuration of index-level settings (searchable fields, filterable fields, sortable fields, ranking rules, typo tolerance, embedding configuration) through the Settings API. Changes to settings trigger automatic reindexing of all documents in the background via the IndexScheduler. Settings are persisted and applied to all subsequent searches without manual intervention.