Distributed Indexing Pipeline With Compression

via “composite-index chaining with automatic routing”

A library for efficient similarity search and clustering of dense vectors.

Unique: Provides pre-built composite index classes (IndexIVFPQ, IndexHNSWPQ) that automatically handle parameter passing and result routing between stages, eliminating manual pipeline orchestration. Enables composition of any two index types via the IndexPreTransform API for custom pipelines.

vs others: More convenient than manually chaining indices because parameter tuning and result routing are handled automatically; more flexible than single-index approaches because it enables joint optimization of filtering and refinement stages.

Efficient and Effective Passage Search via Contextualized Late Interaction over BERT

Unique: Implements a streaming compression pipeline that encodes and compresses documents in a single pass without materializing full-precision embeddings to disk, using CUDA-accelerated compression kernels integrated directly into the indexing loop

vs others: Achieves 10-100x faster indexing than naive approaches by parallelizing encoding across GPUs and compressing on-the-fly, compared to Elasticsearch/Lucene which require separate encoding and indexing phases