infinity

Executes approximate nearest neighbor (ANN) search on dense vector embeddings using HNSW (Hierarchical Navigable Small World) indexing, enabling sub-millisecond retrieval of semantically similar vectors from billion-scale datasets. The system maintains hierarchical graph structures with configurable layer counts and connection parameters, supporting both L2 and cosine distance metrics with SIMD-optimized distance computation.

Executes BM25-based full-text search on sparse vector representations of documents, tokenizing text into terms, computing TF-IDF weights, and ranking results by relevance using the Okapi BM25 probabilistic model. The system maintains inverted indices mapping terms to document IDs with frequency statistics, enabling fast boolean and ranked retrieval without dense embeddings.

Supports bulk import of vectors and metadata from CSV, Parquet, or JSON files, with automatic schema inference and parallel loading across multiple threads. Export functionality writes query results to files in same formats; import uses buffered writes and batch index updates to minimize latency and memory overhead.

Creates and manages indices on vector and metadata columns, supporting HNSW indices for dense vectors, inverted indices for full-text search, and B-tree indices for metadata filtering. Index creation is asynchronous and can be cancelled; index statistics are maintained for query optimization and can be manually refreshed.

Creates point-in-time snapshots of the entire database including vectors, metadata, and indices, enabling recovery to previous states or migration to other systems. Snapshots are incremental and can be stored locally or on remote storage; recovery is atomic and validates data integrity before committing.

Optimizes query execution plans using cost-based optimization that estimates operation costs (I/O, CPU, memory) and selects lowest-cost plan. The optimizer considers index availability, data statistics, and filter selectivity to decide between sequential scan, index scan, and hybrid search paths; execution uses pipelined operators for memory efficiency.

Executes search over multi-vector (tensor) representations where each document contains multiple embedding vectors (e.g., different model outputs or chunked representations), aggregating relevance scores across vectors using configurable fusion strategies (max, mean, weighted sum). The system stores tensors as columnar data structures and applies ANN search independently per vector dimension before combining results.

Combines dense vector search, sparse vector (BM25) search, and full-text search in a single query, executing each search path independently and fusing results using configurable strategies (weighted sum, RRF, learned fusion). The query planner routes subqueries to appropriate indices and merges ranked lists while maintaining result deduplication and score normalization across heterogeneous search types.

Provides SQL query interface extended with vector-specific functions (KNN, MATCH, FUSION) that compile to optimized query execution plans. The SQL parser (built with C++20 modules) handles standard DDL/DML plus vector operations, the query planner applies cost-based optimization for index selection, and the executor dispatches to specialized vector operators (HNSW traversal, BM25 ranking, fusion).

Applies metadata filters (WHERE clauses on non-vector columns) during or after vector search, supporting range queries, equality checks, and boolean combinations on structured fields. The query executor can push filters into index traversal (early termination) or apply post-search filtering depending on selectivity; metadata is stored alongside vectors in columnar format for cache-efficient access.

Provides ACID transaction semantics for vector and metadata operations using Write-Ahead Logging (WAL) for durability and Multi-Version Concurrency Control (MVCC) for isolation. Each transaction maintains a consistent snapshot of the database; writes are logged before applying to in-memory structures, enabling recovery from crashes and concurrent reads during writes without blocking.

Deploys Infinity across multiple nodes with automatic data replication, peer-to-peer synchronization via Thrift RPC, and cluster management for failover and load balancing. The ClusterManager coordinates node roles (leader, replica), distributes data shards across nodes, and handles peer communication for consistency; replication is asynchronous with configurable consistency levels.

Provides Python SDK for programmatic access to Infinity via async/await interface, supporting connection pooling, batch operations, and automatic retry logic. The SDK wraps Thrift RPC calls and HTTP API endpoints, offering high-level abstractions for table creation, vector insertion, and hybrid search while handling serialization and error handling transparently.

Exposes HTTP REST API for vector insertion, search, and metadata management, accepting JSON payloads and returning JSON responses. The HTTP server (built with C++20 async networking) handles request parsing, routes to query executor, and serializes results; supports both synchronous and streaming responses for large result sets.