txtai

Implements a union of sparse (BM25) and dense (neural embedding) vector indexes within a single Embeddings database, enabling hybrid semantic search that combines lexical and semantic relevance. The architecture supports pluggable ANN backends (Faiss, Annoy, HNSW) for dense vectors and automatically routes queries to both index types, merging results via configurable scoring methods. This design allows semantic search to capture meaning while preserving exact-match precision for technical queries.

Builds and maintains knowledge graphs as part of the embeddings database, allowing entities and relationships to be indexed alongside vector embeddings. The system supports graph traversal operations (neighbor queries, path finding) that integrate with vector search results, enabling multi-hop reasoning and relationship-aware retrieval. Graph networks are persisted in the same storage backend as vectors, providing unified indexing without separate graph database dependencies.

Supports quantization of embedding models and LLMs to reduce memory footprint and inference latency for local deployment. Quantization strategies include INT8, INT4, and bfloat16 precision reduction with minimal accuracy loss. The system automatically applies quantization during model loading and handles quantized model inference transparently, enabling deployment on resource-constrained devices.

Enables horizontal scaling of the embeddings database across multiple machines through document sharding and distributed search. The system partitions documents across cluster nodes based on configurable sharding strategies (hash-based, range-based), routes queries to relevant shards, and aggregates results. Clustering is transparent to the application layer, allowing seamless scaling without code changes.

Provides language bindings beyond Python (Java, JavaScript, Go, etc.) enabling txtai to be used from non-Python applications. Bindings wrap the Python core via language-specific interfaces and handle serialization/deserialization of complex types. This design allows polyglot teams to integrate txtai without Python expertise.

Provides pluggable storage backends (SQLite, PostgreSQL, custom) for persisting embeddings, metadata, and indexes to disk or remote storage. The system supports incremental indexing, checkpoint-based recovery, and backup/restore operations. Storage backends are abstracted, allowing seamless migration between storage systems without data loss.

Embeds a relational database (SQLite by default, extensible to other backends) within the embeddings database to store structured metadata, document content, and query results. The system automatically indexes text columns for full-text search and allows SQL queries to filter vector search results by metadata predicates. This design eliminates the need for a separate metadata store, providing co-located structured and unstructured data indexing.

Provides a unified pipeline framework that abstracts over multiple LLM providers (OpenAI, Anthropic, Ollama, local transformers) through a provider-agnostic interface. Pipelines are defined declaratively (YAML or Python) and support chaining multiple LLM calls, prompt templating, and result post-processing. The architecture uses a plugin pattern where each provider implements a standard interface, allowing seamless switching between models without code changes.

Implements a specialized LLM pipeline that automatically retrieves relevant documents from the embeddings database, injects them as context into LLM prompts, and generates responses grounded in retrieved content. The pipeline supports configurable retrieval strategies (top-k, similarity threshold, metadata filtering), prompt templating with context injection, and result post-processing. This design enables fact-grounded generation without requiring manual context management.

Provides a workflow engine that orchestrates multi-step AI tasks with support for sequential execution, conditional branching, parallel task execution, and scheduled runs. Workflows are defined declaratively in YAML and support task dependencies, error handling, and state persistence. The engine integrates with pipelines, agents, and external tools, enabling complex automation without custom orchestration code.

Implements an agent framework that enables LLMs to autonomously plan and execute multi-step tasks by selecting from a registry of tools (search, SQL queries, external APIs). Agents use chain-of-thought reasoning to decompose problems, call tools with generated parameters, and iterate based on results. The architecture supports agent teams with collaboration patterns and integrates with the embeddings database for knowledge access.

Exposes txtai capabilities through a REST API that mimics OpenAI's API endpoints, enabling drop-in compatibility with OpenAI client libraries. Additionally supports the Model Context Protocol (MCP) for integration with Claude and other MCP-compatible clients. The API layer handles request routing, authentication, clustering support, and automatic serialization of complex types (embeddings, search results).

Provides a configuration-driven application layer where all components (embeddings database, pipelines, workflows, agents) are defined declaratively in YAML files. The Application class reads YAML configuration and automatically instantiates and wires components, eliminating boilerplate initialization code. This design enables reproducible deployments and configuration-as-code patterns.

Extends the pipeline framework to support diverse input modalities beyond text: audio transcription and analysis, image processing and OCR, structured data transformation, and training data pipelines. Each modality has specialized preprocessing, embedding, and post-processing steps. Pipelines automatically route inputs to appropriate modality handlers and can chain multi-modal operations (e.g., transcribe audio → extract entities → search documents).