@nestjs-ai/rag

Provides a pluggable vector store interface that integrates seamlessly with NestJS dependency injection, allowing developers to swap between multiple vector database backends (Pinecone, Weaviate, Milvus, etc.) without changing application code. Uses NestJS providers and modules to manage vector store lifecycle, configuration, and connection pooling within the framework's IoC container.

Orchestrates text-to-embedding conversion through a pluggable provider interface supporting OpenAI, Anthropic, Cohere, HuggingFace, and local models. Handles batching, retry logic, rate limiting, and caching of embeddings within NestJS services, with configurable chunk size and normalization strategies to optimize for different vector store backends.

Splits documents into semantically-aware chunks using configurable strategies (fixed-size, semantic boundaries, recursive splitting) and automatically extracts metadata (source, timestamp, section headers) to attach to vectors. Supports multiple document formats (PDF, Markdown, plain text) with format-specific parsing logic and preserves document structure for context-aware retrieval.

Executes vector similarity search against indexed documents and optionally combines results with keyword/BM25 search to improve recall. Implements ranking strategies (reciprocal rank fusion, score normalization) to merge vector and keyword results, with configurable similarity thresholds and result filtering based on metadata predicates.

Automatically constructs LLM prompts by combining retrieved documents with user queries, implementing prompt templates with variable substitution and built-in safeguards against prompt injection attacks. Handles context window management (token counting, truncation) to fit retrieved documents within model limits, with configurable strategies for prioritizing relevant chunks when context exceeds capacity.

Coordinates multi-step RAG workflows (document ingestion → embedding → storage → retrieval → prompt assembly → LLM call) as composable NestJS services with explicit state management and error handling. Implements pipeline patterns (sequential, parallel, conditional) with observability hooks for logging, metrics, and debugging at each stage.

Streams LLM responses token-by-token back to clients while maintaining RAG context, allowing real-time feedback and cancellation. Implements backpressure handling to prevent buffer overflow, token counting for cost tracking, and optional streaming of intermediate retrieval results (e.g., which documents were retrieved) before the LLM response begins.

Collects metrics on RAG system performance including retrieval quality (precision, recall, NDCG), LLM response quality (relevance, factuality), and end-to-end latency. Implements evaluation strategies (ground truth comparison, LLM-as-judge, human feedback) and stores results for analysis and continuous improvement, with integration points for A/B testing different retrieval or generation strategies.

Isolates RAG data and operations between tenants using namespace-based partitioning in vector stores and metadata-based filtering in retrieval queries. Implements tenant-aware authentication and authorization checks at the service layer, ensuring queries only retrieve documents belonging to the authenticated tenant, with audit logging for compliance.