llama-index

Ingests structured and unstructured data from 50+ sources (PDFs, web pages, databases, cloud storage) through a unified Reader abstraction pattern. Each reader implements a common interface that converts heterogeneous data formats into a normalized Document/Node representation with metadata preservation. The framework uses a composition pattern where readers can be chained and configured independently, enabling flexible data pipeline construction without modifying core ingestion logic.

Splits documents into semantically coherent chunks (Nodes) using multiple parsing strategies: recursive character splitting, language-aware parsing (code, markdown), and semantic boundary detection. The NodeParser abstraction allows swapping strategies (SimpleNodeParser, HierarchicalNodeParser, SemanticSplitterNodeParser) based on document type. Preserves document hierarchy, metadata, and relationships between chunks, enabling context-aware retrieval that respects logical document structure rather than arbitrary token boundaries.

Provides comprehensive observability through an event-based instrumentation framework that emits structured events for all framework operations (retrieval, LLM calls, tool execution, workflow steps). Events are captured and can be routed to observability backends (LangSmith, Arize, custom handlers). Includes built-in metrics collection (latency, token usage, cost) and debugging utilities. Supports both synchronous and asynchronous event handling with configurable filtering and sampling.

Provides utilities for generating fine-tuning datasets from RAG workflows and optimizing models through fine-tuning. Captures query-response pairs from production RAG systems, generates synthetic training data using LLMs, and exports datasets in standard formats (OpenAI, Hugging Face). Supports fine-tuning of embedding models, rerankers, and LLMs. Includes evaluation metrics for assessing fine-tuning impact on retrieval and generation quality.

Provides LlamaPacks — pre-built, composable templates for common RAG and agent patterns (e.g., multi-document QA, code analysis, research assistant). Each pack is a self-contained module with configured components (readers, indexers, query engines, agents) that can be instantiated with minimal configuration. Packs are discoverable through a registry and can be customized by swapping components. Enables rapid prototyping of complex applications without building from scratch.

Abstracts storage of indices, documents, and metadata behind a unified StorageContext interface supporting multiple backends (file system, cloud storage, databases). Enables serialization and deserialization of indices without vendor lock-in. Supports incremental updates, versioning, and backup strategies. Integrates with vector stores, graph stores, and document stores for comprehensive persistence. Handles automatic index rebuilding and cache invalidation.

Provides a Settings abstraction for managing framework configuration (LLM models, embedding models, vector stores, chunk sizes, etc.) with environment variable overrides. Supports configuration files (YAML, JSON) and programmatic configuration. Enables easy switching between development and production configurations without code changes. Integrates with dependency injection for component instantiation.

Abstracts vector storage and retrieval behind a unified VectorStore interface, supporting 15+ backends (Pinecone, Weaviate, Milvus, PostgreSQL pgvector, Qdrant, Azure AI Search, etc.). Enables hybrid retrieval combining vector similarity with keyword search, metadata filtering, and graph-based traversal. The Index abstraction (VectorStoreIndex, SummaryIndex, KeywordTableIndex, PropertyGraphIndex) provides different retrieval semantics, allowing developers to choose retrieval strategy based on query characteristics and data structure without changing application code.

Orchestrates complex retrieval and LLM synthesis workflows through composable QueryEngine abstractions. Implements patterns like retrieval-augmented generation (retrieve → synthesize), tree-based summarization (hierarchical retrieval), and multi-document synthesis. Uses a Retriever → Response Synthesizer pipeline where retrievers fetch relevant nodes and synthesizers generate LLM responses with citations. Supports advanced patterns like recursive retrieval (refine queries based on intermediate results) and sub-question query engines (decompose complex queries into sub-questions, retrieve for each, then synthesize).

Provides a Workflow abstraction for building stateful, event-driven LLM applications using a step-based execution model. Workflows are defined as directed acyclic graphs (DAGs) of steps, where each step is an async function that processes events and emits new events. The framework handles event routing, state management, and step scheduling automatically. Supports both sequential and parallel execution, conditional branching based on step outputs, and human-in-the-loop checkpoints. Integrates with LLM tool calling for autonomous agent workflows.

Provides an Agent abstraction for building autonomous LLM agents that use tools (function calling) to accomplish tasks. Agents maintain conversation history and can be composed into multi-agent systems where agents delegate tasks to each other. The framework handles tool schema generation, function calling orchestration, and response parsing across multiple LLM providers (OpenAI, Anthropic, Ollama). Supports different agent types (ReActAgent, OpenAIAgent, FunctionCallingAgent) with varying reasoning strategies. Integrates with memory systems for persistent agent state across sessions.

Abstracts LLM interactions behind a unified LLM interface supporting 20+ providers (OpenAI, Anthropic, Google, AWS Bedrock, Ollama, Hugging Face, etc.). Each provider implementation handles authentication, API communication, message formatting, and response parsing. The framework normalizes different LLM APIs (streaming vs. non-streaming, function calling schemas, token counting) into a consistent interface. Supports both cloud-hosted and self-hosted models, with automatic fallback and retry logic. Integrates with embedding models through a parallel Embeddings abstraction.

Abstracts embedding generation behind a unified Embeddings interface supporting 15+ providers (OpenAI, Hugging Face, Ollama, Google, AWS Bedrock, etc.). Handles batch embedding, caching of computed embeddings, and automatic retry logic. Supports both text and multimodal embeddings. The framework normalizes embedding dimensions and similarity metrics across providers. Integrates with vector stores for automatic embedding generation during indexing and retrieval.

Builds knowledge graphs from documents using LLM-based entity and relationship extraction, storing them in a PropertyGraphIndex. The framework uses LLMs to extract entities, relationships, and properties from text, then constructs a graph representation queryable via graph traversal. Supports multiple graph store backends (Neo4j, TigerGraph, Kuzu, etc.). Enables hybrid retrieval combining semantic search with graph-based relationship traversal. Supports knowledge graph completion and reasoning over extracted relationships.

Generates LLM responses with automatic source attribution and citations using a ResponseSynthesizer abstraction. Implements multiple synthesis strategies: simple concatenation of retrieved context, iterative refinement (generate → retrieve → refine), tree-based summarization (hierarchical synthesis), and compact synthesis (minimize context while maintaining quality). Tracks source provenance throughout synthesis, enabling citation generation with document references and node IDs. Supports custom synthesis prompts and response formatting.