bRAG-langchain

Constructs a complete Retrieval-Augmented Generation pipeline using LangChain Expression Language (LCEL) that separates indexing (one-time document embedding and vector store population) from query execution (per-request retrieval and LLM synthesis). The rag_chain in full_basic_rag.ipynb assembles retriever, prompt templates, and LLM into a single composable expression, enabling declarative pipeline definition without imperative control flow.

Generates multiple semantically-diverse query variants from a single user question using an LLM, then retrieves documents against all variants in parallel, unions the results, and deduplicates to improve recall. Implemented in Notebook 2 via LLM prompt templates that instruct the model to generate alternative phrasings, followed by concurrent retriever calls and result aggregation.

Manages LLM prompts using LangChain PromptTemplate, enabling parameterized prompt construction with context injection, variable substitution, and format specification. Notebooks demonstrate prompts for retrieval evaluation, query generation, answer synthesis, and re-ranking, with explicit separation of system instructions, context, and user input.

Provides five structured Jupyter notebooks (Notebooks 1-5) that progressively introduce RAG techniques from basic setup to advanced retrieval and self-correction. Each notebook builds on the previous, introducing new techniques (multi-query, routing, advanced indexing, re-ranking) with executable code, explanations, and reference links. The progression enables learners to understand RAG incrementally rather than all-at-once.

Provides a self-contained, production-ready RAG chatbot implementation in full_basic_rag.ipynb that can be adapted to custom documents, LLMs, and vector stores. The boilerplate includes document loading, embedding, vector store setup, retrieval chain assembly, and inference loop, enabling developers to fork and customize without building from scratch.

Routes incoming queries to different retrieval or processing paths based on semantic classification or logical rules using LangChain's RunnableBranch construct. Notebook 3 demonstrates routing via LLM classification (e.g., 'is this a factual question or a reasoning task?') and conditional branching to specialized chains (e.g., HyDE for hypothetical document expansion, RAG-Fusion for multi-perspective retrieval).

Implements sophisticated indexing strategies (Notebook 4) including MultiVectorRetriever for storing summaries/questions alongside full documents, InMemoryByteStore for metadata caching, and Parent Document Retriever for retrieving larger context chunks while querying against smaller summaries. These patterns decouple the retrieval unit (summary) from the context unit (full document), improving both precision and context quality.

Applies learned re-ranking to retrieval results using cross-encoder models (e.g., Cohere Rerank API) that score document-query pairs jointly, improving ranking quality beyond embedding-based similarity. Notebook 5 integrates CohereRerank and demonstrates Corrective RAG (CRAG) with LangGraph, which evaluates retrieval quality and iteratively refines queries or retrieves additional documents if confidence is low.

Generates hypothetical documents that would answer the user's query, embeds those hypothetical documents, and uses their embeddings to retrieve real documents. Implemented in Notebook 3, HyDE leverages the LLM's generative capability to imagine relevant document content, then uses those imagined embeddings as retrieval queries, often improving recall for questions where the phrasing differs significantly from document content.

Combines multi-query retrieval with Reciprocal Rank Fusion (RRF), a rank aggregation algorithm that merges results from multiple retrievers by computing harmonic mean of reciprocal ranks. Notebook 3 demonstrates RAG-Fusion, which generates query variants, retrieves from each, and uses RRF to produce a unified ranked list without requiring relevance scores to be comparable across retrievers.

Implements Self-Retrieval-Augmented Generation (Self-RAG) using LangGraph, where the LLM generates responses, evaluates whether retrieval is needed, validates retrieved documents, and iteratively refines answers. Notebook 5 demonstrates this pattern with explicit LLM-based evaluation steps that determine if initial retrieval was sufficient or if additional retrieval/refinement is required.

Loads documents from multiple formats (PDF, markdown, plain text) using LangChain document loaders, chunks them using configurable splitters (recursive character splitting, semantic splitting), and embeds chunks using embedding models (OpenAI, Cohere, local models). Notebook 1 demonstrates the complete indexing pipeline from raw documents to vector store population, with support for metadata extraction and preservation.

Abstracts vector store operations (insert, search, delete, update) across multiple backends including ChromaDB (local/in-memory) and Pinecone (cloud). Notebook 1 demonstrates initialization, population, and querying of both stores, with support for metadata filtering and similarity search. The abstraction enables swapping vector stores without changing retrieval logic.