Private GPT

Converts uploaded documents into vector embeddings using local language models, storing them in a local vector database without sending data to external servers. Uses retrieval-augmented generation (RAG) architecture where documents are chunked, embedded via local transformers, and indexed for semantic search. The entire embedding pipeline runs on-device, enabling privacy-preserving document understanding without cloud dependencies.

Answers questions about uploaded documents using a locally-running large language model, combining retrieved document chunks with the LLM prompt to generate contextual answers. Implements a retrieval-augmented generation (RAG) loop where user queries are embedded, matched against indexed documents, and the top-K relevant chunks are injected into the LLM context window before generation. No query or document content is sent to external LLM APIs.

Exports QA results (questions, answers, source documents) in multiple formats (JSON, CSV, Markdown, PDF) for sharing, archival, or integration with other tools. Supports batch export of entire chat sessions or individual Q&A pairs. Includes options for including/excluding source document references, metadata, and confidence scores in exports.

Exposes Private GPT functionality through a REST API or Python SDK, enabling developers to integrate document QA, semantic search, and embedding capabilities into custom applications. Supports authentication (API keys), rate limiting, and request/response serialization. Allows programmatic control over document indexing, querying, and model configuration without using the GUI.

Searches across multiple documents using semantic similarity rather than keyword matching, embedding the user's search query and comparing it against indexed document chunks to return contextually relevant results. Uses cosine similarity or other distance metrics to rank chunks by relevance, enabling users to find information even when exact keywords don't match. Supports filtering by document metadata (filename, date, tags) before semantic ranking.

Accepts documents in multiple formats (PDF, DOCX, TXT, MD, CSV) and converts them to a unified text representation for embedding and indexing. Uses format-specific parsers (PyPDF2 for PDFs, python-docx for DOCX, CSV readers) to extract text while preserving document structure metadata (page numbers, section headers, table information). Handles OCR for scanned PDFs if enabled, converting image-based text to machine-readable format.

Splits documents into overlapping text chunks optimized for embedding and LLM context windows, using configurable chunk size (typically 256-1024 tokens) and overlap percentage (10-50%) to preserve context across chunk boundaries. Implements smart chunking that respects document structure (paragraph breaks, section headers) rather than naive fixed-size splitting, ensuring semantic coherence within chunks. Metadata (source document, chunk index, page number) is attached to each chunk for source attribution.

Stores vector embeddings and document metadata in a local vector database (e.g., FAISS, Chroma, or SQLite with vector extensions) that persists across sessions, enabling users to build and reuse document indexes without re-embedding on each startup. Supports incremental indexing where new documents are added to existing indexes without rebuilding from scratch. Provides basic CRUD operations (create, read, update, delete) for managing indexed documents.

Supports integration with multiple local LLM providers (Ollama, LM Studio, vLLM, llama.cpp) through a unified interface, allowing users to swap LLM models without changing application code. Handles model loading, inference parameter configuration (temperature, top-p, max tokens), and prompt formatting for different model architectures (Llama 2, Mistral, Phi, etc.). Supports quantized models (GGML, GPTQ) for reduced memory footprint and faster inference.

Maintains conversation history within a session, using previous messages to provide context for follow-up questions and improving answer coherence. Implements a sliding context window that includes recent chat history (typically last 5-10 messages) in the LLM prompt, allowing the model to understand references to previous topics. Supports conversation persistence (saving/loading chat sessions) and optional summarization of long conversations to fit within LLM context limits.

Automatically extracts and assigns metadata to documents (creation date, author, document type, custom tags) from file properties and document content, enabling filtering and organization of document collections. Supports manual tagging where users can assign custom labels to documents for categorization. Metadata is indexed alongside embeddings, allowing search and filtering by document properties (e.g., 'show results only from 2024 documents').

Processes multiple documents in batch mode, embedding and indexing them in parallel or sequential batches to improve throughput compared to processing documents one-at-a-time. Implements progress tracking, error handling, and retry logic for failed documents, allowing users to upload large document collections without manual intervention. Supports resumable batch jobs where interrupted processing can be resumed without reprocessing completed documents.