local-deep-research

Executes deep, multi-turn research workflows that iteratively refine queries based on LLM analysis of intermediate results. The system searches 10+ sources (arXiv, PubMed, web via Brave/SearXNG, private documents) in a coordinated loop, with each iteration using LLM reasoning to identify gaps and reformulate queries. Research execution is managed through a service-oriented architecture with thread-safe settings context, enabling parallel research tasks while maintaining isolation per user and per research session.

Provides per-user data isolation through SQLCipher databases encrypted with AES-256-CBC, where each user's password is derived via PBKDF2-HMAC-SHA512 with 256,000 iterations and a per-user random salt. The database architecture separates user data (research history, collections, settings) from system configuration, with automatic encryption key management and password-based access control. Database encryption check utilities verify SQLCipher compatibility at startup.

Provides a web-based user interface built with Flask backend and modern frontend (likely React or Vue.js based on build system references). The web UI enables real-time research execution with streaming result updates, research history management, and collection/library organization. Frontend communicates with Flask backend via REST API, with WebSocket support for real-time status updates during long-running research.

Implements thread-safe settings management through context variables that enable concurrent research tasks to maintain isolated configuration and state. Each research execution gets its own context (LLM provider, search sources, user credentials) that is thread-local, preventing cross-contamination between concurrent requests. Settings are loaded from environment variables and configuration files with runtime override capability.

Includes built-in benchmarking infrastructure that evaluates research quality against the SimpleQA benchmark, measuring accuracy, citation correctness, and source attribution. The benchmarking system executes research on benchmark queries, compares results against ground truth, and generates accuracy reports. This enables quantitative evaluation of research quality across different LLM providers and configurations.

Automatically downloads and manages research documents (PDFs, web pages) discovered during research, with automatic metadata extraction (title, authors, publication date). Downloaded documents are stored in encrypted database with full-text indexing for later search. Metadata extraction uses heuristics and optional OCR for PDFs, enabling documents to be cited and referenced in future research.

Enables subscription to research topics with automatic periodic research execution and result delivery. The system maintains topic subscriptions in encrypted database, executes research on subscribed topics at configured intervals (daily, weekly, monthly), and delivers results via email or web UI notifications. Subscription management includes filtering, deduplication, and archival of subscription results.

Generates research reports from research results with support for multiple export formats (markdown, HTML, PDF, JSON). Report generation includes automatic formatting, citation insertion, table of contents generation, and optional styling. Exported reports can be shared externally while maintaining citation metadata for verification.

Abstracts multiple LLM providers (OpenAI, Anthropic, Google, Mistral, Ollama) behind a unified Python interface, enabling runtime provider switching without code changes. Configuration is managed through environment variables and thread-safe settings context, with provider-specific parameters (temperature, max_tokens, system prompts) normalized across APIs. The abstraction handles provider-specific response formats, streaming behavior, and error handling transparently.

Coordinates searches across 10+ sources (arXiv, PubMed, Brave web search, SearXNG, Google Scholar, private document collections) through a unified search interface. The Search Engine Integration Layer abstracts provider-specific APIs, query syntax, and result formats into a common result schema with source attribution. Search execution is parallelized where possible, with configurable timeouts and fallback behavior when sources are unavailable.

Indexes private documents (PDFs, markdown, text) into vector embeddings using local or cloud embedding models, enabling semantic search across document collections. Documents are stored in per-user encrypted databases with metadata (source, date, collection), and retrieval uses cosine similarity search on embeddings to find relevant passages. The RAG system integrates with the research workflow to supplement search results from public sources with organization-specific knowledge.

Automatically tracks citations throughout the research process, maintaining evidence chains that link claims in the final report back to original sources. The citation handler (citation_handler.py) extracts source metadata (URL, publication date, authors) from search results and embeds citations in generated content. Reports can be exported with full citation metadata in multiple formats (markdown with footnotes, HTML with hyperlinks, JSON with source provenance).

Exposes research capabilities through a REST API built on Flask with async task execution for long-running research workflows. API endpoints support both synchronous queries (for quick searches) and asynchronous research execution (for deep research that may take 30+ seconds). Task status can be polled via job IDs, with results cached in encrypted database. API authentication uses per-user tokens derived from encrypted credentials.

Provides a Python client library (LDRClient) that enables direct programmatic access to research capabilities without REST API overhead. The client handles user authentication, database access, and research execution through a simple Python interface. Supports both synchronous and asynchronous research execution, with context managers for automatic resource cleanup.

Provides command-line interfaces for research execution and web server management. The `ldr` command enables direct research queries from shell scripts, while `ldr-web` manages the Flask web application lifecycle. CLI tools support configuration via environment variables and command-line flags, with output formatting options (JSON, markdown, plain text).

Implements Model Context Protocol (MCP) server that enables integration with Claude Desktop and other AI assistants. The MCP server exposes research capabilities as tools that Claude can invoke directly, with automatic result formatting and context injection. This enables AI assistants to perform research as part of their reasoning process without external API calls.