WeKnora

Accepts heterogeneous document types (PDF, Word, images, structured data) and processes them through a document upload pipeline that extracts content, applies intelligent chunking strategies, and preserves semantic boundaries. Uses event-driven architecture with async task processing via Asynq to handle large-scale document ingestion without blocking the main service, storing chunks in a vector-indexed database with metadata tags for retrieval.

Implements a hybrid retrieval strategy combining vector similarity search (semantic) with keyword-based matching, using a configurable reranking engine to fuse results from both approaches. The retrieval pipeline queries the vector database for semantic matches and applies optional reranking (e.g., BM25, cross-encoder models) to surface the most relevant chunks before passing them to the LLM context window.

Uses Asynq (Redis-backed task queue) to handle long-running operations asynchronously, including document processing, embedding generation, and knowledge graph construction. Tasks are enqueued with configurable retry policies, priority levels, and deadlines. The system provides task status tracking and allows users to monitor progress without blocking the API.

Implements an event-driven architecture for chat interactions where user messages trigger events that flow through handlers (retrieval, reasoning, response generation). The pipeline supports streaming responses, allowing partial results to be sent to the client as they become available. Events are processed sequentially within a session to maintain conversation order.

Allows organizations to select and configure embedding models from multiple providers (OpenAI, Ollama, local models) at the knowledge base level. Embeddings are generated during document indexing and stored in the vector database. The system supports model switching with re-embedding of existing documents, and provides fallback mechanisms if the primary provider is unavailable.

Allows documents and chunks to be tagged with custom labels, enabling hierarchical organization and filtering during retrieval. Tags are stored in the database and indexed for fast filtering. Queries can be scoped to specific tags, and retrieval results can be filtered by tag combinations. Tags support hierarchical relationships (parent-child).

Provides tools to evaluate RAG pipeline quality by measuring retrieval precision/recall, answer relevance, and end-to-end QA accuracy. Supports benchmark datasets and allows comparing performance across different retrieval strategies, embedding models, and LLM configurations. Evaluation results are stored and can be tracked over time.

Implements a ReAct (Reasoning + Acting) agent engine that decomposes user queries into reasoning steps, selects appropriate tools (web search, knowledge base retrieval, MCP-integrated functions), executes them, and iterates until reaching a conclusion. The agent maintains conversation context across multiple turns, uses dependency injection to wire tools dynamically, and supports both synchronous and streaming responses.

Enforces tenant isolation at the database and API layer, where each organization owns isolated knowledge bases, documents, and chat sessions. Access control is enforced via organization IDs in request contexts, with role-based permissions (admin, editor, viewer) managed through a security layer. The architecture uses dependency injection to inject tenant context into service handlers, ensuring no cross-tenant data leakage.

Provides a dedicated FAQ subsystem where organizations can define frequently asked questions with curated answers, which are automatically indexed as high-priority chunks in the vector database. FAQs are tagged separately and can be weighted higher during retrieval, ensuring common questions are answered with pre-approved responses. The system supports FAQ versioning and allows marking answers as verified or outdated.

Manages conversation sessions where each chat maintains a history of user queries and assistant responses, with configurable context window management. Sessions are stored in PostgreSQL with optional compression, and context is passed to the LLM for multi-turn reasoning. The system supports session titles (auto-generated or user-defined), session forking, and context summarization to handle long conversations without exceeding token limits.

Integrates with MCP servers to expose external tools and functions as callable capabilities within the agent system. Tools are registered via JSON Schema definitions, and the agent can invoke them during reasoning. The system handles MCP protocol serialization/deserialization, manages tool execution timeouts, and returns results back to the agent for further reasoning.

Integrates web search capabilities (via configurable search providers like Google, Bing) into the agent reasoning loop, allowing agents to decide when to search the web vs. query the knowledge base. Search results are ranked and deduplicated before being passed to the LLM. The system supports search result caching to avoid redundant queries.

Processes documents containing images and scanned PDFs by extracting text via OCR (Optical Character Recognition) and optionally analyzing images using vision models. Extracted text and image descriptions are indexed alongside document metadata, allowing semantic search across both text and visual content. The system supports configurable OCR engines and vision model backends.

Builds and maintains knowledge graphs from indexed documents, representing entities and relationships extracted from text. The system supports graph-based retrieval where queries traverse the graph to find related entities and documents, enabling structured reasoning over interconnected knowledge. Graph construction is async and configurable per knowledge base.