langchain4j-aideepin

Implements a hybrid RAG system that indexes documents through both vector embeddings and graph-based semantic relationships, enabling retrieval via semantic similarity search and structural graph traversal. The system processes documents through a dual-path pipeline: vector indexing stores embeddings in vector databases (Milvus, Weaviate, Qdrant) while simultaneously constructing knowledge graphs that capture entity relationships and document hierarchies. Query resolution uses both paths—vector search for semantic relevance and graph traversal for relationship-aware context—then merges results for comprehensive document understanding.

Enables real-time conversational AI with text, audio (ASR/TTS), and vision inputs through Server-Sent Events (SSE) streaming architecture. Conversations are grounded in knowledge bases—each message can reference indexed documents through RAG integration, with streaming token-by-token responses sent to clients via HTTP SSE connections. The system maintains conversation state in a relational database (conversation lifecycle management) while streaming LLM outputs in real-time, supporting interruption and context switching without losing conversation history.

Integrates web search capabilities (Google Search, Bing Search, or compatible APIs) into conversations and workflows, enabling LLMs to search the web for current information. Search results are ranked by relevance, deduplicated, and formatted with citations (URL, title, snippet). Results can be injected into conversation context or used as tool outputs in workflows. Supports search filtering (date range, domain, language) and result caching to reduce API calls for repeated queries.

Provides centralized configuration management for system settings (API keys, database connections, feature flags, model parameters) with support for environment-based overrides (development, staging, production). Configuration is stored in application.yml/properties files and database, with runtime updates for non-critical settings. Supports feature flags to enable/disable functionality without code changes. Configuration changes are logged for audit purposes. Implements configuration validation to catch invalid settings at startup.

Provides a visual workflow builder that compiles workflows into LangGraph4j execution graphs with 16+ predefined node types (LLM, tool call, conditional branching, loops, parallel execution, etc.). Workflows are stored as JSON definitions in the database and executed through a state machine engine that manages node transitions, data flow between nodes, and error handling. Each node type maps to specific LangChain4j operations—LLM nodes invoke language models, tool nodes call MCP-registered functions, conditional nodes evaluate state predicates, and loop nodes repeat subgraphs until termination conditions are met.

Implements a Model Context Protocol (MCP) marketplace that allows users to discover, register, and invoke external tools/services through a unified schema-based interface. Tools are registered with JSON schemas defining their inputs/outputs, then made available to LLM agents and workflows through a function-calling abstraction. The system maintains a registry of available MCP servers, handles tool discovery, manages authentication credentials per tool, and provides schema validation before tool invocation. LLMs can call registered tools through standard function-calling APIs (OpenAI, Anthropic, Ollama), with the system translating function calls to MCP protocol invocations.

Processes documents in multiple formats (PDF, Markdown, plain text, web pages, CSV, JSON) through a unified indexing pipeline that chunks documents, extracts metadata, generates embeddings, and stores in vector/graph databases. The pipeline uses configurable chunking strategies (fixed-size, semantic, sliding window) and metadata extraction rules to preserve document structure. Documents are split into chunks with overlap to maintain context, then embedded using configured embedding models (OpenAI, local models via Ollama). Extracted metadata (title, author, source URL, timestamps) is preserved for filtering and citation purposes.

Abstracts multiple LLM providers (OpenAI, Anthropic, Ollama, Hugging Face, etc.) behind a unified interface, allowing users to configure and switch between models without code changes. The system stores model configurations in the database (API keys, model names, temperature, max tokens, etc.) and provides a factory pattern to instantiate the appropriate LLM client based on configuration. Supports both cloud-hosted models (OpenAI GPT-4, Claude) and local models (Ollama, vLLM) with fallback chains if primary model is unavailable. Uses LangChain4j's ChatLanguageModel abstraction to normalize API differences across providers.

Integrates multiple text-to-image generation platforms (DALL-E, Stable Diffusion, Midjourney, etc.) through a unified image generation API. Users specify generation parameters (prompt, style, size, quality) and the system routes requests to configured backends based on availability and user preferences. Supports image editing (inpainting, outpainting) and background generation as separate operations. Generated images are stored in cloud storage (S3, Azure Blob, local filesystem) with metadata (prompt, model, generation time) for tracking and reuse.

Maintains persistent conversation memory across sessions using a multi-tier storage strategy: recent messages in memory for fast access, older messages in relational database, and summarized context in vector embeddings for semantic retrieval. The system automatically summarizes long conversations to reduce token usage in LLM calls while preserving important context. Memory is scoped per user and conversation, with optional sharing across conversations for cross-conversation context. Implements conversation lifecycle management (creation, archival, deletion) with audit trails.

Implements user authentication, authorization, and multi-tenancy support with role-based access control (RBAC) for conversations, knowledge bases, workflows, and tools. Users are assigned roles (admin, user, viewer) with permissions scoped to specific resources. The system supports organization-level multi-tenancy where users belong to organizations and can only access organization-specific resources. Authentication uses standard mechanisms (JWT tokens, OAuth2) with session management. Resource access is enforced at the API level through permission checks before operations are executed.

Manages file uploads, storage, and retrieval with support for multiple backends (S3, Azure Blob Storage, local filesystem). Files are stored with metadata (filename, size, MIME type, upload timestamp, owner) and can be associated with conversations, knowledge bases, or workflows. Supports file versioning, access control (per-user or per-organization), and cleanup policies (auto-delete old files). Provides signed URLs for secure file access without exposing storage credentials. Integrates with document processing pipeline for automatic indexing of uploaded documents.