TaskingAI

Unifies integration with hundreds of LLM providers (OpenAI, Anthropic, Google Gemini, etc.) through a standardized inference API gateway that abstracts provider-specific APIs into a common interface. The Inference Service handles provider registration, credential management, and request routing via a FastAPI application that translates unified chat completion requests into provider-specific API calls, enabling seamless model switching without application code changes.

Implements a complete RAG pipeline with document ingestion, vector embedding, and semantic search capabilities. The Retrieval System API manages document storage in object storage, maintains vector embeddings in a vector database, and executes semantic search queries to retrieve contextually relevant documents. This enables LLM applications to augment prompts with external knowledge without fine-tuning, using a retrieval-first architecture that separates document indexing from inference.

Implements a dedicated Inference Service that handles communication with various LLM providers through provider-specific API clients. The service translates unified chat completion requests from the Backend into provider-specific formats (OpenAI, Anthropic, Google Gemini, etc.), manages provider credentials, handles streaming responses, and returns standardized results. This service is decoupled from the Backend, enabling independent scaling and updates without affecting other components.

Manages persistent storage of conversation history in PostgreSQL with full message tracking, metadata, and context preservation. Each conversation maintains a complete message history with timestamps, token usage, and provider information. The system enables retrieving conversation history for context injection into subsequent requests, supporting multi-turn interactions where the LLM can reference previous messages. Context is managed at the database level, allowing applications to retrieve and manipulate conversation state independently of the inference service.

Provides a set of pre-built plugins that implement common tool integrations such as web search, calculations, and API calls. These built-in plugins are registered in the Plugin Service with JSON schemas and can be immediately used by assistants without custom development. The plugin architecture allows extending this library with custom plugins, enabling organizations to build domain-specific tools while leveraging common integrations out of the box.

Implements a Redis caching layer that improves performance by caching frequently accessed data such as model configurations, assistant definitions, and retrieval results. The Backend Layer uses Redis to reduce database queries and improve response latency for common operations. Cache invalidation is handled through application logic, ensuring consistency between cached and persistent data.

Integrates with object storage (S3-compatible or local filesystem) to store documents, embeddings, and other binary data used by the RAG system. The Retrieval System API manages document uploads, storage, and retrieval through a standardized object storage interface. This separation of document storage from the database enables efficient handling of large files and reduces database size, while the abstraction allows switching between different storage backends.

Manages a plugin architecture that enables LLMs to call external tools and functions through a standardized interface. The Plugin Service exposes a registry of available tools with JSON schemas, handles function invocation requests from LLMs, executes tool logic, and returns results back to the inference pipeline. Built-in plugins provide common capabilities (web search, calculations, etc.), while custom plugins can be registered via the Plugin API Gateway for domain-specific integrations.

Provides APIs for creating and configuring AI assistants with persistent conversation history and state management. The Assistant Operations API enables defining assistants with specific system prompts, model selections, tool bindings, and RAG configurations. Conversations are stored in PostgreSQL with full history tracking, enabling multi-turn interactions where context is maintained across requests. The architecture separates assistant definitions from conversation instances, allowing multiple conversations per assistant.

Provides a web-based Playground UI that enables interactive testing and experimentation with LLM models and configured assistants. The frontend communicates with the Backend API to execute inference requests, display responses in real-time, and visualize token usage and provider information. The playground supports switching between models, adjusting parameters, and testing tool calling without requiring code changes, serving as both a development tool and a way to validate assistant behavior before deployment.

Provides a web-based interface for registering, configuring, and managing LLM models from various providers. The Models Management UI allows users to add provider credentials, select which models to enable, configure model-specific parameters, and view available models from each provider. This configuration is stored in PostgreSQL and used by the Inference Service to route requests appropriately, centralizing provider credential management and model availability configuration.

Provides a web-based Retrieval Management UI for uploading documents, managing knowledge bases, configuring vector embeddings, and testing semantic search. Users can upload documents (text, PDFs), view indexed documents and their embeddings, configure embedding models, and test retrieval queries to validate that relevant documents are being retrieved. The UI communicates with the Retrieval System API to manage documents in object storage and embeddings in the vector database.

Provides a web-based Plugin Management UI for viewing available plugins, configuring tool parameters, and testing tool execution. Users can see built-in plugins and custom plugins registered in the system, view their JSON schemas, configure tool-specific settings, and test tool calls to validate behavior. The UI communicates with the Plugin API Gateway to manage plugin configurations and execute test invocations.

Implements the Backend Layer as a FastAPI application that exposes RESTful APIs for all TaskingAI operations. The architecture uses clear separation of concerns with distinct API endpoints for Model Operations, Assistant Operations, Retrieval System, and Plugin management. Each API communicates with corresponding services (Inference Service, Plugin Service) through defined interfaces, with PostgreSQL as the primary data store and Redis for caching. The layered design enables independent scaling and testing of each component.

Provides a complete Docker Compose configuration that orchestrates all TaskingAI components (Frontend, Backend, Inference Service, Plugin Service, PostgreSQL, Redis, Object Storage) into a single deployable unit. The docker-compose.yml file defines service dependencies, environment variables, volume mounts, and networking, enabling single-command deployment of the entire system. This approach abstracts infrastructure complexity and ensures consistent environments across development, testing, and production.