dify

Dify implements a Provider and Model Architecture that abstracts multiple LLM providers (OpenAI, Anthropic, Gemini, etc.) through a unified invocation pipeline. The system uses a quota management layer with credit pools to track and limit API consumption per tenant, enforcing rate limits and cost controls at the model invocation level before requests reach external APIs. This architecture enables seamless provider switching and cost governance across multi-tenant deployments.

Dify's Workflow Engine uses a Directed Acyclic Graph (DAG) execution model where workflows are composed of typed nodes (LLM, HTTP, Code, Knowledge Retrieval, Human Input) connected by edges. The engine executes nodes sequentially or in parallel based on dependencies, with a pause-resume mechanism that allows Human Input nodes to block execution and wait for external input before continuing. Node Factory and Dependency Injection patterns enable dynamic node instantiation and testing via mock systems.

Dify provides Docker Build Process with Multi-Stage Images for containerized deployment, supporting both API and frontend services. The system uses Environment Configuration and Runtime Modes to manage settings across development, staging, and production environments. Docker Compose Stack orchestrates the full application stack (API, frontend, PostgreSQL, Redis, vector database) for local development and testing, while production deployments use Kubernetes or managed container services.

Dify abstracts three Application Types (Chatbot, Agent, Workflow) with different execution models and capabilities. Chatbots use simple LLM calls with conversation history; Agents use ReAct-style reasoning with tool calling and multi-step planning; Workflows use explicit DAG execution with node composition. The Application Type determines available features (tool calling, knowledge retrieval, human input) and execution modes (streaming, async, batch).

Dify's Tool and Plugin Ecosystem supports three tool types: built-in tools (web search, calculator, etc.), API-based tools (HTTP requests with schema validation), and MCP tools (via MCP protocol). Tools are registered in a unified Tool Manager with JSON Schema definitions for parameter validation. When agents or workflows invoke tools, parameters are validated against schemas before execution, preventing invalid API calls and improving error handling.

Dify's Knowledge Base and RAG System manages document ingestion, chunking, embedding, and retrieval across multiple vector database backends (Pinecone, Weaviate, Qdrant, Milvus, etc.). The Document Indexing Pipeline processes uploaded files through a configurable chunking strategy, generates embeddings via provider-agnostic APIs, and stores vectors with metadata filtering. The RAG Pipeline Workflow retrieves relevant documents based on semantic similarity and metadata filters, then passes them to LLM nodes for context-aware generation.

Dify integrates the Model Context Protocol (MCP) to enable dynamic tool and plugin discovery, schema registration, and execution. The MCP Client (SSE and streamable variants) communicates with MCP servers to fetch tool schemas, invoke tools with validated parameters, and handle streaming responses. Tools are registered in a unified Tool Manager that abstracts MCP, built-in, and API-based tools, allowing workflows to call external tools through a consistent interface without hardcoding tool implementations.

Dify implements a Tenant Model with Resource Isolation that separates workspaces, datasets, workflows, and API keys by tenant. Role-Based Access Control (RBAC) enforces permissions at the workspace and member level, with roles (Admin, Editor, Viewer) controlling access to applications, datasets, and workflow execution. Authentication Methods support API keys, OAuth, and SAML, with Account Lifecycle Management handling user provisioning, deprovisioning, and workspace membership.

Dify exposes Chat and Completion APIs that accept user messages, route them through application logic (RAG, workflows, or simple LLM calls), and return responses with streaming support. Conversation history is persisted in PostgreSQL, enabling context-aware multi-turn interactions. Feedback APIs allow users to rate responses and provide corrections, which are stored for model fine-tuning and quality monitoring. The API Architecture uses request-response patterns with optional streaming via Server-Sent Events (SSE).

Dify's Web Frontend Architecture provides a visual workflow builder UI built with Next.js that enables drag-and-drop composition of workflow nodes. The frontend renders Workflow Node UI Components for each node type (LLM, HTTP, Code, Knowledge Retrieval, Human Input), with real-time validation of node configurations and connections. The Chat Interface and Message Interactions component provides a conversational UI for testing applications and viewing multi-turn conversations with streaming message updates.

Dify's File Upload and Management APIs handle document ingestion in multiple formats (PDF, DOCX, TXT, Markdown, CSV, JSON) with automatic format detection and conversion. Uploaded files are stored in configurable backends (local filesystem, S3, Azure Blob Storage) and associated with datasets for RAG indexing. The system extracts text and metadata from documents, applies document-level access controls, and tracks file lifecycle (upload, indexing, deletion).

Dify uses Celery with Redis as a message broker to execute long-running workflows asynchronously in the background. Workflow execution requests are enqueued as Celery tasks, processed by worker processes, and results are stored in PostgreSQL for later retrieval. The Async Workflow Service manages task lifecycle (enqueue, execute, store results, handle failures), with support for task retries, timeouts, and dead-letter queues for failed tasks.

Dify integrates OpenTelemetry for distributed tracing and Sentry for error tracking and performance monitoring. The Trace Manager collects traces from workflow execution, LLM API calls, and database queries, exporting them to OpenTelemetry-compatible backends (Jaeger, Datadog, etc.). Sentry integration captures exceptions, performance metrics, and user feedback, enabling real-time alerting and post-mortem analysis of production issues.