Dify

Dify implements a node factory pattern with dependency injection to construct directed acyclic graphs (DAGs) where each node type (LLM, HTTP, code execution, knowledge retrieval, human input) is instantiated via a registry. The workflow engine executes nodes sequentially or in parallel based on graph topology, with built-in pause-resume mechanisms for human-in-the-loop workflows. Node state is persisted across execution boundaries, enabling long-running workflows with intermediate checkpoints.

Dify abstracts LLM provider differences (OpenAI, Anthropic, Ollama, local models, etc.) through a provider and model architecture layer that normalizes API calls, token counting, and cost tracking. The model invocation pipeline routes requests to the appropriate provider SDK, applies quota limits per workspace/user, and deducts credits from a shared pool. Supports both streaming and non-streaming responses with unified error handling and fallback logic.

Dify's workflow testing system allows users to execute workflows with mock data (injected variables) without invoking external APIs or LLM providers. The test runner supports single-node testing (test individual nodes in isolation) and full workflow testing, with execution traces showing node outputs, errors, and execution time. Mock responses can be configured for LLM nodes, HTTP requests, and tool calls, enabling rapid iteration without incurring API costs.

Dify supports file uploads (PDF, DOCX, TXT, Markdown, images) with automatic format detection and content extraction. Files are processed asynchronously via Celery, with support for OCR on images and PDF text extraction. Uploaded files can be used as workflow inputs, indexed into knowledge bases, or referenced in prompts. File metadata (size, type, upload time) is stored in the database, and files are persisted in configurable storage backends (local filesystem, S3, Azure Blob Storage).

Dify's annotation system allows users to rate and comment on LLM outputs within conversations or workflows. Feedback is collected through the chat UI or API, stored in the database with user context (user ID, conversation ID, timestamp), and can be exported for analysis or fine-tuning. The annotation interface supports multiple rating scales (thumbs up/down, 1-5 stars, custom scales) and free-form comments, enabling continuous improvement of LLM applications.

Dify maintains a complete execution history for each workflow, storing run records with execution status, input variables, output results, and execution traces. The run management system supports filtering, searching, and exporting runs, and includes archival functionality to move old runs to cold storage while maintaining queryability. Archived runs can be restored if needed, enabling long-term retention without impacting database performance.

Dify's RAG system decouples document indexing, storage, and retrieval through a vector database factory pattern that supports Weaviate, Pinecone, Milvus, and other backends. The retrieval pipeline implements multiple strategies (semantic search, BM25 hybrid search, metadata filtering, summary index generation) and applies them based on query type. Documents are indexed asynchronously via Celery, with support for chunking strategies, embedding models, and external knowledge base integration (e.g., Notion, GitHub).

Dify implements a tool provider architecture that supports built-in tools (Google Search, Slack, Zapier), API-based tools (custom HTTP endpoints), and Model Context Protocol (MCP) tools via a plugin daemon. Tools are registered in a tool manager with schema definitions (input parameters, output types) and bound to LLM nodes via function calling. MCP integration uses SSE (Server-Sent Events) for bidirectional communication with external tool providers, enabling dynamic tool discovery and execution.

Dify implements a tenant model where each workspace is an isolated resource container with its own datasets, workflows, API keys, and member roles. Authentication supports multiple flows (email/password, OAuth, SAML) with role-based access control (Owner, Admin, Editor, Viewer) that restricts access to workflows, datasets, and API endpoints. Resource quotas (API calls, token usage, storage) are enforced at the workspace level via the credit pool system, and audit logs track all user actions.

Dify provides a visual prompt editor that supports prompt templating (variable substitution), model parameter tuning (temperature, max_tokens, top_p), and version history with git-like diffs. The IDE includes built-in testing with mock data, A/B testing to compare prompt variants, and an annotation system that collects user feedback on LLM outputs. Feedback is stored and can be used to fine-tune models or improve prompts via a feedback loop.

Dify's dataset service provides a unified interface for managing knowledge bases across multiple document sources (file uploads, web crawling, database queries, external integrations like Notion and GitHub). Documents are processed asynchronously via Celery with configurable chunking strategies, embedded using pluggable embedding models, and indexed into the selected vector database. The dataset UI shows indexing progress, allows manual document management (delete, re-index), and supports metadata tagging for retrieval filtering.

Dify exposes REST and WebSocket APIs for chat interactions that support streaming responses (Server-Sent Events), message history persistence, and conversation context management. The chat API accepts user messages, routes them through the workflow or agent, and returns LLM responses with metadata (tokens used, cost, latency). Conversations are stored in the database with full history, enabling context-aware follow-up messages and conversation analytics.

Dify integrates OpenTelemetry for distributed tracing and Sentry for error tracking, capturing traces of workflow execution, LLM calls, and tool invocations. The trace manager records spans for each operation (LLM inference, tool execution, database query) with metadata (tokens, cost, latency, errors). Traces can be exported to external observability platforms (Jaeger, Datadog, New Relic) or viewed in the Dify console, enabling debugging and performance monitoring.

Dify generates REST API endpoints for deployed applications (workflows, agents, chatbots) with automatic OpenAPI documentation. Each application has workspace-level and app-level API keys for authentication, and supports rate limiting, CORS configuration, and request/response logging. The API layer handles request routing to the appropriate workflow or agent, manages conversation state, and returns structured responses with metadata (tokens, cost, execution time).