dagu

Dagu parses YAML files into directed acyclic graphs (DAGs) where each step is a node with dependencies explicitly declared. The engine validates the DAG structure at parse time, detects cycles, and builds an execution plan that respects task dependencies. This file-based approach eliminates the need for a UI or database schema — workflows are version-controllable text artifacts that can be audited, diffed, and reviewed like code.

Dagu compiles to a single Go binary that can run standalone on a laptop or scale to a distributed cluster by spawning worker processes or connecting to remote nodes. The engine uses a local scheduler for single-machine execution and supports remote task execution via SSH or custom executors. This architecture eliminates the need for separate control planes, message brokers, or container orchestration — the same binary handles both local cron-like scheduling and distributed task dispatch.

Dagu enforces task ordering through explicit dependency declarations in YAML — each task specifies which tasks it depends on, creating a directed acyclic graph (DAG) of execution order. The engine validates dependencies at parse time, detects cycles, and builds an execution plan that respects the DAG. This ensures tasks run in the correct order without race conditions, and enables parallel execution of independent tasks.

Dagu supports defining reusable step templates that can be instantiated multiple times in a workflow with different parameters. Templates encapsulate common task patterns (e.g., 'run a Docker container', 'call an API', 'execute a script') and can be parameterized to avoid duplication. This enables DRY (Don't Repeat Yourself) workflow definitions where common patterns are defined once and reused across multiple workflows.

Dagu implements signal handling (SIGTERM, SIGINT) to gracefully shut down running workflows and tasks. When a shutdown signal is received, the engine attempts to stop currently executing tasks cleanly (allowing them to finish or respond to signals) rather than forcefully killing them. This enables safe workflow interruption without data corruption or orphaned processes, and supports deployment scenarios where the Dagu daemon needs to be restarted or updated.

Dagu tracks task execution state (pending, running, success, failure) and persists this state to enable automatic retries, resume-on-failure, and idempotent re-execution. When a task fails, the engine can automatically retry with exponential backoff or skip to the next step based on configured policies. Failed workflows can be resumed from the point of failure without re-executing completed steps, enabling long-running pipelines to recover from transient failures without manual intervention.

Dagu embeds a cron scheduler that interprets standard cron expressions (minute, hour, day, month, day-of-week) to trigger workflows on a schedule. The scheduler runs as part of the Dagu daemon and can trigger workflows based on wall-clock time or custom events. This eliminates the need for external cron daemons or scheduling services — the workflow engine itself handles scheduling, making it suitable for air-gapped environments where external services are unavailable.

Dagu exposes a web dashboard and REST API that provide real-time visibility into workflow execution, task status, logs, and history. The UI displays DAG visualizations, execution timelines, and task output; the API enables programmatic workflow triggering, status queries, and log retrieval. This allows operators to monitor and control workflows without SSH access or command-line tools, and enables integration with external systems (Slack notifications, custom dashboards, alerting systems).

Dagu supports passing environment variables and parameters to individual tasks through YAML configuration, command-line arguments, or API calls. Variables can be defined globally (workflow-level), per-task, or dynamically from previous task outputs. The engine substitutes variables into task commands before execution, enabling parameterized workflows that adapt to different environments (dev, staging, prod) without modifying the YAML definition.

Dagu supports conditional execution of tasks based on the exit code or output of previous tasks. Tasks can be marked as optional (continue on failure), skipped based on conditions, or executed only if upstream tasks succeed. This enables branching workflows where different paths are taken based on runtime conditions, without requiring explicit if-then-else constructs — the DAG structure itself encodes the branching logic through task dependencies and conditions.

Dagu captures stdout/stderr from task execution and makes it available to downstream tasks through variable substitution or API queries. Tasks can write structured output (JSON, key=value pairs) that is parsed and injected as environment variables for subsequent tasks. This enables data flow through the workflow — one task produces output that becomes input to the next task, without requiring external message queues or databases.

Dagu maintains a persistent execution history of all workflow runs, including task status, exit codes, start/end times, and logs. This history is queryable via the REST API and web UI, enabling audit trails, performance analysis, and debugging. The engine stores execution metadata (who triggered the workflow, when, with what parameters) and task-level details (duration, resource usage if available), providing full observability into workflow behavior over time.

Dagu supports custom executors that define how tasks are executed — beyond the default shell command execution. Executors can be implemented as external programs or plugins that receive task definitions and execute them in custom environments (Docker containers, Kubernetes pods, remote services, custom runtimes). This enables Dagu to orchestrate tasks across heterogeneous infrastructure without modifying the core engine or workflow definitions.