Inngest

Executes multi-step workflows as durable functions that survive process crashes and network failures by persisting execution state to Redis. Uses an Executor service that orchestrates step execution through an HTTP Driver, maintaining checkpoint state at each step boundary. Steps are defined declaratively and executed sequentially or in parallel patterns, with automatic resumption from the last completed step on retry.

Implements configurable retry logic with exponential backoff for failed steps, using Redis queue operations to requeue failed executions with calculated delay. Retries are managed through Lua scripts that atomically update queue state and reschedule execution, supporting custom backoff multipliers and maximum retry counts defined in function configuration.

Provides command-line tools for initializing new functions, managing function definitions, and deploying to Inngest cloud. CLI commands include `inngest init` for scaffolding, `inngest deploy` for pushing function definitions, and `inngest dev` for running the local development server. CLI integrates with SDK to generate boilerplate code and manage function configuration.

Uses Command Query Responsibility Segregation (CQRS) pattern to separate event storage (write model) from query models, with events stored in Redis and queryable via GraphQL. Events represent state transitions (execution started, step completed, execution failed) and are immutable. Query models are built from events and cached for fast access, enabling eventual consistency across the system.

Provides SDKs for Node.js, Python, and Go that implement a unified execution interface, allowing developers to define workflow functions in their preferred language. SDKs handle serialization/deserialization of step inputs/outputs, communicate with Inngest core via HTTP or WebSocket, and provide decorators/annotations for defining steps. Each SDK maintains compatibility with the same function schema and execution model.

Enforces concurrency limits and rate limiting through a partition-based queue system where executions are distributed across Redis-backed partitions with per-partition lease management. Constraints are defined in function configuration and enforced via Lua scripts that check available capacity before dequeuing, preventing more than N concurrent executions of the same function or matching a concurrency key pattern.

Triggers workflow execution based on incoming events matched against function trigger definitions using pattern matching logic. Events are ingested via REST API or GraphQL mutations, compared against trigger patterns defined in CUE configuration, and matching functions are enqueued for execution with event data as input. Supports multiple trigger types including event name matching and conditional filters.

Allows workflows to pause execution at any step and resume when a specific event is received, implemented through pause state stored in Redis and event matching logic. When a step returns a pause action, execution state is persisted and the workflow waits for a matching event. Upon event arrival, the pause is cleared and execution resumes from the paused step with event data as input.

Communicates with SDKs (Node.js, Python, Go) through an HTTP Driver that sends step execution requests and receives responses, with optional WebSocket fallback via Connect Gateway for environments where HTTP polling is inefficient. The HTTP Driver (httpv2.go) handles request serialization, response parsing, and error handling, while Connect Gateway provides bidirectional WebSocket communication for long-lived connections.

Defines workflow functions declaratively using CUE configuration language, which specifies function name, triggers, steps, concurrency constraints, and input/output schemas. CUE schemas are compiled and stored in the system, enabling validation of event payloads and step inputs. Function configuration is versioned and can be updated without redeploying the entire system.

Implements a distributed queue using Redis as the backing store, with Lua scripts ensuring atomic operations for enqueue, dequeue, lease management, and requeue. Queue operations are partitioned for scalability, with each partition maintaining its own backlog and lease tracking. Lua scripts (enqueue.lua, dequeue.lua, lease.lua) handle complex state transitions atomically, preventing race conditions in distributed execution.

Provides comprehensive execution tracing through a GraphQL API that exposes function runs, step executions, and execution timeline. Traces are stored in the backend and queryable via GraphQL resolvers (function_run.resolver.go), enabling visualization of execution flow, step durations, and error details. Trace data includes step inputs/outputs, execution state transitions, and timing information.

Provides a local development server (devserver.go) that runs Inngest core locally, enabling developers to test functions without deploying to production. The dev server includes a UI dashboard for viewing function runs and traces, supports hot reload of function definitions, and communicates with local SDKs via HTTP. Developers can trigger functions manually through the UI or via API calls.