Instill

Drag-and-drop interface that constructs directed acyclic graphs (DAGs) representing multi-step AI pipelines without code. Users connect nodes representing data sources, transformations, model invocations, and outputs; the platform compiles these visual definitions into executable workflow specifications that handle data flow, error propagation, and conditional branching between steps.

Native integration layer that abstracts over heterogeneous AI model APIs (OpenAI, Anthropic, Hugging Face, local models) through a unified interface. The platform translates pipeline-level model invocation requests into provider-specific API calls, handling authentication, request/response transformation, rate limiting, and fallback logic across different model families without requiring custom adapter code.

Centralized credential storage system that securely manages API keys, database passwords, and authentication tokens used by pipeline connectors and model providers. Credentials are encrypted at rest, rotated automatically, and accessed by pipelines through secure references rather than hardcoded values. Supports multiple authentication methods (API keys, OAuth, basic auth, custom headers).

Pre-built pipeline templates for common use cases (sentiment analysis, document classification, data enrichment) that users can clone and customize. The platform provides a template marketplace where community members can share templates, with versioning and dependency tracking. Templates include documentation, example inputs/outputs, and configuration guides.

Monitoring dashboard that tracks pipeline health metrics (success rate, average latency, error rate) and enables users to configure alerts based on thresholds or anomalies. The platform collects metrics from all pipeline executions, aggregates them by time window, and sends notifications via email or webhooks when conditions are met. Supports custom metrics from pipeline steps.

Pre-built connectors for common data sources (databases, APIs, cloud storage, data warehouses) that automatically infer schema and handle authentication. When a user connects a data source, the platform introspects the source to discover available tables/fields, generates type information, and exposes this metadata to downstream pipeline steps for validation and transformation planning.

Runtime execution engine that processes pipeline DAGs step-by-step, capturing detailed execution traces including input/output data, latency, errors, and model invocation details at each node. The platform provides a web-based dashboard showing real-time execution status, historical run logs, and performance metrics that enable debugging and optimization without accessing logs directly.

Built-in transformation operators (filtering, mapping, aggregation, type conversion, text processing) that can be inserted into pipelines to clean and reshape data between source and model invocation. These nodes support both visual configuration (for simple transformations) and code-based custom logic (for complex operations), with type validation ensuring data contracts between pipeline steps.

Control flow operators that enable pipelines to branch based on data conditions (if/else logic) and handle errors gracefully through retry policies, fallback steps, and error-specific routing. The platform evaluates conditions at runtime and directs execution to different pipeline paths, with support for timeout handling and dead-letter queues for failed executions.

Version control system for pipeline definitions that tracks changes, enables rollback to previous versions, and manages deployment across environments (dev, staging, production). The platform stores pipeline versions in its database and provides a UI for comparing versions, promoting pipelines between environments, and scheduling deployments with approval workflows.

HTTP endpoint generation that allows external systems to trigger pipeline execution via POST requests with JSON payloads. The platform creates unique webhook URLs for each pipeline, validates incoming requests, maps request body fields to pipeline input parameters, and returns execution results or status asynchronously. Supports authentication via API keys and request signing for security.

Scheduling system that enables pipelines to run on fixed schedules (cron-like expressions) or process large datasets in batches. The platform queues batch jobs, distributes execution across available workers, and provides progress tracking and result aggregation. Supports both time-based triggers (e.g., daily at 2 AM) and data-driven triggers (e.g., when new files appear in S3).

Ability to insert custom JavaScript or Python code into pipelines that executes in a sandboxed runtime environment. The platform provides access to pipeline context (previous step outputs, input parameters) through language-specific SDKs, handles dependency management, and isolates code execution to prevent security issues. Custom nodes are treated as first-class pipeline steps with input/output validation.