promptflow

Enables declarative definition of LLM application workflows using YAML (flow.dag.yaml) that specify a directed acyclic graph of nodes representing LLM calls, prompts, and custom Python functions. The execution engine parses the YAML, validates node dependencies, and executes nodes in topological order with automatic input/output mapping between connected nodes. Supports conditional branching, loops, and dynamic node instantiation through template variables.

Allows developers to define flows as Python functions or classes decorated with @flow and @tool decorators, enabling programmatic control flow with full Python expressiveness. The framework introspects function signatures to automatically extract input/output schemas, handles dependency injection of connections and tools, and executes flows with the same observability and tracing infrastructure as YAML-based DAG flows. Supports async/await patterns for concurrent execution.

Provides comprehensive command-line interface for flow operations including creation, testing, execution, and deployment. CLI commands enable developers to test flows locally, run batch evaluations, manage connections, and deploy to cloud platforms. Integrates with VS Code extension for IDE-based flow development and visualization.

Maintains a persistent record of all flow executions (runs) including inputs, outputs, execution time, and resource usage. Runs can be queried, compared, and visualized to understand flow behavior over time. Supports local SQLite storage for development and Azure ML backend for production, enabling run data to be accessed across environments.

Supports processing of images and documents within flows, including image loading, resizing, format conversion, and OCR for text extraction. Integrates with vision LLM models (GPT-4V, etc.) for image understanding tasks. Handles various input formats (PNG, JPEG, PDF) and automatically manages image encoding for LLM APIs.

Provides deep integration with Azure ML platform enabling flows to be executed on cloud compute clusters, stored in Azure ML registries, and deployed as managed endpoints. Handles authentication, compute resource management, and integration with Azure ML monitoring and governance tools. Enables seamless transition from local development to cloud production.

Introduces a lightweight .prompty file format that bundles prompt templates, LLM configuration (model, temperature, max_tokens), and Python code in a single file for simple LLM interactions. The format uses YAML frontmatter for metadata and configuration, followed by Jinja2 template syntax for the prompt, enabling quick iteration on prompts without managing separate files. Prompty files can be executed directly via CLI or imported as flows.

Provides pre-built tool nodes for common LLM providers (OpenAI, Azure OpenAI, Anthropic, Ollama) with standardized interfaces that abstract provider-specific API differences. Tools handle authentication via connection objects, parameter validation, token counting, and response parsing. Developers can reference these tools in flows without implementing provider-specific logic, and the framework automatically manages API calls, retries, and error handling.

Enables developers to wrap Python functions as reusable tools using @tool decorator, which automatically generates JSON schemas from function signatures for input validation and type coercion. Tools are registered in a tool registry that can be referenced in flows, supporting both synchronous and asynchronous functions. The framework validates inputs against schemas before execution and provides type hints for IDE autocomplete.

Supports executing flows over large datasets by processing multiple input rows in parallel, with configurable batch sizes and worker processes. The batch executor reads input data from JSONL files, distributes rows across worker processes, and aggregates results into output files. Integrates with Azure ML for distributed cloud execution, enabling horizontal scaling without code changes.

Provides a framework for defining evaluation flows that measure quality of primary flow outputs using custom metrics (accuracy, relevance, safety, etc.). Evaluation flows are themselves flows that take primary flow outputs as inputs and produce metric scores. The framework aggregates metrics across batch runs and generates quality reports, enabling systematic comparison of flow variants.

Instruments flow execution to capture detailed traces of all operations including LLM calls, tool invocations, and custom code execution. Uses OpenTelemetry standards to emit traces that can be exported to various backends (Azure Application Insights, Jaeger, etc.). Captures inputs, outputs, latency, token usage, and errors for each operation, enabling post-execution analysis and debugging.

Provides a centralized connection management system where API keys, connection strings, and other credentials are stored securely and referenced by name in flows. Connections are environment-specific (local, cloud) and can be managed via CLI or SDK. Supports multiple connection types (OpenAI, Azure OpenAI, SQL, custom) with provider-specific configuration validation.

Enables deployment of flows as REST APIs that can be invoked via HTTP requests. The serving layer automatically generates OpenAPI schemas from flow input/output definitions, handles request validation, and manages concurrent requests. Supports deployment to various platforms including local Flask/FastAPI servers, Azure Container Instances, and Kubernetes.