Prompt Flow

Enables users to construct directed acyclic graph (DAG) pipelines through a dual-mode editor: a visual node-and-edge canvas for drag-and-drop composition, and a YAML-based `flow.dag.yaml` file for declarative pipeline definition. The visual editor generates and synchronizes with the underlying YAML representation, allowing both graphical and text-based editing modes. Nodes represent LLM calls, tool invocations, or Python functions; edges define data flow between nodes. The extension parses the YAML DAG structure and renders it as an interactive graph in the sidebar and editor overlay.

Executes DAG-based flows within a selected local Python interpreter, leveraging the VS Code Python extension to discover and manage Python environments. The extension invokes the promptflow SDK to parse the flow.dag.yaml, instantiate nodes (LLM calls, tools, Python functions), and execute the DAG sequentially or in parallel based on dependencies. Debug mode (F5) attaches a debugger to the execution context, enabling breakpoints and step-through inspection. Test execution (Shift+F5) runs predefined test cases against the flow and reports pass/fail results.

Tracks all flow executions (runs) with detailed metadata including inputs, outputs, execution time, token usage, and error information. Runs are stored in a run database (local or Azure) with full artifact storage (logs, traces, intermediate results). The run dashboard visualizes execution history, enables filtering and comparison across runs, and displays detailed execution traces with node-level granularity.

Integrates with CI/CD pipelines (GitHub Actions, Azure Pipelines) to automatically run flows against test datasets, compute evaluation metrics, and enforce quality gates based on metric thresholds. Provides CLI commands for batch execution, evaluation, and result reporting. Supports pull request workflows where new prompt versions are tested against baselines before merging.

Introduces a .prompty file format that combines prompt template, model configuration, and metadata in a single YAML/JSON file. Prompty files can be executed directly or embedded in flows, enabling lightweight prompt experimentation without full flow definitions. Supports variable substitution, model selection, and hyperparameter configuration within the file.

Supports processing multimedia inputs (images, PDFs, documents) within flows through built-in tools for image analysis, OCR, and document parsing. Images can be passed to vision-capable LLMs (GPT-4V, Claude), and documents are automatically converted to text or embeddings. The framework handles format conversion, size optimization, and error handling transparently.

Integrates with Azure ML workspaces for cloud-based flow execution, enabling managed compute, auto-scaling, and enterprise features (RBAC, audit logging). Flows can be registered as Azure ML models, deployed as endpoints, and monitored with Azure's observability tools. Supports both batch execution on compute clusters and real-time serving on managed endpoints.

Provides a sidebar-based connection manager that abstracts credential handling for external services (LLM APIs, databases, etc.). Connections are defined as YAML files with key-value pairs for authentication details (API keys, endpoints, OAuth tokens). The extension stores connection definitions locally in the workspace, with inline YAML comments providing configuration guidance. When a flow node references a connection by name, the extension resolves the connection YAML at runtime and injects credentials into the node's execution context. The sidebar UI allows users to create, edit, and delete connections without manual YAML editing.

Enables users to define evaluation metrics and run variant tests against flows to measure performance and correctness. The extension supports creating evaluation flows that assess outputs from a main flow (e.g., comparing LLM-generated text against ground truth using metrics like BLEU, similarity scores, or custom Python functions). Variant testing allows users to test multiple versions of a flow (e.g., different prompts, model parameters) against the same test dataset and compare results side-by-side. Evaluation results are aggregated and displayed in a results dashboard within the extension.

Abstracts LLM provider APIs and tool integrations through a node-based system where each node encapsulates a specific operation (LLM call, tool invocation, Python function). Nodes are configured with provider-agnostic parameters (e.g., model name, temperature, max_tokens) and reference connections for credentials. The extension resolves the connection type at runtime and routes the node execution to the appropriate provider SDK (OpenAI, Azure OpenAI, Anthropic, etc.). Built-in tool nodes provide access to common operations (web search, code execution, database queries) without requiring custom Python code.

Provides a VS Code sidebar pivot labeled 'Prompt flow' that serves as a project hub for managing flows, connections, and dependencies. The sidebar displays a hierarchical view of flows in the workspace, quick-access buttons for common tasks (dependency installation, connection creation), and sections for browsing available connections. Right-click context menus on flows and connections enable actions like create, edit, delete, and rename. The sidebar integrates with VS Code's file explorer, allowing users to navigate and open flow files directly.

Adds inline code lens actions to flow.dag.yaml files in the VS Code editor, providing quick access to common operations without context menu navigation. Code lens actions include 'Visual editor' (opens the visual DAG editor), 'Debug' (F5 equivalent), 'Run tests' (Shift+F5 equivalent), and 'Create connection' (for connection YAML files). These actions are rendered as clickable links above the YAML content, enabling one-click access to flow operations from the text editor.

Integrates with the VS Code Python extension to discover installed Python interpreters and manage promptflow SDK dependencies. The extension detects available Python environments (system Python, virtual environments, conda environments) and allows users to select a target environment for flow execution. A 'Quick access' button in the sidebar triggers dependency installation, which runs `pip install promptflow promptflow-tools` in the selected environment. The extension validates that required SDKs are installed before executing flows and provides error messages if dependencies are missing.

Provides integration points with Azure AI services, enabling flows to be deployed to Azure AI platforms and leverage Azure-hosted LLM models. The extension supports Azure OpenAI connections, allowing flows to call Azure-hosted GPT models. While specific cloud deployment mechanisms are not documented, the architecture suggests flows can be packaged and deployed to Azure AI without significant modification. Azure integration is positioned as a primary use case in the product description, indicating native support for Azure authentication, model selection, and resource management.

Allows users to define custom Python function nodes within flows, enabling arbitrary Python code execution as part of the DAG. Custom nodes are defined in flow.dag.yaml with a reference to a Python function (e.g., `my_module.my_function`), and the extension executes the function with inputs from upstream nodes. Users can edit custom node code directly in the VS Code editor, and the extension validates Python syntax and function signatures. Custom nodes support input/output type hints, enabling type checking and IDE autocomplete for node connections.