Stagehand

Converts natural language commands (e.g., 'click the login button') into browser actions by fusing visual understanding with DOM analysis. The act() primitive uses the LLM to interpret intent, then executes via Playwright's CDP connection with fallback strategies when selectors fail. Implements a hybrid approach where vision provides context and DOM provides precision, enabling resilience to UI changes without brittle selectors.

The extract() primitive uses LLM-guided vision and DOM analysis to pull structured data from web pages according to developer-defined schemas. It combines screenshot analysis with DOM tree traversal to locate and parse data, then validates output against the provided schema. Supports TypeScript/JSON schema definitions for type-safe extraction with automatic validation and error handling.

The browse CLI tool provides command-line access to Stagehand automation without writing code. It implements a daemon architecture where a long-running server manages browser sessions and accepts commands via HTTP API or CLI. Supports session persistence, network capture for debugging, and multi-region routing for cloud execution. Enables non-developers to define automation workflows through CLI commands or YAML configuration.

Stagehand includes a built-in evaluation system for measuring automation success rates, latency, cost, and correctness. Developers define evaluation tasks with expected outcomes, run them against different models/configurations, and get detailed metrics. Supports multiple evaluation categories (navigation, extraction, interaction, reasoning) and integrates with CI/CD for regression testing. Enables data-driven model selection and configuration tuning.

Stagehand implements a comprehensive error handling system that classifies errors into categories (network, LLM, browser, automation logic) and provides recovery strategies. SDK errors include detailed context (page state, action history, error trace) enabling debugging. Built-in retry logic with exponential backoff for transient failures; developers can implement custom error handlers for domain-specific recovery.

Stagehand emits structured events throughout execution (action start/end, LLM calls, errors, cache hits) enabling comprehensive observability. Events include timing, resource usage, and contextual metadata. Integrates with standard logging frameworks and metrics collectors (OpenTelemetry, Datadog, etc.). Developers can subscribe to events for custom monitoring, alerting, or analytics without modifying automation code.

The observe() primitive identifies interactive elements on a page by combining visual analysis with DOM tree inspection. It returns a list of observable elements with their visual properties, accessibility labels, and interaction hints. Uses screenshot analysis to understand visual hierarchy and element prominence, then correlates with DOM structure to provide both visual and programmatic element references.

The agent() system enables autonomous multi-step task execution by combining LLM reasoning with a tool registry (act, extract, observe, custom tools). Agents decompose complex goals into sequences of actions, maintain context across steps, and self-correct using feedback loops. Supports three tool modes: DOM-only (fast, deterministic), Hybrid (vision+DOM), and Computer Use Agent (CUA, full screen control). Implements streaming callbacks for real-time progress visibility and built-in caching for deterministic replay.

The ActCache system records action outcomes and caches them to enable deterministic replay and self-healing. When an action is executed, its result is cached with a hash of the page state; on subsequent runs, if the page state matches, the cached result is returned without re-executing the LLM. If page state diverges, the cache invalidates and the action re-executes. AgentCache extends this to multi-step workflows, caching entire agent execution paths for replay and debugging.

Stagehand abstracts LLM provider differences through a unified LLMClient interface supporting OpenAI, Anthropic, Google, Ollama, and custom providers. Developers specify model preferences via configuration; the framework handles API key management, request formatting, and response parsing. Supports model-specific features (vision, function calling, streaming) with automatic capability detection and graceful degradation when features unavailable.

CUA mode enables agents to control the entire screen using visual coordinates rather than DOM selectors, mimicking human computer use. Agents receive full-page screenshots, reason about visual elements, and issue click/type commands by pixel coordinates. Supports multiple CUA providers (Anthropic, OpenAI, Browserbase) with provider-specific vision models and reasoning capabilities. Enables automation of non-web applications and complex UI patterns that resist DOM-based automation.

Agents can be extended with custom tools beyond the built-in act/extract/observe primitives through a tool registry system supporting the Model Context Protocol (MCP). Developers define custom tools as functions with schema definitions; the agent's LLM can call these tools as part of its reasoning loop. Tools receive agent context (page state, variables) and return results that feed back into agent reasoning, enabling integration with external APIs, databases, or specialized automation logic.

Stagehand abstracts browser connectivity through a CDP (Chrome DevTools Protocol) connection layer supporting both local browsers and Browserbase cloud instances. The V3Context manages page/frame lifecycle, handles connection pooling, and provides unified APIs regardless of execution environment. Developers can switch between local and cloud execution by configuration change; the framework handles session management, browser lifecycle, and network resilience transparently.

Agents support streaming callbacks that emit real-time events during execution: tool calls, observations, reasoning steps, and state changes. Developers can subscribe to these events to build progress UIs, logging systems, or adaptive workflows that respond to agent decisions in real-time. Streaming is provider-dependent (supported by OpenAI, Anthropic, Browserbase CUA); fallback to non-streaming execution if provider doesn't support it.