Trajectory Recording And Replay For Debugging And Evaluation

via “trajectory recording and agent execution tracing with hud visualization”

Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).

Unique: Implements a trajectory recording system that captures complete execution context (screenshots, action commands, VLM reasoning, timestamps, environment state) with HUD integration for visual overlay of agent actions on screenshots. Supports multiple export formats for compatibility with OSWorld and other benchmarking frameworks.

More comprehensive than simple logging because it captures visual context and enables deterministic replay; HUD visualization provides better debugging UX than text-only logs, while trajectory export enables standardized benchmarking vs. proprietary evaluation formats.