Cross Platform Action Execution With Unified Controller Abstraction

via “cross-platform os-level action execution with semantic understanding”

Open-source infrastructure for Computer-Use Agents. Sandboxes, SDKs, and benchmarks to train and evaluate AI agents that can control full desktops (macOS, Linux, Windows).

Unique: Implements OS-specific action handlers that translate semantic action commands into native OS APIs (macOS Quartz events, Linux X11/Wayland input, Windows SendInput), with coordinate mapping that understands UI element positions from VLM output rather than relying on brittle selectors or hardcoded coordinates.

vs others: More robust than selector-based automation (Selenium, UiAutomator) because it uses VLM-driven semantic understanding of UI layout; more portable than OS-specific tools because unified action interface abstracts platform differences.