ARC-AGI

Generates and renders abstract visual puzzle tasks as interactive game environments where agents must explore state spaces, plan actions, and achieve goals through a Percept → Plan → Action cycle. Tasks are presented in configurable rendering modes (terminal text-based or programmatic API access) and support memory persistence across action sequences, enabling agents to learn patterns from minimal examples.

Provides a Python SDK (arc-agi package) for local execution of benchmark tasks with configurable rendering modes and performance optimization. The SDK exposes a GameAction class for discrete action specification, an Arcade environment factory for task instantiation, and a scorecard evaluation system. Execution runs entirely client-side without mandatory cloud dependencies, achieving 2000+ FPS when rendering is disabled.

Implements the core agent-environment interaction loop through env.step(action), which executes an action, updates task state, and returns observations. The step function encapsulates the Percept → Plan → Action cycle, enabling agents to iteratively explore tasks and learn patterns. Step returns observation, done flag, and implicit feedback enabling agents to assess action effectiveness.

Provides open-source access to benchmark tasks, evaluation infrastructure, and reference implementations, enabling community-driven research and algorithm development. The benchmark is published on GitHub with MIT license (implied by open-source claim), supporting reproducibility, contribution, and derivative work. Foundation explicitly emphasizes 'open-source ecosystem' and rewards open-source contributions through ARC Prize 2026.

Exposes benchmark tasks and evaluation through a REST API (documented at https://docs.arcprize.org) with API key authentication, enabling remote task access without local installation. The API abstracts task execution and scoring, allowing integration into web-based systems, cloud pipelines, and multi-language environments. Authentication uses API keys (with anonymous access available but limited).

Measures an AI system's ability to recognize and generalize abstract patterns from minimal examples (1-5 training demonstrations) without domain-specific knowledge or pre-training on similar tasks. Evaluation is based on whether agents can infer transformation rules, spatial relationships, and logical operations from limited visual evidence and apply them to novel test cases. This capability directly measures fluid intelligence and learning efficiency rather than memorized knowledge.

Supports agent memory persistence and goal acquisition across action sequences, enabling agents to maintain state, learn from observations, and dynamically discover task objectives. The Percept → Plan → Action cycle allows agents to accumulate knowledge across multiple steps, with memory mechanisms enabling pattern recognition and strategy refinement. Goals are not explicitly provided; agents must infer them from task structure and feedback.

Provides dual rendering modes for task visualization: terminal-based text rendering for human inspection and programmatic access (no rendering) for high-performance evaluation. Terminal mode enables visual debugging and human understanding of task state, while the no-render mode optimizes for throughput (2000+ FPS) by eliminating rendering overhead. Rendering mode is configurable per task instantiation.

Aggregates task performance into a structured scorecard that summarizes agent evaluation results across the benchmark. The scorecard is generated via arc.get_scorecard() and provides aggregated metrics, though the exact structure and metrics are not formally documented. Scorecard enables comparison across agents and tracking of performance progress.

Integrates with the ARC Prize 2026 competition infrastructure, enabling researchers to submit solutions, receive evaluation on held-out test sets, and compete for $2M in prizes. Competition is hosted on Kaggle and provides standardized submission mechanisms, leaderboard tracking, and prize distribution. The foundation rewards open-source contributions and novel algorithmic progress.

Enables task instantiation by task ID (e.g., 'ls20', 'ft09') through the Arcade.make() factory method, abstracting task loading and initialization. Task IDs map to specific puzzle instances in the benchmark, allowing reproducible task selection and batch evaluation. The factory pattern supports configurable rendering modes and other task parameters.

Defines a discrete action space through the GameAction enum, enabling agents to interact with tasks through a fixed set of predefined actions. Actions are specified as enum values (e.g., GameAction.ACTION1) and passed to env.step(), abstracting the underlying action semantics. The action space is task-agnostic, supporting a consistent interface across all benchmark tasks.