Skales – I built a desktop AI agent a 6-year-old can use

Skales implements a conversational interface that translates plain English instructions into executable desktop actions without requiring technical syntax or command-line knowledge. The system uses an LLM backbone to parse user intent from natural language and map it to underlying system capabilities, abstracting away complexity through a chat-like interaction model designed for non-technical users including children.

Skales coordinates multiple desktop system actions (file operations, application launches, window management, text input) by using an LLM to decompose natural language requests into a sequence of executable steps. The system likely maintains an action registry that maps LLM outputs to concrete system APIs, with error handling and state tracking across multi-step operations.

Skales maintains conversation history and user context across multiple interactions, allowing the LLM to reference previous requests and build on prior actions. The system likely stores conversation state (either in-memory or persisted) and passes relevant context to the LLM on each new request, enabling multi-turn workflows where later actions depend on earlier ones.

Skales implements safety guardrails to prevent harmful or inappropriate actions, likely through a combination of action whitelisting, LLM-level instruction tuning, and runtime validation. The system restricts executable actions to a safe subset and may include content filtering to prevent the agent from executing dangerous system commands or accessing sensitive data.

Skales abstracts OS-specific automation APIs (Windows COM/WinAPI, macOS Accessibility Framework, Linux D-Bus) behind a unified action interface that the LLM can reason about. The system likely uses platform-specific bindings or a compatibility layer to translate high-level action requests into native system calls, enabling the same natural language request to work across different operating systems.

Skales abstracts the underlying LLM provider, allowing users to choose between different models (OpenAI, Anthropic, local LLMs) without changing the agent's behavior. The system likely implements a provider interface that normalizes API calls, response formats, and error handling across different LLM backends, enabling users to swap models based on cost, latency, or privacy requirements.

Skales provides real-time visual feedback on agent actions and maintains detailed execution logs, allowing users (especially children) to understand what the agent is doing and why. The system likely displays action sequences, success/failure status, and reasoning steps in the UI, with persistent logs for debugging and auditing.