leon

Leon processes speech input through local speech-to-text engines (supporting multiple STT backends like Sphinx, Google Cloud Speech, or Azure), converts recognized text to structured intents via a modular skill-matching system, and executes corresponding actions without requiring cloud connectivity. The architecture uses a plugin-based skill loader that maps utterances to Python/Node.js modules, enabling offline operation while maintaining privacy by keeping audio processing local.

Leon implements a skill-based architecture where each capability is a self-contained module (Python or Node.js) that registers itself with a central intent router. Skills declare their trigger phrases, required parameters, and execution logic; the router uses fuzzy string matching or regex patterns to map user utterances to the appropriate skill, then invokes it with extracted parameters. This design enables non-developers to add new capabilities by dropping a skill file into a directory without modifying core agent code.

Leon skills can execute system commands (shell scripts, executables) through a sandboxed execution layer, enabling automation of OS-level tasks like file operations, process management, or system configuration. Skills invoke commands via a wrapper that captures output and errors, returning results to the user. This enables voice control of system administration tasks, file management, and integration with command-line tools.

Leon maintains optional user profiles and skill state (stored in JSON files or external databases) that skills can access during execution. Skills can read user preferences (language, timezone, favorite contacts) and maintain state (reminders, task lists, conversation history) to provide personalized responses. This enables skills to adapt behavior based on user context without requiring explicit parameters in every utterance.

Leon generates spoken responses by routing text through configurable TTS backends (local engines like eSpeak, or cloud APIs like Google Cloud Text-to-Speech, Azure, or Amazon Polly). The TTS layer abstracts backend selection, allowing users to choose between offline synthesis (lower quality, no latency) and cloud synthesis (higher quality, requires API key). Audio output is streamed or buffered to system speakers, with support for multiple voices and languages depending on backend capabilities.

Leon converts audio input to text using pluggable STT backends: offline engines (PocketSphinx, CMU Sphinx) for privacy and zero-latency operation, or cloud APIs (Google Cloud Speech-to-Text, Azure Speech Services, Deepgram) for higher accuracy. The STT layer handles audio format conversion, noise filtering, and streaming transcription, returning recognized text with optional confidence scores. Users configure their preferred backend via environment variables or config files.

Leon maps recognized user utterances to executable tasks by extracting parameters from text using regex patterns or simple NLU heuristics, then invoking the corresponding skill with structured parameters. For example, 'remind me to call John at 3 PM' extracts the action (remind), target (John), and time (3 PM), passing them to a reminder skill. This enables users to trigger complex workflows through natural language without explicit API calls or structured input.

Leon runs as a standalone agent on Windows, macOS, and Linux using Node.js as the core runtime, with Python support for skill execution. The agent loads skills dynamically from a skills directory, manages audio I/O through system APIs, and exposes a local HTTP API for programmatic control. Users can deploy Leon on personal computers, Raspberry Pi, or lightweight servers without cloud infrastructure, maintaining full control over data and execution.

Leon exposes a local HTTP API that allows external applications to trigger skills, query agent status, and manage configuration without using voice. Developers can POST requests with intent names and parameters to invoke skills, GET agent state, or configure TTS/STT backends. This enables integration with web frontends, mobile apps, or other services that need to control the assistant programmatically.

Leon uses environment variables and config files (JSON or YAML) to specify which TTS and STT backends to use, API credentials, language preferences, and voice selections. Users can switch from offline (eSpeak, PocketSphinx) to cloud (Google, Azure, Deepgram) backends by editing config without code changes. This enables different deployment profiles: offline-first for privacy, cloud-based for accuracy, or hybrid for flexibility.

Leon monitors the skills directory for new, updated, or deleted skill files and dynamically loads/unloads them without restarting the agent. Each skill is loaded as an isolated module with its own execution context, allowing developers to iterate on skills rapidly. The skill loader validates skill manifests (trigger phrases, parameters, description) and registers them with the intent router, enabling new capabilities to be added at runtime.

Leon supports multiple languages by allowing skills to define language-specific trigger phrases and responses. When a user speaks in a particular language (detected via STT language setting), Leon routes to language-specific skill variants if available, falling back to default language if not. This enables building multilingual assistants where skills can respond in the user's language without requiring separate agent instances.