aichat

Abstracts 20+ LLM providers (OpenAI, Anthropic, Claude, Gemini, Ollama, etc.) behind a single Client trait, enabling seamless provider switching via configuration without code changes. Uses a provider registry pattern with dynamic model loading from models.yaml, handling provider-specific request/response transformations and token counting internally. Supports both cloud and local (Ollama) providers through the same interface.

Implements a role system that encapsulates system prompts, instructions, and behavioral templates as reusable conversation contexts. Roles are stored as YAML configurations and can be dynamically switched during a session, automatically injecting role-specific instructions into the message building pipeline. Supports role variables (e.g., {{language}}, {{tone}}) that are interpolated at runtime, enabling parameterized conversation templates.

Implements a message building pipeline that constructs LLM requests by combining user input, conversation history, role instructions, RAG context, and agent instructions. The system tracks token usage across all components and implements token budget management to ensure requests fit within the LLM's context window. When context exceeds the budget, the system intelligently truncates conversation history while preserving recent messages and system instructions. Token counting is provider-specific and uses provider APIs or local approximations.

Provides a built-in testing framework for validating provider integrations and debugging provider-specific issues. The framework allows developers to test provider connectivity, model availability, function calling support, and streaming behavior without writing external test code. Tests are defined declaratively and can be run via CLI commands, providing detailed output about provider health and capability support.

Implements a macro system that enables users to define reusable command sequences and prompt templates as macros stored in configuration. Macros can reference variables, other macros, and built-in functions, enabling complex prompt composition without manual repetition. Macros are invoked via CLI syntax and are expanded before sending to the LLM, supporting both simple text substitution and complex conditional logic.

Manages conversation sessions as persistent state stored on disk, enabling users to resume multi-turn conversations across CLI invocations. Sessions store message history, role context, model selection, and conversation metadata. The session system uses Arc<RwLock<Config>> for thread-safe state coordination and supports session switching, listing, and deletion via CLI commands. Sessions are serialized to disk and reloaded on startup.

Implements a Retrieval-Augmented Generation (RAG) system that ingests documents (PDFs, text, code, URLs) into a local vector database, then performs hybrid search combining semantic similarity (vector embeddings) and keyword matching to retrieve relevant context. Documents are chunked, embedded using provider-specific embeddings, and indexed for fast retrieval. Retrieved context is automatically injected into prompts before sending to the LLM, enabling knowledge-grounded responses without fine-tuning.

Implements a function calling system that enables LLMs to invoke external tools and functions defined in YAML configuration. When an LLM requests a function call, aichat executes the function (shell commands, API calls, etc.), captures the result, and feeds it back to the LLM for further processing. Supports recursive tool calling where the LLM can chain multiple function calls to accomplish complex tasks. Function schemas are defined declaratively and passed to providers that support function calling (OpenAI, Anthropic).

Provides three distinct operational modes: CMD (one-shot command execution), REPL (interactive read-eval-print loop with session persistence), and Server (HTTP API for remote access). Each mode shares the same underlying LLM client and feature infrastructure but presents different interaction patterns. CMD mode processes a single prompt and exits; REPL mode maintains session state and enables interactive multi-turn conversations; Server mode exposes an HTTP API for programmatic access. Mode selection is determined by CLI arguments at startup.

Processes diverse input types (text, files, URLs, shell command outputs) through a unified input pipeline that automatically detects input type and loads content appropriately. Supports inline text, file paths (with automatic content loading), URLs (with web scraping), shell command execution (backtick syntax), and piped stdin. Input is normalized into a message format that can be combined with RAG context, role instructions, and conversation history before sending to the LLM.

Streams LLM responses token-by-token to the terminal with real-time rendering, applying markdown formatting (syntax highlighting, code blocks, tables) when appropriate. Uses terminal capability detection to determine whether to render markdown or raw text — markdown is rendered only when the terminal supports it. Streaming is implemented via async I/O (tokio) to avoid blocking the CLI. Response rendering respects terminal width and applies color coding for code syntax highlighting.

Implements a dynamic instruction system where agents (specialized AI personas) can have parameterized instructions with variables (e.g., {{language}}, {{domain}}) that are interpolated at runtime. Instructions are stored in YAML configuration and can reference agent variables, enabling flexible behavior customization without code changes. The system supports conditional instructions and instruction composition, allowing agents to adapt their behavior based on context and user-provided parameters.

Implements a centralized configuration system that loads settings from multiple sources (models.yaml, config.yaml, environment variables) with a defined precedence order. Configuration is wrapped in Arc<RwLock<Config>> for thread-safe access across async tasks. The system supports configuration hot-reloading in REPL mode and provides a unified interface for accessing provider settings, model definitions, role configurations, and agent definitions. Configuration changes are coordinated through the RwLock to prevent race conditions.