aichat

Abstracts 20+ LLM providers (OpenAI, Anthropic, Claude, Gemini, Ollama, etc.) behind a single Client trait with unified request/response handling. Uses a provider registry pattern loaded from models.yaml that maps provider identifiers to concrete client implementations, enabling seamless provider switching without code changes. Token counting and model selection are handled uniformly across all providers through a centralized model registry system.

Provides an interactive shell interface (REPL) that maintains conversation state across multiple turns, with support for role-based context switching and session persistence. The REPL mode loads configuration from GlobalConfig (wrapped in Arc<RwLock<Config>>), manages message history in memory, and supports commands for switching roles, models, and sessions. Sessions can be saved to disk and resumed later, preserving the full conversation context.

Manages application configuration through YAML files (models.yaml, config.yaml) that define available LLM providers, models, roles, agents, and tools. Configuration is loaded at startup and wrapped in Arc<RwLock<Config>> for thread-safe access across async tasks. The system supports configuration merging from multiple sources (system defaults, user config, environment variables) with clear precedence rules.

Implements token counting for different models to ensure prompts fit within context windows. The system uses model-specific tokenizers (or approximations) to count tokens in messages, truncates long inputs to fit within limits, and provides warnings when approaching context limits. Token counting is integrated into the message building pipeline, ensuring all inputs are validated before sending to the LLM.

Provides a macro system that enables text substitution and templating within prompts and configuration. Macros can reference environment variables, configuration values, or built-in functions (e.g., {{date}}, {{user}}, {{env:VAR_NAME}}). Macros are expanded at runtime before sending prompts to the LLM, enabling dynamic context injection without manual editing.

Provides a CMD mode for single-turn LLM interactions where a prompt is passed as a command-line argument, the LLM generates a response, and the process exits. This mode is optimized for scripting and piping, with minimal overhead and no interactive state management. CMD mode uses the same underlying LLM client and configuration system as REPL mode, ensuring consistent behavior.

Implements a role system where each role encapsulates a set of system instructions, model preferences, and conversation parameters. Roles are defined in configuration files and can be dynamically selected at runtime. The system supports variable substitution within role instructions (e.g., {{date}}, {{user}}) through a dynamic instructions system, enabling context-aware prompting without manual editing.

Implements a Retrieval-Augmented Generation (RAG) system that ingests documents through a multi-format pipeline (text, PDF, markdown, URLs), chunks them using configurable strategies, and stores embeddings in a local vector database. The hybrid search system combines keyword-based BM25 search with semantic vector similarity search to retrieve relevant documents. Retrieved documents are automatically injected into the LLM context before generating responses.

Implements a function calling system that parses LLM-generated function calls from structured output (JSON), validates them against a schema registry, and executes them with error handling and retry logic. The system supports recursive tool calling where the LLM can call tools, receive results, and call additional tools based on those results. Tool definitions are loaded from configuration and support variable substitution and conditional execution.

Processes diverse input types (text, files, URLs, command outputs, stdin) through a unified input pipeline that detects input type, loads content, and normalizes it into a message format. The system supports file references via @ syntax, URL fetching with automatic content extraction, shell command execution with output capture, and stdin piping. Input is tokenized and truncated to fit within model context windows using token counting.

Renders LLM responses in real-time as they stream from the provider, with terminal-aware markdown formatting that includes syntax highlighting for code blocks, bold/italic text, and proper line wrapping. The system detects whether the output is a TTY and applies markdown rendering only when appropriate, falling back to raw text for non-interactive contexts. Streaming is handled asynchronously using tokio to avoid blocking the terminal.

Exposes aichat functionality as an HTTP server with REST endpoints for chat, RAG, function calling, and session management. The server mode uses the same underlying LLM client abstraction and configuration system as CLI/REPL modes, enabling consistent behavior across all interfaces. Requests are processed asynchronously using tokio, with support for streaming responses via Server-Sent Events (SSE) or chunked transfer encoding.

Implements an agent system where complex tasks are decomposed into subtasks defined in configuration files. Each agent has a set of variables (inputs), instructions (system prompt), and a sequence of steps that can reference variables and previous step outputs. The agent executor runs steps sequentially, substituting variables at each step and collecting outputs for use in subsequent steps. Agents support conditional execution and error handling.

Parses CLI arguments to determine execution mode (CMD, REPL, Server), model selection, role selection, and input sources. The parser supports context-aware defaults where missing arguments fall back to configuration file values or environment variables. Argument parsing is done using a custom parser that understands aichat-specific syntax (@ for files, | for commands, http:// for URLs).