llama-vscode

Provides real-time inline code suggestions using the Fill-In-Middle pattern, where the LLM predicts code between cursor position and surrounding context. The extension sends the current file content with cursor position to a local llama.cpp server, which generates completions constrained by a configurable max generation time (preventing UI blocking). Suggestions appear as inline overlays in the editor and can be accepted via Tab, Shift+Tab for first line only, or Ctrl+Right for next word.

Dynamically constructs context for completions by combining the current file content with configurable window size around cursor position, plus optional chunks from other open/edited files. The extension maintains a smart context reuse cache to avoid redundant re-computation on low-end hardware. Context scope and cache reuse parameters are user-configurable via settings, allowing developers to trade off suggestion quality vs inference latency.

Provides predefined llama.cpp command configurations optimized for five hardware tiers: >64GB VRAM (Qwen2.5-Coder 30B), >16GB VRAM (7B), <16GB VRAM (3B), <8GB VRAM (1.5B), and CPU-only (0.5B or 1.5B). Each preset includes optimized batch size (-b, -ub), context size (--ctx-size), and cache reuse (--cache-reuse 256) parameters. Users select hardware tier via environment selection, and extension applies preset parameters automatically without manual tuning.

Manages model file storage in OS-specific cache directories: ~/Library/Caches/llama.cpp/ (Mac OS), ~/.cache/llama.cpp (Linux), LOCALAPPDATA (Windows). Models are downloaded from Huggingface or user-provided paths and cached locally to avoid re-downloading. The extension maintains a model registry tracking available models and their locations. Cache directory location is OS-specific and not user-configurable.

Supports code completion and chat for multiple file types including JavaScript, TypeScript, Python, and plaintext. The extension sends file content to llama.cpp without language-specific preprocessing, allowing FIM models to handle language detection and completion. No explicit language detection or syntax-aware parsing documented; completion works uniformly across supported file types.

Includes clipboard or yanked text as part of the context sent to the LLM for completions and chat. This allows users to reference code snippets, documentation, or other text without manually copying into the file. Clipboard content is automatically detected and included in the context window alongside current file and open files.

Provides a conversational chat UI accessible via the Explorer sidebar, allowing users to interact with selected chat models running on the local llama.cpp server. Chat context includes access to current file, open files, and clipboard content. The extension manages model selection per-task (completion vs chat vs embeddings) and supports both predefined models (Qwen2.5-Coder, gpt-oss 20B) and custom models via add/remove/export/import functionality.

Enables Llama Agent functionality for autonomous coding tasks, where the AI can decompose user requests into sub-tasks and execute them with access to MCP (Model Context Protocol) tools. The agent runs locally on the llama.cpp server and can invoke selected MCP tools from VS Code-installed MCP Servers. Documentation indicates support for local models (gpt-oss 20B recommended) but details are incomplete.

Provides a model registry system where users can add/remove/export/import models for different tasks (completion, chat, embeddings, tools). The extension groups models into 'environments' — predefined configurations optimized for specific hardware tiers (>64GB VRAM, >16GB VRAM, <16GB VRAM, <8GB VRAM, CPU-only). Each environment selects appropriate model sizes and llama.cpp parameters (batch size, context size, cache reuse). Predefined models include Qwen2.5-Coder series (30B, 7B, 3B, 1.5B, 0.5B) and gpt-oss 20B.

Integrates with VS Code-installed MCP Servers to expose tools for use by chat and agentic workflows. The extension discovers available MCP tools, parses their schemas, and passes them to the LLM as function-calling definitions. Users manually select which MCP tools to enable per-session. Tools are invoked by the LLM during chat or agent execution with arguments generated by the model based on tool schemas.

Provides multiple keybindings for accepting code suggestions with different granularity levels: Tab accepts full suggestion, Shift+Tab accepts only first line, Ctrl+Right accepts next word. Ctrl+L manually triggers suggestion generation. Ctrl+Shift+M opens the llama-vscode menu. This design allows developers to accept suggestions at the level of detail they need without full acceptance or rejection.

Provides one-click installation of llama.cpp server via VS Code command palette ('Install/Upgrade llama.cpp'). On Mac and Windows, the extension automatically downloads and installs appropriate binaries. On Linux, users must manually install from source or binaries. The extension manages llama.cpp server lifecycle (start/stop) and exposes configuration options for batch size, context size, and cache reuse parameters via llama.cpp CLI flags.

Displays extension status in VS Code status bar (bottom right) with clickable access to the llama-vscode menu. Status bar shows current model/environment selection and server status. Clicking status bar or pressing Ctrl+Shift+M opens menu for model selection, environment switching, and configuration. This provides quick access to extension controls without opening settings or command palette.

Provides a dedicated Explorer sidebar panel for Llama Agent functionality, displaying agent tasks, execution status, and results. The sidebar UI allows users to initiate agentic workflows, monitor execution progress, and view agent-generated code changes. Integration with VS Code's Explorer sidebar keeps agent workflows visible alongside file tree and other sidebar panels.