code2prompt

Recursively discovers files in a codebase while respecting .gitignore rules through native git integration, building an in-memory file tree that filters out ignored paths before processing. Uses the ignore crate to parse .gitignore patterns and applies them during traversal, avoiding unnecessary I/O on excluded directories. This enables developers to automatically exclude vendor directories, build artifacts, and other non-essential files without manual configuration.

Applies glob patterns to filter files discovered during directory traversal, supporting both inclusion and exclusion patterns with explicit user overrides that take precedence over defaults. The filtering engine evaluates patterns in sequence (include patterns first, then exclusions) and allows users to force-include files that would normally be filtered out via CLI flags or configuration. This enables fine-grained control over which files appear in the final prompt without re-running the entire traversal.

Implements a Code2PromptSession struct that maintains state across multiple configuration and generation steps, enabling developers to build multi-step workflows (configure filters, select files, generate prompt) without re-traversing the filesystem. Sessions encapsulate the file tree, token map, configuration, and template state, allowing incremental modifications and multiple prompt generations from the same session. This is particularly useful for interactive workflows where users make multiple selections before final output.

Detects binary files using magic byte analysis (checking file headers for known binary signatures) and handles them safely by either skipping them or base64-encoding them for inclusion in prompts. This prevents binary data from corrupting text-based prompts while preserving the option to include binary metadata if needed. The detection uses heuristics (null bytes, non-UTF8 sequences) to identify binary files with high accuracy.

Organizes files in the generated prompt using customizable sorting strategies (alphabetical, by size, by modification time, by directory depth) to improve readability and enable LLMs to process related files together. Files can be grouped by directory, sorted within groups, and presented in a hierarchical structure that mirrors the filesystem. This enables developers to control how files appear in the prompt without modifying the underlying file tree.

Processes specialized file types (CSV, JSONL, Jupyter notebooks, binary files) into structured text representations suitable for LLM consumption, with format-specific handlers that preserve semantic information. CSV files are converted to markdown tables, JSONL is pretty-printed with indentation, Jupyter notebooks extract code cells and markdown, and binary files are detected and either skipped or base64-encoded. Each processor is modular and can be extended to support additional formats without modifying the core pipeline.

Counts tokens using tiktoken-rs (OpenAI's tokenizer) to track context usage and prevent exceeding LLM context window limits, providing per-file token counts and cumulative totals. The system tracks tokens for file content, templates, and metadata separately, allowing developers to see exactly which files consume the most tokens and make informed decisions about inclusion. A token map is maintained during processing to enable interactive token-aware file selection in the TUI.

Integrates with git to include version control information in prompts, supporting git diffs (staged/unstaged changes), commit logs, and branch comparisons. Developers can include recent commits, changes between branches, or the current diff to provide LLMs with context about recent modifications. This is implemented via git2-rs bindings that query the repository's git objects directly, avoiding shell invocations and enabling cross-platform compatibility.

Generates prompts using Handlebars-style templates that support variable substitution, conditional blocks, and iteration over file lists. Templates are rendered with context variables (codebase structure, file contents, git information) and can include conditional sections (e.g., 'if has_git_info'). This enables developers to create reusable prompt templates that adapt to different codebases without manual editing, with built-in templates for common scenarios (code review, documentation generation, etc.).

Provides a terminal user interface (TUI) built with ratatui using an Elm/Redux architecture for state management, enabling interactive file selection with real-time token counting feedback. Users can navigate a file tree, toggle files on/off, see token impact of each selection, and preview the generated prompt before output. The TUI maintains a Redux-style state machine where user actions (select file, toggle, navigate) dispatch events that update the model and re-render the view.

Exposes the core code2prompt_core library through three distinct interfaces: a CLI binary (via clap argument parsing), Python bindings (via PyO3), and an MCP (Model Context Protocol) server for agentic applications. Each interface wraps the core library with domain-specific concerns (argument parsing for CLI, Python type conversion for SDK, MCP message handling for agents). This architecture allows the same business logic to be consumed by shell scripts, Python applications, and AI agents without duplication.

Supports configuration files (YAML or TOML format) that specify default settings for file filtering, templates, output formats, and token limits, with CLI arguments taking precedence over file-based settings. Configuration files are loaded from standard locations (.code2prompt.yaml, code2prompt.toml) and can be overridden per-invocation via CLI flags. This enables teams to establish project-wide defaults while allowing individual developers to customize behavior without modifying shared configuration.

Routes generated prompts to multiple output destinations (stdout, file, clipboard) with support for different output formats (plain text, JSON, markdown). The output system abstracts destination handling so the same prompt can be written to different targets without code changes. Clipboard output uses the arboard crate for cross-platform compatibility, and file output supports automatic path resolution and directory creation.