llm-code-highlighter

Extracts and highlights essential code elements (function signatures, class definitions, imports, key logic) while removing boilerplate and comments, using a simplified repomap technique adapted from Aider Chat. The tool parses source code into an AST-like representation to identify structural boundaries and preserve semantic relationships, then outputs a condensed version that maintains enough context for LLM analysis without token bloat.

Detects source code language from file extension or content, then applies language-specific parsing rules to identify structural elements (function/class definitions, imports, decorators). Falls back to heuristic-based line filtering for unsupported languages, ensuring graceful degradation across diverse codebases without requiring external parser dependencies.

Estimates token consumption of condensed code using language-model-specific tokenizers (OpenAI, Anthropic, etc.) and provides feedback on compression ratio achieved. Allows developers to tune condensation aggressiveness (preserve more detail vs. maximize compression) based on target token budget, enabling predictable context window usage.

Processes entire directory trees recursively, applying condensation rules to all source files matching specified patterns (glob filters, language filters). Outputs a structured map of condensed files with metadata (original size, condensed size, token count, language), enabling efficient analysis of large monorepos or multi-module projects.

Offers multiple condensation profiles (aggressive, balanced, conservative) that control which code elements are preserved (imports, comments, docstrings, blank lines, etc.). Users can define custom profiles via configuration files, enabling consistent condensation behavior across teams and projects without per-file parameter tuning.

Identifies and extracts import statements, require() calls, and dependency declarations from source code, then maps relationships between modules (which files import which). Outputs a dependency graph or adjacency list that helps LLMs understand module structure and interdependencies without analyzing full file contents.

Parses source code to extract function/method signatures, class definitions, and type annotations, preserving parameter names, return types, and decorators. Outputs a structured list of callable interfaces with optional docstring summaries, enabling LLMs to understand the public API of a module without reading implementation details.

Identifies and selectively removes or preserves comments, docstrings, and documentation blocks based on configurable rules (remove all, keep docstrings only, keep type hints, etc.). Supports multiple comment styles (single-line, block, inline) across languages, enabling fine-grained control over documentation preservation in condensed code.

Removes unnecessary whitespace (blank lines, excessive indentation, trailing spaces) while preserving code structure and readability. Normalizes indentation to a consistent level (spaces or tabs) and collapses multiple blank lines into single lines, reducing token count without affecting code semantics.

Applies heuristic scoring to individual lines of code to determine importance (function definitions score high, blank lines score low, etc.), then filters lines below a configurable threshold. Uses pattern matching to identify structural elements (imports, definitions, key statements) and removes low-value lines (blank lines, comments, verbose formatting) while preserving semantic content.