tree-sitter-based code definition extraction with language-specific query files
Extracts function, class, module definitions and reference calls from source code using Tree-sitter parsers with language-specific query files (@definition.function, @definition.class, @reference.call). The system maintains a Tag namedtuple structure (rel_fname, fname, line, name, kind) that captures extracted code entities with their locations and types. This enables structurally-aware parsing across 40+ languages without regex-based heuristics, producing precise AST-based extraction that preserves semantic relationships.
Unique: Uses Tree-sitter AST parsing with language-specific query files (get_tags_raw method in repomap_class.py) instead of regex or heuristic-based extraction, enabling structurally-aware definition and reference extraction across 40+ languages with consistent semantics. The Tag namedtuple structure preserves full context (relative filename, absolute filename, line number, entity name, entity kind) for downstream processing.
vs alternatives: More accurate than regex-based code extraction and faster than LSP-based approaches because it parses locally without network overhead; more portable than language-specific parsers because Tree-sitter provides unified interface across languages.
pagerank-based code importance ranking with dependency graph analysis
Analyzes code dependency graphs using PageRank algorithm to rank files and functions by importance, identifying which components are most central to understanding the repository. The system builds a directed graph from function calls and class references extracted by Tree-sitter, then applies iterative PageRank computation to assign importance scores. This graph-based approach recognizes that frequently-called functions and heavily-referenced classes are more important for LLM context than isolated utility functions.
Unique: Applies PageRank algorithm (from Aider.chat) to code dependency graphs to rank importance, treating the codebase as a directed graph where edges represent function calls and class references. This graph-based approach identifies central components more accurately than heuristics like file size or modification time, and integrates seamlessly with the Tree-sitter extraction pipeline.
vs alternatives: More sophisticated than simple heuristics (file size, recency) because it understands code structure; more efficient than full semantic analysis because it operates on extracted call graphs rather than re-parsing code.
token-optimized context window packing with binary search
Intelligently selects and formats code content to fit within LLM context windows using binary search over token counts. The try_tags() function performs binary search on ranked code entities, progressively including more code while monitoring token consumption via tiktoken or equivalent tokenizer. This ensures the output respects token limits while maximizing the amount of relevant code included, formatting results with function prototypes and file relationships in order of PageRank importance.
Unique: Uses binary search (try_tags function in repomap_class.py) to efficiently pack code into token-limited context windows, iteratively including ranked entities while monitoring token consumption. This approach balances code coverage with token constraints more efficiently than greedy selection, and integrates with the PageRank ranking to ensure most-important code is included first.
vs alternatives: More efficient than greedy token packing because binary search finds optimal cutoff point; more flexible than fixed-size summaries because it adapts to available token budget; more intelligent than random sampling because it respects PageRank importance ordering.
persistent disk-based caching with file modification tracking
Caches extracted code tags and PageRank computations to disk using diskcache library, with automatic invalidation based on file modification times. The load_tags_cache() and save_tags_cache() methods manage persistent storage, checking file mtimes to determine cache validity. This avoids re-parsing and re-ranking unchanged files across multiple invocations, significantly accelerating repeated analyses of the same codebase while ensuring cache freshness when files change.
Unique: Implements persistent caching with file modification time tracking (load_tags_cache/save_tags_cache in repomap_class.py) using diskcache, automatically invalidating cache entries when source files change. This approach avoids expensive re-parsing and re-ranking while maintaining correctness across tool invocations.
vs alternatives: More efficient than in-memory caching because it persists across process invocations; more accurate than time-based cache expiration because it tracks actual file changes; more practical than no caching because it significantly speeds up repeated analyses.
command-line interface for on-demand repo map generation
Provides a CLI tool that accepts repository paths and optional configuration parameters, generating formatted repo maps for immediate use or piping to other tools. The CLI interface wraps the RepoMap class, exposing methods like get_repo_map() with configurable token limits, file filters, and output formats. This enables developers to quickly analyze any codebase from the terminal, integrate RepoMapper into shell scripts, or use it as a preprocessing step for other tools.
Unique: Exposes RepoMap core engine as a CLI tool that wraps the RepoMap class methods (get_repo_map, get_ranked_tags_map_uncached) with command-line argument parsing, enabling direct terminal access to repo map generation without writing Python code. Supports piping output to other tools and integration into shell scripts.
vs alternatives: More accessible than Python API for shell-based workflows; more flexible than web-based tools because it runs locally without network overhead; more scriptable than GUI tools because it integrates with standard Unix pipes and redirection.
model context protocol (mcp) server for continuous integration with llm tools
Implements an MCP server that exposes RepoMapper functionality as callable tools for LLM-powered applications and agents. The MCP server wraps RepoMap methods as MCP tools with standardized schemas, enabling seamless integration with Claude, other LLMs, and MCP-compatible applications. This allows LLMs to request repo maps on-demand during conversations, with automatic caching and incremental updates, without requiring the LLM to understand RepoMapper's internal architecture.
Unique: Implements MCP server interface that exposes RepoMap functionality as standardized callable tools, enabling LLMs and MCP-compatible applications to request repo maps on-demand. This architecture allows seamless integration with Claude and other LLM-powered tools without requiring them to understand RepoMapper's internal implementation.
vs alternatives: More integrated than CLI-based approaches because LLMs can call it directly; more standardized than custom API endpoints because it uses MCP protocol; more flexible than hardcoded context because it allows dynamic repo map generation during conversations.
language-agnostic code entity extraction with configurable language support
Supports code extraction across 40+ programming languages through Tree-sitter grammar integration, with a pluggable architecture for adding new languages. The system maintains language-specific query files that define how to extract definitions and references for each language, allowing consistent extraction semantics across Python, JavaScript, TypeScript, Go, Rust, Java, C++, and others. New language support can be added by defining query files without modifying core extraction logic.
Unique: Provides pluggable language support through Tree-sitter query files, enabling extraction across 40+ languages with consistent semantics. New languages can be added by defining query files without modifying core extraction logic, making the system extensible for emerging languages.
vs alternatives: More flexible than language-specific tools because it supports multiple languages with unified interface; more maintainable than hardcoded language support because query files are declarative; more future-proof because it can easily add new languages as Tree-sitter grammars improve.
file prioritization and filtering with configurable rules
Allows users to specify which files should be included or excluded from repo map generation through configurable filtering rules. The system supports patterns for ignoring common non-essential files (node_modules, .git, __pycache__, etc.) and prioritizing important files (main entry points, configuration files). This enables focused analysis of relevant code while reducing noise from dependencies and generated files, with rules that can be customized per repository.
Unique: Provides configurable file filtering and prioritization rules that operate at the file level before extraction, allowing users to exclude dependency directories and generated files while prioritizing important source files. This reduces noise in repo maps and focuses analysis on relevant code.
vs alternatives: More flexible than hardcoded exclusion lists because rules are configurable; more efficient than post-processing filtering because it excludes files before expensive parsing; more practical than no filtering because it handles common patterns (node_modules, .git, etc.).
+1 more capabilities