Semgrep CLI

Semgrep's core scanning engine uses tree-sitter parsers to build abstract syntax trees (ASTs) for 30+ programming languages, then applies user-defined pattern rules against these ASTs to detect code anomalies. The OCaml-based semgrep-core performs the computationally intensive pattern matching via RPC from the Python CLI, enabling language-agnostic rule definitions that work across syntactically different codebases without regex fragility. Patterns are matched structurally rather than textually, allowing rules to capture semantic intent (e.g., 'any function call to dangerous_api()' regardless of whitespace or formatting).

Semgrep performs intra-procedural (single-function) taint tracking in the Community Edition by tracing how untrusted data (sources like user input) flows through variables and function parameters to dangerous sinks (like SQL queries or command execution). The taint engine marks data as 'tainted' at source points, propagates taint through assignments and function calls within a function scope, and flags violations when tainted data reaches a sink without sanitization. The Pro Engine extends this to cross-function and cross-file dataflow, reducing false positives by ~25% and increasing true positives by ~250% through improved reachability analysis.

Semgrep supports local-only scanning via `semgrep scan` command, which runs entirely on the developer's machine without cloud dependencies. The local scan uses local rule files or fetches rules from the Semgrep Registry (requires network access). For teams using Semgrep App, the local scan can optionally authenticate to fetch organization policies and enable finding deduplication, but this is optional. The Python CLI orchestrates the workflow, calling semgrep-core for analysis and optionally uploading findings to Semgrep App for triaging.

Semgrep optimizes scanning performance through parallel processing (scanning multiple files concurrently) and incremental analysis (only re-scanning changed files in CI/CD). The Python CLI distributes files across multiple worker processes, each calling semgrep-core to analyze a subset of files. For CI/CD, Semgrep can fetch the list of changed files from Git and only scan those, significantly reducing scan time on large codebases. The OCaml core is designed for single-file analysis, enabling efficient parallelization without synchronization overhead.

Semgrep provides an MCP (Model Context Protocol) server that enables integration with IDEs and editors (VS Code, Neovim, etc.) for real-time scanning and inline findings. The MCP server exposes Semgrep's scanning capabilities as a standardized interface, allowing IDE plugins to invoke scans, fetch findings, and display them inline without embedding Semgrep directly. The server handles authentication, rule management, and finding formatting, providing a clean abstraction for IDE integration.

The `semgrep ci` command integrates Semgrep into CI/CD pipelines by authenticating to semgrep.dev, uploading scan findings, comparing against baseline scans, and enforcing organization-wide policies. The CI mode fetches rules from the Semgrep App (centralized policy management), applies them to the codebase, and blocks merges or deployments if findings violate configured severity thresholds or policy rules. The Python CLI orchestrates this workflow via RPC calls to semgrep-core for analysis, then communicates findings back to the Semgrep App API for deduplication, triaging, and historical tracking.

Semgrep rules are defined in YAML or JSON with a declarative syntax that specifies patterns (what code to match), metadata (severity, CWE, OWASP category), and actions (report, fix, or suppress). The rule engine supports multiple pattern types: simple string matching, regex, AST patterns (e.g., 'any function call to X'), and metavariable binding (e.g., 'capture variable $VAR and ensure it's sanitized'). Rules are human-readable and version-controllable, enabling security teams to collaborate on rule development without writing code. The Python CLI parses rules and passes them to semgrep-core for compilation and execution.

Semgrep supports incremental scanning by comparing current scan results against a baseline (previous scan) to surface only new or fixed findings, reducing alert fatigue in CI/CD. The baseline is stored in Semgrep App and includes finding fingerprints (hash of file, line, rule, and matched text) to deduplicate identical findings across scans. When a finding is triaged or suppressed in the App, subsequent scans automatically filter it out, enabling teams to focus on genuinely new issues. The Python CLI handles baseline retrieval and comparison logic, while the OCaml core performs the actual scanning.

Semgrep's configuration resolver loads rules from multiple sources: local YAML files, the community Semgrep Registry (via HTTP), and organization policies from Semgrep App. The Python CLI resolves rule paths (e.g., `p/owasp-top-ten`, `p/security-audit`) to fetch rule definitions from the Registry or App, then passes them to semgrep-core for compilation. Configuration can be specified via CLI flags, `.semgrep.yml` files in the repository, or organization policies in Semgrep App. The resolver handles rule versioning, caching, and conflict resolution when multiple sources define overlapping rules.

Semgrep supports multiple output formats to integrate with different CI/CD tools and dashboards: JSON (for programmatic processing), SARIF (for GitHub Security tab, GitLab SAST, and other SAST dashboards), plain text (for console output), and table format (for human-readable summaries). The Python CLI handles output formatting after semgrep-core returns findings, allowing findings to be piped to downstream tools or stored in artifact repositories. SARIF output includes rich metadata (rule definitions, code snippets, severity levels) for visualization in GitHub Advanced Security and other platforms.

Semgrep includes a secrets detection capability (available in Semgrep App) that identifies hardcoded credentials, API keys, and tokens using pattern matching combined with semantic validation and entropy analysis. The detector recognizes common secret patterns (AWS keys, GitHub tokens, private keys) and validates them against known formats and checksums to reduce false positives. Entropy analysis detects high-entropy strings that may be secrets even if they don't match known patterns. The Pro Engine extends this with reachability analysis to determine if secrets are actually exposed (e.g., committed to a public repository or logged).

Semgrep Assistant (available in Semgrep App) uses AI to generate automated remediation suggestions for detected findings. When a vulnerability is found, the Assistant analyzes the code context and generates a fix suggestion (e.g., 'add input validation here', 'use parameterized queries instead of string concatenation'). The suggestion is displayed in the Semgrep App dashboard and can be applied directly or reviewed before merging. The Assistant is powered by LLMs and trained on common vulnerability patterns and fixes.

Semgrep's supply chain scanning (available in Semgrep App) detects vulnerable dependencies by scanning lock files (package-lock.json, Gemfile.lock, requirements.txt, etc.) and comparing them against a vulnerability database. The Pro Engine extends this with reachability analysis to determine if a vulnerable dependency is actually used in the codebase, reducing false positives from unused transitive dependencies. The scanner identifies the vulnerable function/class and traces whether it's called from application code, enabling teams to prioritize remediation based on actual exposure.