Firecrawl MCP Server

Scrapes individual web pages via the firecrawl_scrape tool by accepting a URL and optional parameters (formats, wait time, headers), then converts HTML content to clean markdown using Firecrawl's built-in extraction engine. The tool integrates with the @mendable/firecrawl-js client library which handles HTTP transport, DOM parsing, and markdown serialization, returning structured output with metadata (title, description, links, images). Supports both cloud and self-hosted Firecrawl instances through unified configuration.

Scrapes multiple URLs in a single operation via the firecrawl_batch_scrape tool, accepting an array of URLs and shared options, then returns an array of markdown-converted results. The tool leverages Firecrawl's backend batch processing which parallelizes requests across multiple workers, reducing total execution time compared to sequential single-page scrapes. Each URL is processed independently with the same markdown conversion pipeline, and results include per-URL status indicators and error handling.

Supports both Firecrawl cloud API and self-hosted Firecrawl instances through unified configuration via the @mendable/firecrawl-js client library. The API endpoint is configurable via FIRECRAWL_API_URL environment variable; when set to a self-hosted instance URL, all tool calls are routed to that instance instead of the cloud API. Authentication uses the same API key mechanism for both cloud and self-hosted, enabling seamless switching between deployments.

Crawls entire websites starting from a base URL via the firecrawl_crawl tool, which recursively discovers and scrapes all linked pages within the domain. The tool accepts a base URL and optional parameters (max depth, max pages, allowed domains), then returns a structured list of all discovered pages with their markdown content and metadata. Internally, Firecrawl maintains a URL frontier, respects robots.txt, and implements breadth-first traversal with deduplication to avoid revisiting pages.

Monitors the status of in-progress crawl operations via the firecrawl_crawl_status tool, accepting a crawl ID and returning current progress (pages processed, pages remaining, completion percentage), error logs, and partial results. The tool polls the Firecrawl backend API to fetch job state without requiring the client to maintain state; results can be streamed incrementally as pages are discovered, enabling real-time progress updates in long-running crawls.

Extracts structured data from web pages using a JSON schema via the firecrawl_extract tool, which accepts a URL, a schema definition, and optional parameters, then returns parsed data matching the schema. The tool leverages Firecrawl's LLM-powered extraction engine which understands semantic meaning (e.g., 'price' field extracts numeric values even if HTML structure varies), handles missing fields gracefully, and validates output against the schema. Supports complex nested schemas and arrays for extracting lists of items.

Discovers and retrieves web content based on search queries via the firecrawl_search tool, which accepts a search query and optional parameters (number of results, search engine), then scrapes the top results and returns their markdown content. The tool integrates with web search APIs (Google, Bing, or Firecrawl's internal index) to find relevant pages, then automatically scrapes each result without requiring the user to specify URLs. Results include search ranking, relevance scores, and full page content.

Implements automatic retry logic for failed requests via configurable exponential backoff parameters (FIRECRAWL_RETRY_MAX_ATTEMPTS, FIRECRAWL_RETRY_INITIAL_DELAY, FIRECRAWL_RETRY_MAX_DELAY, FIRECRAWL_RETRY_BACKOFF_FACTOR). When a Firecrawl API call fails (timeout, rate limit, transient error), the MCP server automatically retries with increasing delays: delay = min(initial_delay × backoff_factor^attempt, max_delay). Retries are transparent to the client — failures are only reported after all retries are exhausted.

Monitors Firecrawl API credit consumption via built-in tracking that logs warnings and critical alerts when credit levels fall below configurable thresholds (FIRECRAWL_CREDIT_WARNING_THRESHOLD, FIRECRAWL_CREDIT_CRITICAL_THRESHOLD). The MCP server fetches credit balance after each operation and compares against thresholds, emitting structured log messages (warning, critical) without blocking operations. Thresholds are configurable per deployment, enabling different alert levels for development vs production.

Supports multiple communication transports for MCP client connections via configurable transport modes: stdio (default, for CLI/desktop clients), SSE (Server-Sent Events for cloud deployments), and SSE_LOCAL (for local web integration). The transport layer is abstracted by the @modelcontextprotocol/sdk, allowing the same server code to run on different transports by setting environment variables (SSE_LOCAL=true for local, default stdio for CLI). Each transport has different latency, scalability, and deployment characteristics.

Validates and parses MCP tool invocations using JSON schema definitions for each tool's arguments via the @modelcontextprotocol/sdk. Each tool (firecrawl_scrape, firecrawl_crawl, etc.) has a defined schema specifying required/optional arguments, types, and constraints. The MCP server validates incoming tool calls against these schemas before passing to Firecrawl, rejecting invalid calls with structured error messages. Schema validation prevents malformed requests from reaching the Firecrawl API.