Firecrawl MCP Server

Scrapes a single URL and converts HTML content to clean markdown using Firecrawl's content extraction pipeline. The firecrawl_scrape tool accepts a URL and optional parameters (formats, headers, wait time, screenshot capability) and returns structured markdown output with automatic cleanup of boilerplate, navigation, and ads. Implements MCP tool handler pattern that marshals arguments through the @mendable/firecrawl-js client library to Firecrawl's backend processing engine.

Scrapes multiple URLs in a single operation using Firecrawl's batch processing pipeline. The firecrawl_batch_scrape tool accepts an array of URLs and shared options, submitting them to Firecrawl's backend which processes them in parallel and returns an array of markdown-converted content objects. Implements batching through the @mendable/firecrawl-js client's batch method, which handles request queuing, parallel execution, and result aggregation without requiring client-side coordination.

Packages the Firecrawl MCP server as a Docker container with environment-based configuration, enabling deployment to containerized infrastructure (Kubernetes, Docker Compose, cloud platforms). The Dockerfile builds a Node.js runtime with the server code and exposes configuration through environment variables, allowing operators to deploy without modifying code. Supports both cloud and self-hosted Firecrawl instances through configuration.

Registers the Firecrawl MCP server in the Smithery registry, enabling one-click installation and discovery through Smithery's MCP client marketplace. The server is published to Smithery with metadata (description, tags, configuration schema) allowing users to discover and install it without manual setup. Smithery handles server distribution, version management, and client integration.

Supports connecting to self-hosted Firecrawl instances in addition to Firecrawl's cloud service through configurable API endpoint. The FIRECRAWL_API_URL environment variable allows operators to specify a custom Firecrawl endpoint, enabling deployment scenarios where Firecrawl runs on-premises or in a private cloud. The @mendable/firecrawl-js client library handles endpoint abstraction, routing all API calls to the configured endpoint.

Discovers all URLs within a website by crawling from a base URL and building a sitemap-like structure. The firecrawl_map tool accepts a base URL and optional parameters (max depth, include patterns, exclude patterns) and returns a hierarchical array of discovered URLs with metadata about page structure. Uses Firecrawl's crawler to traverse internal links up to specified depth, filtering by inclusion/exclusion patterns, and returns the complete URL graph without fetching full page content.

Crawls an entire website and extracts content from all discovered pages in a single asynchronous operation. The firecrawl_crawl tool accepts a base URL and options (max pages, allowed domains, exclude patterns, scrape options) and returns a crawl ID for polling. The crawler discovers URLs, extracts markdown content from each page, and stores results server-side. Clients poll firecrawl_crawl_status to retrieve results as they complete, implementing an async job pattern rather than blocking until completion.

Polls the status of an in-progress or completed website crawl and retrieves extracted content. The firecrawl_crawl_status tool accepts a crawl ID and returns current progress (pages crawled, pages remaining, completion percentage), status state (running/completed/failed), and paginated results. Implements polling pattern where clients repeatedly call this tool with the same crawl ID to check progress and incrementally retrieve content as pages are processed, supporting streaming-like result consumption.

Abstracts communication between MCP clients and the Firecrawl server across multiple transport modes (stdio, SSE local, SSE cloud) using the @modelcontextprotocol/sdk. The server implements the MCP specification with tool definitions, argument schemas, and response marshaling, allowing any MCP-compatible client (Claude Desktop, custom agents, Smithery) to invoke Firecrawl tools without transport-specific code. Transport mode is configured via environment variables (SSE_LOCAL, SSE_CLOUD) and automatically selected at startup.

Implements automatic retry logic for failed Firecrawl API calls using exponential backoff with configurable parameters. The retry mechanism is configured via environment variables (FIRECRAWL_RETRY_MAX_ATTEMPTS, FIRECRAWL_RETRY_INITIAL_DELAY, FIRECRAWL_RETRY_MAX_DELAY, FIRECRAWL_RETRY_BACKOFF_FACTOR) and automatically retries transient failures (network errors, rate limits, timeouts) without client intervention. Each retry doubles the delay (or uses custom backoff factor) up to maximum delay, then gives up and returns error.

Monitors Firecrawl account credit balance and emits warnings/alerts when balance falls below configurable thresholds. The monitoring is configured via environment variables (FIRECRAWL_CREDIT_WARNING_THRESHOLD, FIRECRAWL_CREDIT_CRITICAL_THRESHOLD) and checks credit balance after each operation, logging warnings to stderr when thresholds are crossed. Enables operators to detect credit exhaustion before it causes service disruption.

Extracts structured data from web pages using a provided JSON schema, returning parsed objects instead of raw markdown. The firecrawl_extract tool accepts a URL, a JSON schema defining desired fields, and optional parameters, and returns extracted data conforming to the schema. Uses Firecrawl's LLM-based extraction engine to identify and parse relevant content from the page, handling variations in page structure and content format automatically.

Searches the web for URLs matching a query and returns ranked results with relevance scores. The firecrawl_search tool accepts a search query and optional parameters (number of results, search type) and returns an array of URLs ranked by relevance. Integrates with web search APIs to discover relevant pages without requiring a known base URL, enabling agents to find sources for research or fact-checking.