Scrapezy

Implements the Model Context Protocol (MCP) as a standardized interface for web scraping operations, allowing LLM agents and applications to invoke scraping capabilities through a schema-based tool registry. The MCP server exposes scraping functions as callable tools with JSON-RPC 2.0 transport, enabling seamless integration with Claude, other LLMs, and MCP-compatible clients without custom API wrappers.

Accepts CSS selectors, XPath expressions, or declarative extraction schemas to target and extract specific HTML elements from web pages. The extraction engine parses the DOM, applies selector queries, and transforms matched elements into structured output, supporting both single-element and multi-element (list) extraction patterns with optional data transformation rules.

Orchestrates a multi-step pipeline that fetches a website, parses its HTML structure, applies extraction rules, and outputs structured datasets in formats like JSON or CSV. The pipeline handles URL normalization, response caching, error recovery, and format conversion, abstracting away the complexity of coordinating fetch, parse, extract, and serialize operations.

Leverages the LLM's understanding of natural language to automatically generate CSS selectors or extraction schemas from human-readable descriptions of desired data. When an LLM agent receives a scraping request, it can interpret the intent (e.g., 'extract product names and prices') and generate appropriate selectors without pre-defined templates, enabling adaptive scraping for novel websites.

Enables LLM agents to autonomously navigate multi-page websites by reasoning about pagination patterns, generating next-page URLs, and iteratively scraping content across pages. The agent can detect pagination links, follow them, and consolidate results from multiple pages into a single dataset, handling common pagination patterns (numbered pages, 'next' buttons, infinite scroll detection).

Implements a caching layer that stores fetched page content and extracted datasets, preventing redundant requests to the same URLs and avoiding duplicate data in output. The cache is keyed by URL and extraction parameters, allowing subsequent requests for the same content to return cached results with configurable TTL and invalidation strategies.

Implements automatic retry mechanisms for failed requests with exponential backoff, handling transient network errors, rate limiting (HTTP 429), and server errors (5xx). The system tracks retry attempts, applies increasing delays between retries, and provides detailed error reporting to the agent, allowing graceful degradation when scraping fails.

Validates extracted data against a defined schema, ensuring that extracted fields match expected types, formats, and constraints. The validation engine checks data types (string, number, date), required fields, value ranges, and custom validation rules, providing detailed error reports for invalid data and optionally filtering or transforming invalid records.