Oxylabs

Scrapes any website by executing JavaScript in a headless browser environment before content extraction, enabling access to client-rendered content that static HTML scrapers cannot retrieve. Uses Oxylabs' distributed proxy infrastructure to render pages server-side, returning fully-executed DOM state rather than raw HTML. Supports configurable render timeouts and JavaScript execution policies to balance completeness vs latency.

Circumvents sophisticated anti-scraping defenses (Cloudflare, Akamai, DataDome, etc.) by routing requests through Oxylabs' Web Unblocker proxy network, which maintains residential IP pools and browser fingerprinting to appear as legitimate user traffic. Transparently handles CAPTCHA solving, IP rotation, and challenge page navigation without exposing these details to the caller.

Implements comprehensive error handling for scraping failures, including network errors, authentication failures, parsing errors, and Oxylabs API errors. Returns detailed error messages and diagnostics to help diagnose issues (e.g., 'Cloudflare protection detected', 'CAPTCHA solving failed', 'Invalid URL format'). Includes retry logic for transient failures and graceful degradation when specific features (parsing, rendering) are unavailable.

Supports deployment through multiple distribution methods: Smithery CLI (hosted MCP registry), uvx (Python package execution), npx (Node.js package execution), and local uv development setup. Each deployment method handles dependency installation, credential configuration, and MCP server startup differently, allowing flexibility in deployment environments (cloud, local, containerized).

Scrapes Google Search results pages and parses them into structured JSON containing title, URL, snippet, and metadata for each result. Uses domain-specific parsing logic to extract search result elements from Google's HTML structure, handling pagination and result formatting variations. Integrates with Oxylabs' Web Unblocker to bypass Google's bot detection on search queries.

Scrapes Amazon search results pages and extracts structured product data including ASIN, title, price, rating, and availability status. Uses specialized parsing logic to navigate Amazon's dynamic product listing HTML, handling sponsored results, pagination, and price formatting variations. Integrates Web Unblocker to bypass Amazon's anti-bot protections.

Scrapes individual Amazon product pages and extracts detailed product information including full description, specifications, images, reviews summary, and seller details. Uses specialized parsing to navigate Amazon's complex product page DOM structure, handling variations across product categories (books, electronics, clothing, etc.). Combines JavaScript rendering with domain-specific extraction logic.

Converts raw HTML content into readable Markdown format, removing unnecessary HTML elements, scripts, styles, and formatting noise while preserving semantic structure (headings, lists, links, emphasis). Applies heuristic-based cleaning to extract main content and convert it to Markdown syntax suitable for LLM consumption. Reduces token count compared to raw HTML while maintaining readability.

Extracts and parses website content into structured JSON based on domain-specific extraction rules, identifying key entities (products, articles, listings, etc.) and their attributes from HTML. Uses pattern matching and heuristic-based parsing to recognize common content patterns (product listings, article metadata, pricing tables) and convert them to structured formats. Supports pre-built parsers for common domains (Amazon, Google, etc.) and generic extraction for unknown sites.

Accesses location-specific content versions by routing requests through proxy nodes in different geographic regions, enabling retrieval of geo-restricted content or location-specific pricing/availability. Supports specifying target country/region via parameters, with Oxylabs' proxy infrastructure automatically routing the request through an IP address in that location. Useful for accessing content blocked outside specific regions or retrieving localized pricing.

Exposes web scraping capabilities through the Model Context Protocol (MCP) standard, allowing AI models (Claude, Cursor, etc.) to invoke scraping tools as native functions. Built on FastMCP framework, which handles MCP request/response serialization, tool schema definition, and error handling. Enables AI models to discover available scraping tools, understand their parameters, and invoke them with natural language intent.

Manages Oxylabs API credentials (username/password) securely within the MCP server, handling authentication to Oxylabs Web API and passing credentials transparently to all scraping requests. Supports credential configuration via environment variables or MCP client configuration, with credentials stored in memory during server runtime. Implements error handling for authentication failures and credential validation.