Web Search MCP

Performs web searches across three independent search engines (Bing, Brave, DuckDuckGo) with automatic cascading fallback when primary engines fail or return insufficient results. The system queries engines sequentially, aggregating results and applying quality assessment filters to ensure relevance before returning up to 10 ranked results. This architecture eliminates single points of failure inherent in API-dependent search solutions.

Extracts complete page content from multiple search result URLs concurrently using a two-tier strategy: fast HTTP requests with cheerio-based HTML parsing as primary method, automatically falling back to Playwright browser automation for JavaScript-heavy or dynamically-rendered pages. The system manages a pool of up to 3 browser instances with health checking to prevent resource exhaustion while maintaining extraction reliability across diverse page types.

Defines strict TypeScript types for all tool parameters, search results, and extracted content, with runtime schema validation to ensure MCP clients send correctly-formatted requests. The type system includes interfaces for search results, page content, extraction metadata, and configuration, enabling type-safe tool invocation and IDE autocomplete for client developers. Schema validation prevents malformed requests from reaching the extraction pipeline.

Provides a performance-optimized search tool that returns only search engine snippets (titles, URLs, and brief descriptions) without extracting full page content. This tool uses the same multi-engine search infrastructure as the full-search capability but skips the content extraction pipeline entirely, reducing latency by 80-90% and eliminating browser resource consumption. Includes explicit browser cleanup to prevent resource leaks in long-running agent scenarios.

Extracts and returns the complete content from a single specified URL, applying the same dual-strategy extraction pipeline (HTTP+cheerio first, Playwright fallback) as the full-search tool but optimized for direct URL input rather than search results. Preserves page structure, metadata (title, description, author), and content formatting while filtering common boilerplate elements. Useful for agents that need to investigate specific URLs discovered through other means.

Manages a configurable pool of up to 3 Playwright browser instances with automatic health checking, graceful cleanup, and resource leak prevention. The pool implements queue-based request scheduling to prevent browser exhaustion, monitors instance health (detecting crashed or unresponsive browsers), and automatically restarts failed instances. This infrastructure enables concurrent content extraction across multiple pages while maintaining predictable resource consumption in long-running agent scenarios.

Applies configurable quality filters to search results after aggregation from multiple engines, assessing relevance based on query-to-result similarity, content length, and domain reputation heuristics. The system ranks results by relevance score and filters out low-quality matches before returning to the client. Quality thresholds are configurable via environment variables, allowing tuning for different use cases (strict filtering for research vs. permissive for exploration).

Implements the Model Context Protocol (MCP) as a TypeScript server that communicates with MCP clients (Claude Desktop, LM Studio, custom implementations) via JSON-RPC over stdin/stdout. The server exposes three tools (full-web-search, get-web-search-summaries, get-single-web-page-content) as MCP resources with typed schemas, enabling seamless integration with any MCP-compatible client without custom integration code. Handles protocol versioning, error responses, and graceful shutdown.

Provides extensive configurability through environment variables for search timeouts, content extraction timeouts, quality thresholds, browser pool size, result limits, and rate limiting parameters. Configuration is applied at startup and affects all subsequent requests, enabling operators to tune the server for different deployment scenarios (low-latency vs. comprehensive, resource-constrained vs. unlimited) without code changes. Includes sensible defaults for all parameters.

Implements configurable rate limiting to prevent overwhelming search engines with rapid requests, using request queuing and per-engine throttling. The system tracks request rates per search engine and delays requests if thresholds are exceeded, preventing IP blocking or temporary bans. Rate limits are configurable via environment variables and can be tuned based on deployment requirements and search engine policies.

Implements multi-level error handling that gracefully degrades when individual extraction attempts fail: if HTTP extraction fails, automatically falls back to Playwright; if a single page extraction fails, continues processing other pages rather than failing the entire request; if a search engine is unavailable, cascades to the next engine. Errors are logged with context but do not block the overall operation, allowing partial results to be returned even when some components fail.