XHS-Downloader

Parses XiaoHongShu work URLs and extracts structured metadata including title, description, author info, media counts, and engagement metrics. Uses HTTP request interception with custom headers and cookie-based authentication to bypass platform restrictions, then deserializes JSON responses from XHS API endpoints into typed data structures for downstream processing.

Downloads images and videos from XiaoHongShu without platform watermarks by fetching clean media URLs from the platform's CDN, then stores files locally with configurable naming patterns and folder organization. Implements async batch downloading using httpx with connection pooling, progress tracking, and retry logic for failed transfers.

Extracts work URLs in bulk from XiaoHongShu user profiles (published works, favorites, likes), collections, and search results by paginating through API responses and collecting all work IDs. Implements pagination logic with configurable page size and maximum result limits, deduplication of extracted URLs, and progress tracking for long-running extractions. Returns a list of work URLs ready for batch downloading.

Provides multi-language support for CLI, TUI, and API responses through a centralized i18n system that loads language files (JSON) at startup and substitutes localized strings throughout the application. Supports Chinese (Simplified/Traditional) and English with fallback to English if requested language is unavailable. Language selection is configurable via settings.json or environment variables.

Implements a shared async HTTP client using httpx with connection pooling, automatic retry on transient failures (5xx errors, timeouts), exponential backoff, and custom headers (User-Agent, cookies) for XHS API requests. Reuses the same client instance across all requests to maximize connection reuse and minimize overhead. Implements timeout handling and graceful degradation on network errors.

Builds standalone executables for Windows, macOS, and Linux using PyInstaller, bundling Python runtime, dependencies, and application code into a single distributable file. Implements CI/CD workflows (GitHub Actions) that automatically compile executables on each release, with platform-specific optimizations and code signing for macOS. Executables include all required resources (i18n files, config templates) without external dependencies.

Maintains a local SQLite database tracking all downloaded works, including work IDs, metadata snapshots, download timestamps, and file paths. Implements schema migrations for version compatibility, deduplication checks to prevent re-downloading, and query interfaces for filtering by date, author, or content type. Database operations use async SQLite bindings to avoid blocking the main event loop.

Single entry point (main.py) dispatches to five distinct execution modes (CLI, TUI, API Server, MCP Server, UserScript) based on command-line arguments or environment configuration. All modes converge on the shared XHS core class, ensuring consistent business logic while allowing interface-specific input/output handling. Uses a layered architecture where the Manager class handles configuration, authentication, and resource lifecycle across all modes.

Manages XiaoHongShu authentication through persistent cookies stored in configuration, with optional automatic extraction from browser cookies using platform-specific APIs (Windows Registry for Edge/Chrome, macOS Keychain, Linux SQLite). Implements cookie validation, expiration checking, and fallback to manual cookie input if automatic extraction fails. Cookies are encrypted in the settings.json file and decrypted at runtime.

Provides an interactive terminal UI built with the Textual framework, featuring real-time clipboard monitoring for XHS URLs, interactive work preview, selective image index selection, and in-app settings management. Monitors the system clipboard in a background thread, detects XHS URLs, and prompts users to download. Implements keyboard shortcuts for navigation, filtering, and bulk operations on work lists.

Exposes XHS-Downloader functionality via FastAPI REST API server running on port 5556, providing endpoints for work detail retrieval, batch downloads, and history queries. Implements async request handling, request validation using Pydantic models, and JSON response serialization. Supports CORS for cross-origin requests and includes basic rate limiting via request queuing.

Exposes XHS-Downloader as an MCP server compatible with Claude, ChatGPT, and other AI assistants, allowing LLMs to call tools for work extraction, metadata retrieval, and batch downloads. Implements MCP protocol handlers for tool definition, tool invocation, and result serialization. Enables AI agents to autonomously discover and use XHS capabilities through the standard MCP interface.

Injects a Tampermonkey/Greasemonkey UserScript into XiaoHongShu web pages that detects work links and provides download buttons. Operates in two modes: standalone (downloads directly via browser APIs) or server-push (sends download requests to a running XHS-Downloader instance). Implements DOM manipulation to inject UI elements, event listeners for download triggers, and optional WebSocket communication to the backend server.

Provides customizable file naming and folder structure through template variables (work ID, author, title, date, etc.) defined in settings.json. Implements a template engine that substitutes variables at download time, sanitizes file names to remove invalid characters, and creates nested folder hierarchies based on template patterns. Supports conditional folder creation based on content type (image vs. video).