Sqlite Based Job Deduplication And Persistence

via “sqlite-based download history and metadata persistence”

小红书（XiaoHongShu、RedNote）链接提取/作品采集工具：提取账号发布、收藏、点赞、专辑作品链接；提取搜索结果作品、用户链接；采集小红书作品信息；提取小红书作品下载地址；下载小红书作品文件

Unique: Integrates async SQLite operations into the main event loop using aiosqlite, enabling non-blocking database queries during batch downloads while maintaining ACID guarantees for deduplication checks and metadata snapshots.

vs others: Async SQLite integration prevents blocking the download pipeline on database writes, while local persistence avoids external database dependencies that REST API tools require.