03 · AI Agent
crawlio.app
Visit ↗An AI-native web crawler that runs on your Mac, not in someone else's cloud.
The Problem
JS-heavy sites and bot challenges break traditional scrapers, while SaaS crawl APIs charge per page and still return incomplete markup.
For who
AI engineers feeding real sites into Claude or Cursor, SEO teams auditing competitors, and macOS developers who want a one-time-purchase tool instead of a metered subscription.
Overview
Crawlio is a Swift 6 macOS app that clones an entire site — HTML, CSS, JS, images, and structure — into a local folder you own, with an embedded MCP server exposing 36 tools so Claude, Cursor, and ChatGPT can drive the full crawler natively.
Key Capabilities
Extract
Clones the frontend exactly as your browser saw it — HTML, CSS, JS, images, and structure — so the captured site is yours to keep offline.
Audit
Runs 19 site analyzers covering SEO, accessibility, security, content quality, link architecture, hreflang, and tracking.
MCP server for AI
Exposes 36 full-mode and 6 code-mode tools over stdio and HTTP so any MCP-compatible agent can drive Crawlio directly.
Seven export formats
Folder, ZIP, single HTML, WARC, PDF, deploy-ready, and extracted JSON — plus HTML→Markdown and HTML→JSX/Tailwind replication.
Stack