Directory
‹ Back to apps

crawlbrulee: EU-native web scraping for AI agents

EU-native web scraping for AI agents. one call turns any url into clean markdown, screenshots, metadata and links, or just the values you name by CSS selector, processed fully in the EU.

What’s inside

1 MCP server · 10 skills
  • MCP servercrawlbruleethe crawlbrulee mcp server: scrape, map and screenshot pages from your agent. set CRAWLBRULEE_API_KEY (free key at dashboard.crawlbrulee.com)
  • Skillcrawlbrulee-apiuse when you need crawlbrulee's shared api contract — the base url, bearer auth, the full endpoint list, the page_status_code field, what gets billed, the response_meta usage object, caching, zero data retention, proxy tiers, location targeting, and the error model. read this whichever interface you call from. also covers calling the http api directly with curl or a generated client.
  • Skillcrawlbrulee-cliuse when driving crawlbrulee from a terminal or a shell script — the `crawlbrulee` / `npx crawlbrulee` command. covers scrape url, scrape status/result/wait, map, usage, whoami, login/logout/view-config, every flag, picking values by css selector (--element / --elements), the screenshot shorthand, and text-vs-json output.
  • Skillcrawlbrulee-mapuse when discovering which urls exist on a site with crawlbrulee before scraping them — sitemap plus homepage link discovery, filtering to internal, external, or subdomain links, pagination through large result sets, and reading truncation info. this is how you plan a crawl, since there is no crawl endpoint.
  • Skillcrawlbrulee-mcpuse when an mcp-aware agent or editor should scrape pages (or pull named values out of them by css selector), map sites, run background scrape jobs, or check crawlbrulee credits as native tool calls. covers installing the `@crawlbrulee/mcp` stdio server and its seven tools.
  • Skillcrawlbrulee-scrape-asyncuse when a crawlbrulee scrape shouldn't hold the connection open — submitting a background job, polling its status, fetching the result later, or being notified by a signed scrape.complete webhook instead of polling. covers the job lifecycle, webhook payloads, signature verification, and secret rotation.
  • Skillcrawlbrulee-scrapeuse when you need the crawlbrulee scrape contract — turning a url into markdown, cleaned html, raw html, links, images, or page metadata, or pulling named values (prices, titles, links, table rows) out of a page by css selector. covers every extract format and which are on by default, js rendering, removing parts of the page (nav, footers), cache and proxy options, and the full response shape including the page's own status (page_status_code), warnings and unsupported fields.
  • Skillcrawlbrulee-screenshotsuse when capturing a screenshot of a web page with crawlbrulee — viewport vs full-page, desktop vs mobile, custom viewport size and scale factor, ad and popup cleanup, scrolling or waiting before capture, slicing tall pages into tiles, and where the image url lands in the response.
  • Skillcrawlbrulee-sdk-jsuse when calling crawlbrulee from Node.js, TypeScript, Deno, or Bun code — the `@crawlbrulee/sdk` package. covers constructing the Crawlbrulee client, every method, background jobs with waitForScrape, webhook signature verification, the typed error classes, and cancellation with AbortSignal.
  • Skillcrawlbrulee-sdk-pythonuse when calling crawlbrulee from Python code, sync or async — the `crawlbrulee` PyPI package. covers the Crawlbrulee and AsyncCrawlbrulee clients, every method, background jobs with wait_for_scrape, webhook signature verification, and the typed error classes.
  • Skillcrawlbruleeuse when you need web data from a url with crawlbrulee — scraping a page into markdown or html, pulling its links or images, screenshotting it, or discovering which urls exist on a site. start here to learn what crawlbrulee does, set up the api key, and pick an interface (cli, mcp, js/ts sdk, python sdk, or raw http).