Directory
‹ Back to apps

Olostep - web search, scraping and crawling for AI

Turn the entire web into LLM-ready context. Olostep integrates web search, scraping, crawling, and AI answers directly into Cursor. Essential for agents workflows: debug obscure errors by scraping GitHub issues, automate framework migrations by scraping breaking changes, turn websites into typed JSON schemas, or crawl entire API docs into context. Handles JS rendering, anti-bot, and proxies automatically.

What’s inside

1 MCP server · 13 skills
  • MCP serverolostep webMCP server: olostep web search, scraping and crawling
  • SkillanswersGet AI-powered answers with citations from live web data using Olostep. Use when the user needs up-to-date facts, competitive intelligence, pricing comparisons, structured data from the web, or wants web-sourced answers in a specific JSON shape they can use directly in code or docs.
  • SkillbatchScrape up to 10,000 URLs in parallel using Olostep. Use when the user has a list of URLs to extract content from at once — product pages, job listings, pricing pages, changelogs, docs, or any set of known pages. Much faster than scraping one-by-one.
  • SkillcrawlAutonomously crawl an entire website by following links from a starting URL using Olostep. Use when the user wants to ingest an entire docs site, blog, knowledge base, or any multi-page site as context for coding, writing, analysis, or building a knowledge base.
  • Skilldebug-errorSearch the web for a specific error message or bug using Olostep. Use when the user pastes an error stack trace, says their code is failing, or asks how to fix a bug that requires looking up recent GitHub issues, StackOverflow threads, or framework documentation.
  • Skilldocs-to-codeScrape API documentation or library docs and use them to write working, up-to-date code. Use when the user wants to integrate a third-party API, learn a new library, generate SDK usage examples, write code based on a docs URL, or update code after a library version change. This is the flagship workflow — never hallucinate API parameters when you can scrape the real docs.
  • Skillextract-schemaScrape a webpage with Olostep and extract specific structured data matching a TypeScript interface, JSON schema, or database model. Use when the user wants to turn a website (like a product list, directory, or article) into clean, structured JSON or seed data.
  • SkillintegrateAutomatically integrate the Olostep SDK into the user's codebase. Analyzes the project, detects the language and framework, chooses the right integration pattern, installs the SDK, writes all the code, and verifies it works — with minimal prompting. Use when the user wants to add Olostep web scraping, search, or AI answers to their project.
  • SkillmapDiscover and list all URLs on a website using Olostep. Use when the user wants to see what pages exist on a site, find specific sections (docs, blog, API reference), audit a site's structure, or get a URL list to feed into batch or crawl.
  • Skillmigrate-codeScrape a migration guide, changelog, or breaking changes document using Olostep and automatically update the user's local code. Use when the user is upgrading a framework, moving to a new API version, or says "help me migrate this code based on these docs".
  • SkillresearchDo thorough, cited web research to help the user make a decision. Compare tools, libraries, or services. Evaluate competitors. Research tech stacks, market data, or architecture patterns. Use when the user needs a well-sourced answer before making a technical, product, or business decision.
  • SkillscrapeScrape a single webpage and extract its content as clean markdown, HTML, JSON, or text using Olostep. Use when the user shares a URL and wants to read it, extract content from it, understand its structure, use it as context for coding, or turn a page into data. Handles JavaScript-rendered pages, SPAs, and bot-protected sites automatically.
  • SkillsearchSearch the web using Olostep for live, up-to-date information. Three tools available — answers (AI-synthesised answers with citations), google_search (structured Google results), and get_website_urls (find pages within a specific site). Use when the user needs current data, comparisons, or facts the AI's training data may not have.
  • SkillsetupConfigure the Olostep API key so the MCP server and all Olostep skills work correctly. Use when the user is setting up Olostep for the first time, has no API key configured, or is getting authentication errors.