- Crawlbase Web MCP Server
Crawlbase Web MCP Server
Crawlbase Web MCP Server (Model Context Protocol) connects AI agents and large language models (LLMs) with real-time web data. Built on Crawlbase’s proven web scraping and crawling infrastructure, it handles JavaScript rendering, anti-bot protection, and web data extraction at scale — powering Claude, Cursor, and Windsurf. It delivers a reliable, production-ready pipeline of live, structured data for AI workflows and intelligent applications.
Overview
What is Crawlbase MCP?
Crawlbase MCP is a Model Context Protocol (MCP) server that bridges AI agents and the live web. Instead of relying on outdated training data, your LLMs can now fetch fresh, structured, real-time content — powered by Crawlbase’s proven crawling infrastructure trusted by 70,000+ developers worldwide.
It handles the complexity of scraping for you:
- JavaScript rendering for modern web apps
- Proxy rotation & anti-bot evasion
- Structured outputs (HTML, Markdown, screenshots)
How It Works
- Get Free Crawlbase Tokens → Sign up at Crawlbase ↗, get free Normal, and JavaScript tokens.
- Add MCP Config → Connect Crawlbase MCP to Claude, Cursor, or Windsurf by updating your
mcpServersconfig. - Start Crawling → Use commands like
crawl,crawl_markdown, orcrawl_screenshotto bring live web data into your AI agent.
Features
- Real-time web scraping for AI agents
- JavaScript rendering (SPAs & dynamic pages)
- Proxy rotation to bypass blocks & captchas
- Structured outputs (HTML, Markdown, Screenshots)
- Seamless MCP integration with Claude, Cursor & Windsurf
Use Cases
- Research with up-to-date articles & reports
- Monitor e-commerce products & prices
- Fetch real-time news & financial data
- Aggregate content for data pipelines
- Power AI agents with fresh, accurate information
Server Config
{
"mcpServers": {
"crawlbase": {
"type": "stdio",
"command": "npx",
"args": [
"@crawlbase/mcp@latest"
],
"env": {
"CRAWLBASE_TOKEN": "your_token_here",
"CRAWLBASE_JS_TOKEN": "your_js_token_here"
}
}
}
}Project Info
Created At
10 months agoUpdated At
9 months agoAuthor Name
crawlbaseStar
-Language
-License
-Recommend Servers
View All虎嗅嗅 Ai行业参谋
6 hours ago
Memory
@modelcontextprotocol
a year ago
Shotapi
@smallhandsome
a day ago
Voyei
36 minutes ago
Playwright Mcp
@microsoft
Playwright MCP server
TypeScript
10 months ago
Airtreks Mcp
@SEKeener
a day ago
Fabric
@fabric-agent-plugins
4 hours ago
Amap Maps
@amap
高德地图官方 MCP Server
a year ago