- Web Scraper MCP
Web Scraper MCP
Overview
What is MCP Web Scraper?
MCP Web Scraper is a lightweight and efficient web scraping server that allows users to scrape websites and interact with their data using the Model Context Protocol (MCP).
How to use MCP Web Scraper?
To use MCP Web Scraper, you can either automate the setup by cloning the repository and running the setup script, or manually set it up by creating a virtual environment and installing the required dependencies.
Key features of MCP Web Scraper?
- Text, link, image, and table data extraction with CSS selectors.
- Comprehensive metadata extraction including Open Graph and Twitter Cards.
- Integration with Claude Desktop for seamless operation.
- Configurable result limits and error handling.
Use cases of MCP Web Scraper?
- Extracting text content from various websites.
- Gathering headlines and metadata for news articles.
- Scraping images and tables for data analysis.
FAQ from MCP Web Scraper?
- Can MCP Web Scraper handle all types of websites?
Yes, it can scrape a wide variety of websites as long as they allow it in their robots.txt file.
- Is there a limit to the number of results I can scrape?
Yes, you can configure the maximum number of results to prevent overload.
- What dependencies does MCP Web Scraper require?
It requires libraries like requests, beautifulsoup4, and lxml for web scraping.
Server Config
{
"mcpServers": {
"web-scraper": {
"command": "/full/path/to/your/venv/bin/python",
"args": [
"/full/path/to/your/app_mcp.py"
]
}
}
}Project Info
Created At
a year agoUpdated At
a year agoAuthor Name
navin4078Star
-Language
-License
-Category
developer-tools
Tags
Recommend Servers
View AllAmap Maps
@amap
高德地图官方 MCP Server
a year ago
Sigstore
@3089464667
a day ago
Mcp Server Chatsum
@chatmcp
summarize chat message
typescript
a year ago
虎嗅嗅 Ai行业参谋
5 hours ago