Documentation Crawler & MCP Server

Created By
alizdavoodia year ago
This project provides a toolset to crawl websites wikis, tool/library documentions and generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, designed for integration with tools like Cursor.
Overview

What is MCPDocSearch?

MCPDocSearch is a toolset designed to crawl websites, generate Markdown documentation, and make that documentation searchable via a Model Context Protocol (MCP) server, facilitating integration with tools like Cursor.

How to use MCPDocSearch?

To use MCPDocSearch, you first run the crawler_cli to crawl a website and generate a Markdown file. Then, you run the mcp_server to load and serve the documentation, allowing clients like Cursor to query the content.

Key features of MCPDocSearch?

  • Web Crawler (crawler_cli): Configurable crawling of websites with options for depth, URL patterns, and HTML cleaning.
  • MCP Server (mcp_server): Loads Markdown files, parses them into semantic chunks, and exposes tools for searching and retrieving documentation.
  • Cursor Integration: Designed for seamless operation with Cursor, allowing for easy querying of documentation.

Use cases of MCPDocSearch?

  1. Crawling and documenting API references from various websites.
  2. Creating searchable documentation for internal company resources.
  3. Integrating with tools like Cursor for enhanced documentation accessibility.

FAQ from MCPDocSearch?

  • Can MCPDocSearch crawl any website?

Yes, as long as the website allows crawling and follows the robots.txt rules.

  • Is there a limit to the crawl depth?

Yes, the maximum crawl depth is configurable, typically between 1 and 5.

  • How do I integrate MCPDocSearch with Cursor?

You need to configure a .cursor/mcp.json file in the project root with the appropriate settings for the MCP server.

Project Info
Created At
a year ago
Updated At
a year ago
Author Name
alizdavoodi
Star
8
Language
Python
License
MIT license

Recommend Servers

View All
Enigmata
@Enigmata

Enigmata MCP provides AI clients with secure access to personalized Enigmata astrological forecasts, including day, week, month, and thematic forecast contexts. Forecasts contain the percentage of favorability of any day and the probability of any event. It helps assistants retrieve structured guidance, compare forecast periods, search themes, check profile readiness, and complete required solar-location setup when needed. Tools get_mcp_capabilities - Returns metadata about the available Enigmata MCP tools and their capabilities. get_day_forecast - Returns an AI-friendly personalized day forecast. get_day_forecast_section - Returns a specific section of a day forecast, useful when a client needs smaller responses. get_week_forecast - Returns an AI-friendly personalized week forecast. get_month_forecast - Returns an AI-friendly personalized month forecast. get_theme_forecast - Returns a forecast for a selected theme, including favorability or probability context. get_theme_best_days - Returns the best days for a selected forecast theme. compare_theme_days - Compares two dates for a selected theme without returning the full theme calendar. get_theme_probability_trends - Returns the subscription-period probability trend for a selected theme. get_theme_catalog - Returns the user’s saved themes and popular available themes. search_themes - Searches forecast themes and returns ranked candidates for precise theme selection. compare_forecasts - Compares two day, week, month, or theme forecast contexts. get_profile_readiness - Returns the user’s profile and onboarding readiness for forecast access. get_solar_requirement - Checks whether a solar meeting city is required for the user’s forecast flow. search_cities - Searches cities that can be used as a solar meeting place. submit_solar_place - Submits the user’s solar meeting city when it is required to unlock or update forecasts.

2 days ago