From 6d922fea2670b4c567c5e7b2b3a6dabf79ee7611 Mon Sep 17 00:00:00 2001 From: Grubbomatic Date: Thu, 17 Sep 2026 18:35:13 -0400 Subject: [PATCH] Add Crawl Readiness MCP server (AI SEO) Adds crawl-readiness-mcp to the Search & Data Extraction category: 7 tools for scoring AI crawler visibility, validating JSON-LD/robots.txt, and generating llms.txt/robots.txt/JSON-LD schema. - npm: https://www.npmjs.com/package/crawl-readiness-mcp - Source: https://github.com/Grubbomatic/crawl-readiness-mcp Co-Authored-By: Claude Opus 4.7 --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index 45f85250f..bba2dd985 100644 --- a/README.md +++ b/README.md @@ -3468,6 +3468,7 @@ Tools for conducting research, surveys, interviews, and data collection. - [vectara/vectara-mcp](https://github.com/vectara/vectara-mcp) 🐍 🏠 ☁️ - An MCP server for accessing Vectara's trusted RAG-as-a-service platform. - [zaharajabeen13-create/ai-rete-rag-mcp](https://github.com/zaharajabeen13-create/ai-rete-rag-mcp) [![zaharajabeen13-create/ai-rete-rag-mcp MCP server](https://glama.ai/mcp/servers/zaharajabeen13-create/ai-rete-rag-mcp/badges/score.svg)](https://glama.ai/mcp/servers/zaharajabeen13-create/ai-rete-rag-mcp) 🐍 ☁️ - Decisions the model doesn't make: a Rete rule engine returns the verdict, retrieval over your own policy documents explains it — so an explanation can't invent a threshold your rulebook doesn't have. Turn a written policy into draft rules that cite the sentence each encodes, save them as YAML, then decide; every verdict names the rule that fired, and you can ask why any rule didn't. ### 🔎 Search & Data Extraction +- [Grubbomatic/crawl-readiness-mcp](https://github.com/Grubbomatic/crawl-readiness-mcp) [![Grubbomatic/crawl-readiness-mcp MCP server](https://glama.ai/mcp/servers/Grubbomatic/crawl-readiness-mcp/badges/score.svg)](https://glama.ai/mcp/servers/Grubbomatic/crawl-readiness-mcp) 📇 ☁️ 🏠 - AI SEO audit and fix tools: score any site 0-100 against 50+ AI crawlers (ChatGPT, Claude, Perplexity, Google AI), validate JSON-LD and robots.txt, compare human vs AI-crawler views, and generate llms.txt, robots.txt, and JSON-LD schema. Install via `npx -y crawl-readiness-mcp`. - [HasData/web-scraping-mcp](https://github.com/HasData/web-scraping-mcp) [![HasData/web-scraping-mcp MCP server](https://glama.ai/mcp/servers/HasData/web-scraping-mcp/badges/score.svg)](https://glama.ai/mcp/servers/HasData/web-scraping-mcp) 📇 ☁️ - Remote MCP server for web scraping: fetch any public page with optional JavaScript rendering, wait conditions and CSS or AI extraction rules, as markdown, text, HTML or JSON. - [ToolTrace-io/mcp-server](https://github.com/ToolTrace-io/mcp-server) [![ToolTrace-io/mcp-server MCP server](https://glama.ai/mcp/servers/ToolTrace-io/mcp-server/badges/score.svg)](https://glama.ai/mcp/servers/ToolTrace-io/mcp-server) 🎖️ 📇 ☁️ - Web page data for agents: scrape to Markdown, extract metadata, links and JSON-LD schema, audit on-page SEO, detect tech stacks, and validate XML sitemaps. Free tier, no card. Install via `npx -y @tooltrace/mcp-server`, or use the hosted endpoint at `https://mcp.tooltrace.io/mcp`. - [JerryLiu369/agent-web-search](https://github.com/JerryLiu369/agent-web-search) [![JerryLiu369/agent-web-search MCP server](https://glama.ai/mcp/servers/JerryLiu369/agent-web-search/badges/score.svg)](https://glama.ai/mcp/servers/JerryLiu369/agent-web-search) 🐍 ☁️ 🏠 - Agent-native web search for AI agents: one provider-neutral `web_search` tool aggregating model-native grounding (ARK, Gemini, Grok, DeepSeek, Zhipu) and agent search APIs (Exa, Parallel, Brave, Perplexity, Tavily, You.com). Keyless free defaults, stdio + stateless Streamable-HTTP MCP, CLI, Python API, and Hermes plugin. Install via `pipx install agent-web-search-mcp`.