Crawlsight

● Active

io.github.CloudCorpRecords/crawlsight

Check a site's AI visibility: crawler access, llms.txt generate and validate, robots.txt lint.

Connect this server

One click to install, or copy the config below. Suits: Claude Desktop · Cursor.

Add to Cursor Add to VS CodeClaude Desktop / Claude Code: paste the config below.

Or add it manually — claude_desktop_config.json (Claude Desktop) or .cursor/mcp.json (Cursor):

{
  "mcpServers": {
    "crawlsight": {
      "command": "npx",
      "args": [
        "-y",
        "@reneturcios/crawlsight@0.1.0",
        "mcp"
      ]
    }
  }
}
One line, from the terminal
Claude Code
claude mcp add crawlsight -- npx -y @reneturcios/crawlsight@0.1.0 mcp
OpenAI Codex CLI
codex mcp add crawlsight -- npx -y @reneturcios/crawlsight@0.1.0 mcp
VS Code
code --add-mcp '{"name":"crawlsight","type":"stdio","command":"npx","args":["-y","@reneturcios/crawlsight@0.1.0","mcp"]}'

Syntax from each client's docs (Claude Code: claude mcp add; Codex: codex mcp add; VS Code: code --add-mcp). Placeholders in angle brackets are values you fill in.

A starting point — check the repository for exact arguments and setup. Restart your client after editing the config.

Category
Search & Web
Version
0.1.0
Registry
active
Published
11/17/2026
Listed in:Official MCP registry ↗how this is checked

Is it safe? Is it maintained?

Serverdex check: Nov 19, 2026

Looks healthy: Crawlsight is maintained, open source.

Maintained
Active
Last commit 4 days ago (Nov 17, 2026), 0 open issues.
Official registry
Active
Published as io.github.CloudCorpRecords/crawlsight in the official MCP registry, version 0.1.0.
Source
Open source, MIT
Public repository under the MIT licence, 0 stars, 0 forks.
Runs
On your machine
A local npm/Docker package started by your client; it can reach whatever your user account can.
Credentials
None declared
The registry entry declares no required environment variables.
Owner
Unclaimed
Nobody has claimed this page yet; the data above comes from the registry, GitHub and the live check.

Facts only: last commit and licence from GitHub, the entry's status from the official registry, credentials from the live check. Not a code audit — run the security scan on the endpoint yourself.

On GitHub

read Nov 18, 2026
Stars
0
Forks
0
Last push
Nov 17, 2026
License
MIT

TypeScript

From the README

crawlsight How visible is a website to AI search? crawlsight is a small forwarding client for the hosted AI Visibility service: the tool list is served from your own machine using the captured definitions, and each call is sent upstream, with the service's answer returned exactly as it arrived. All four captured tools are exposed. You can run a full visibility check on a site, generate a fresh llms.txt for it from its real pages, validate the llms.txt it already ships, and lint its robots.txt for the kinds of mistakes that quietly block AI crawlers. Every input is a public URL or pasted public text, and every tool only reads and analyses public endpoints. The tools • check_site — Check a site's AI-search visibility: which AI crawlers (OAI-SearchBot, PerplexityBot, GPTBot, ClaudeBot and more) its robots.txt allows, whether the homepage ships readable text without JavaScript, and whether an llms.txt exists. Takes a required url. • generatellmstxt — Generate an llms.txt file from a site's real pages and sitemap. Takes a required url and an optional credit boolean (default true) that adds a one-line attribution back to the tool. • validatellmstxt — Validate an existing llms.txt against the spec. Takes a required url; the site's /llms.txt is fetched and validated. • lintrobotstxt — Lint a robots.txt for AI-crawler-specific mistakes. Pass url to fetch it live, or robots_txt to lint pasted text directly.…

Read the full README on GitHub →

The most-starred Search & Web servers in the index, with their live-check result.

ServerStarsLast pushLive check
Browser Use121,623Nov 2026Working
Scrapling MCP Server104,822Nov 2026Local / not checked
Chrome DevTools MCP58,028Nov 2026Local / not checked
Chrome DevTools MCP58,014Nov 2026Local / not checked
com.puter MCP Server44,182Nov 2026Needs sign-in

Crawlsight: questions people ask

Is Crawlsight a remote or a local MCP server?
According to its entry in the official MCP registry, Crawlsight ships as a local package (npm, Docker) that your client starts on your machine.
Is Crawlsight safe to use?
Looks healthy: Crawlsight is maintained, open source. In detail — maintained: Active; official registry: Active; source: Open source, MIT; runs: On your machine; credentials: None declared. Serverdex check of Nov 19, 2026. This is a reading of the repository, the registry entry and the live check, not a code audit; run the Serverdex security scan on the endpoint before trusting it with real data.
Is Crawlsight still maintained?
Active: the repository's last commit 4 days ago. Last push Nov 17, 2026, 0 GitHub stars, MIT licence. Read from GitHub on Nov 18, 2026.
How do I add Crawlsight to Claude, Cursor or VS Code?
Copy the config in "Connect this server" into your client's MCP config file (claude_desktop_config.json, ~/.cursor/mcp.json or .vscode/mcp.json), or use the one-click Add to Cursor / Add to VS Code buttons and the one-line commands for Claude Code and Codex; then restart the client.

More from CloudCorpRecords

All 4 servers by CloudCorpRecords →
  1. Muse MarketplaceOpen marketplace and registry for shareable Muse assets: avatars, personas, voices, and bundles.
  2. OpensourceitAI-welcoming open source community: agents submit and discuss projects. No AI ban.
  3. ShelfSignalThe open store index for AI agents: merchants list real stock, agents find local food and stores.
  4. ProdForgeConnects MCP servers, packages, security checks, and release gates in one repo.
Browse all Search & Web →

Add the badge to your site

Show visitors that Crawlsight is listed here — it’s also the link we check for a free listing.

Crawlsight — listed on Serverdex

HTML — for your website

<a href="https://serverdex.io/server/io-github-cloudcorprecords-crawlsight?utm_source=badge"><img src="https://serverdex.io/brand/badge.svg" alt="Crawlsight — listed on Serverdex" width="123" height="40" loading="lazy"></a>

Recommended. A dofollow link from your own domain is the one that counts for search engines, and it's the link we check for a free listing.

Markdown — for your README

[![Crawlsight — listed on Serverdex](https://serverdex.io/brand/badge.svg)](https://serverdex.io/server/io-github-cloudcorprecords-crawlsight?utm_source=badge)

Nice for visibility, but GitHub marks README links nofollow, so it doesn't pass authority. Use the HTML badge on your site as well.

Add to CursorAdd to VS CodeOpen in Claude Code

Install buttons — for your README

[![Add to Cursor](https://serverdex.io/brand/install-cursor.svg)](https://serverdex.io/install/cursor/io-github-cloudcorprecords-crawlsight) [![Add to VS Code](https://serverdex.io/brand/install-vscode.svg)](https://serverdex.io/install/vscode/io-github-cloudcorprecords-crawlsight) [![Open in Claude Code](https://serverdex.io/brand/install-claude-code.svg)](https://serverdex.io/install/claude-code/io-github-cloudcorprecords-crawlsight)

One click opens Cursor or VS Code with this server's config prefilled; the Claude Code button lands on the one-line command. The links point at Serverdex, so when your config changes the buttons keep working.

Install buttons — HTML for your docs

<a href="https://serverdex.io/install/cursor/io-github-cloudcorprecords-crawlsight"><img src="https://serverdex.io/brand/install-cursor.svg" alt="Add to Cursor — via Serverdex" width="140" height="28"></a>
<a href="https://serverdex.io/install/vscode/io-github-cloudcorprecords-crawlsight"><img src="https://serverdex.io/brand/install-vscode.svg" alt="Add to VS Code — via Serverdex" width="147" height="28"></a>
<a href="https://serverdex.io/install/claude-code/io-github-cloudcorprecords-crawlsight"><img src="https://serverdex.io/brand/install-claude-code.svg" alt="Open in Claude Code — via Serverdex" width="180" height="28"></a>

Same buttons as plain links for a docs page or website.

Prefer a live badge with the score, stars and live check? Score badge (links to a page that explains the number and back to this listing).