Agent Web Census

● Active

org.datacensus/agents

Open data on MCP servers (with live health) and which AI crawlers top websites block.

Connect this server

One click to install, or copy the config below. Suits: Cursor · HTTP clients.

Add to Cursor Add to VS CodeClaude Desktop / Claude Code: paste the config below.

Or add it manually — claude_desktop_config.json (Claude Desktop) or .cursor/mcp.json (Cursor):

{
  "mcpServers": {
    "agents": {
      "url": "https://datacensus.org/mcp"
    }
  }
}
One line, from the terminal
Claude Code
claude mcp add --transport http agents https://datacensus.org/mcp
VS Code
code --add-mcp '{"name":"agents","type":"http","url":"https://datacensus.org/mcp"}'

Syntax from each client's docs (Claude Code: claude mcp add; Codex: codex mcp add; VS Code: code --add-mcp). Placeholders in angle brackets are values you fill in. Codex takes hosted servers in config.toml ([mcp_servers.<name>] url = …) rather than on the command line.

A starting point — check the repository for exact arguments and setup. Restart your client after editing the config.

Category
Search & Web
Version
0.1.0
Registry
active
Published
11/19/2026
Listed in:Official MCP registry ↗how this is checked

Is it safe? Is it maintained?

Serverdex check: Nov 22, 2026

Looks healthy: Agent Web Census is maintained, open source, hosted endpoint answering without a key.

Maintained
Active
Last commit today (Nov 20, 2026), 0 open issues.
Official registry
Active
Published as org.datacensus/agents in the official MCP registry, version 0.1.0.
Source
Open source, MIT
Public repository under the MIT licence, 0 stars, 0 forks.
Runs
Hosted by the publisher
Your client talks to datacensus.org: prompts and tool arguments leave your machine for that host.
Credentials
None needed to connect
The hosted endpoint completed the Serverdex initialize + tools/list handshake with no key (checked Nov 22, 2026); individual tools may still ask for one.
Owner
Unclaimed
Nobody has claimed this page yet; the data above comes from the registry, GitHub and the live check.

Facts only: last commit and licence from GitHub, the entry's status from the official registry, credentials from the live check. Not a code audit — run the security scan on the endpoint yourself and see the status history.

Live check

checked Nov 22, 2026, 07:08 UTCStatus history & uptime →
● Working54 msprotocol 2025-06-18v0.1.0tools

The endpoint answered the MCP handshake as "datacensus" and listed 4 tools. Answering since Nov 21, 2026.

Context cost ≈ 694 tokens of tool definitions a client loads per conversation (estimate: characters ÷ 4) across 4 tools.

Tools (4)

  • find_mcp_servers5 params

    Search the official MCP Registry (~65,000 servers) by keyword. Matches name, title and description; ranks servers whose remote endpoint passed a live initialize check first, then by number of tools. Returns name, title, description, remote/package types, health (ok = works without auth, auth = alive but needs auth, fail = unreachable), tool count and a registry link.

    Show parameters
    • querystringrequiredKeywords, e.g. 'postgres database' or 'github issues'.
    • categorystringOptional category filter, e.g. 'finance & crypto', 'databases' (substring match).
    • transportstringremote = hosted endpoint, package = installable (npm/pypi/...), any (default).
    • working_onlybooleanOnly servers whose remote endpoint responded OK or requires auth (default true). Package-only servers are kept unless transport is 'remote'.
    • limitintegerMax results (1-25).
  • mcp_ecosystem_stats

    Headline numbers of the MCP ecosystem: servers in the official registry, new servers in the last 7/30 days, remote vs package share, top package registries and categories, plus the weekly health check of remote servers (share working, requiring auth, failing, median tools and latency).

  • site_ai_crawler_policy1 param

    Which AI crawlers does a website block in robots.txt? Looks up a domain in our scan of the top 10,000 websites (Tranco list) and returns, per AI bot (GPTBot, ClaudeBot, Google-Extended, PerplexityBot, ...), whether it is blocked, partly restricted or explicitly allowed, plus llms.txt presence.

    Show parameters
    • domainstringrequiredDomain or URL, e.g. 'nytimes.com' or 'https://www.nytimes.com/section'.
  • ai_crawler_stats1 param

    Aggregate AI-crawler blocking statistics across the top 10,000 websites: percentage of sites with a reachable robots.txt that fully block, partly restrict or explicitly allow each AI bot. Give a bot name for one bot, or omit for the overall table.

    Show parameters
    • botstringOptional bot name, e.g. 'GPTBot', 'ClaudeBot', 'Google-Extended' (case-insensitive).

Checked by Serverdex with a plain initialize + tools/list call to datacensus.org/mcp, no credentials, about once a day. A failed check is never a removal.

On GitHub

read Nov 21, 2026
Stars
0
Forks
0
Last push
Nov 20, 2026
License
MIT

Python

From the README

Agent Web Census Live, daily-updated, measured statistics on the AI agent ecosystem — no estimates, no made-up numbers. • MCP servers: full daily crawl of the official MCP Registry → totals, growth, deployment (remote vs. package), registries, categories, publishers. • AI crawlers & the web: weekly scan of robots.txt and llms.txt on the top 10,000 websites (Tranco) → who blocks GPTBot, ClaudeBot, Google-Extended, CCBot… Site: GitHub Pages (built by .github/workflows/update.yml). Data: data//latest.json, data//history.csv — CC BY 4.0, cite as “Agent Web Census”. Transparency This project is built and operated by an AI agent (Claude, by Anthropic) with a human sponsor. The scanner identifies itself as AgentWebCensusBot and fetches only robots.txt and llms.txt (max 3 requests per domain per week). Run locally

Read the full README on GitHub →

Remotes

  • streamable-httphttps://datacensus.org/mcp

The most-starred Search & Web servers in the index, with their live-check result.

ServerStarsLast pushLive check
Browser Use121,623Nov 2026Working
Scrapling MCP Server104,822Nov 2026Local / not checked
Chrome DevTools MCP58,028Nov 2026Local / not checked
Chrome DevTools MCP58,014Nov 2026Local / not checked
com.puter MCP Server44,182Nov 2026Needs sign-in

Agent Web Census: questions people ask

Is Agent Web Census a remote or a local MCP server?
According to its entry in the official MCP registry, Agent Web Census ships as a hosted endpoint (streamable-http) that your client connects to by URL.
Does Agent Web Census need an API key?
Its hosted endpoint completed the Serverdex live check — an MCP initialize + tools/list call — with no credentials on Nov 22, 2026. Individual tools may still ask for a key when called.
Is Agent Web Census safe to use?
Looks healthy: Agent Web Census is maintained, open source, hosted endpoint answering without a key. In detail — maintained: Active; official registry: Active; source: Open source, MIT; runs: Hosted by the publisher; credentials: None needed to connect. Serverdex check of Nov 22, 2026. This is a reading of the repository, the registry entry and the live check, not a code audit; run the Serverdex security scan on the endpoint before trusting it with real data.
Is Agent Web Census still maintained?
Active: the repository's last commit today. Last push Nov 20, 2026, 0 GitHub stars, MIT licence. Read from GitHub on Nov 21, 2026.
What tools does Agent Web Census expose?
4 tools as listed by the server itself during the live check, including find_mcp_servers, mcp_ecosystem_stats, site_ai_crawler_policy, ai_crawler_stats. The full list with parameters is in the Live check section above.
How do I add Agent Web Census to Claude, Cursor or VS Code?
Copy the config in "Connect this server" into your client's MCP config file (claude_desktop_config.json, ~/.cursor/mcp.json or .vscode/mcp.json), or use the one-click Add to Cursor / Add to VS Code buttons and the one-line commands for Claude Code and Codex; then restart the client.
Browse all Search & Web →

Add the badge to your site

Show visitors that Agent Web Census is listed here — it’s also the link we check for a free listing.

Agent Web Census — listed on Serverdex

HTML — for your website

<a href="https://serverdex.io/server/org-datacensus-agents?utm_source=badge"><img src="https://serverdex.io/brand/badge.svg" alt="Agent Web Census — listed on Serverdex" width="123" height="40" loading="lazy"></a>

Recommended. A dofollow link from your own domain is the one that counts for search engines, and it's the link we check for a free listing.

Markdown — for your README

[![Agent Web Census — listed on Serverdex](https://serverdex.io/brand/badge.svg)](https://serverdex.io/server/org-datacensus-agents?utm_source=badge)

Nice for visibility, but GitHub marks README links nofollow, so it doesn't pass authority. Use the HTML badge on your site as well.

Add to CursorAdd to VS CodeOpen in Claude Code

Install buttons — for your README

[![Add to Cursor](https://serverdex.io/brand/install-cursor.svg)](https://serverdex.io/install/cursor/org-datacensus-agents) [![Add to VS Code](https://serverdex.io/brand/install-vscode.svg)](https://serverdex.io/install/vscode/org-datacensus-agents) [![Open in Claude Code](https://serverdex.io/brand/install-claude-code.svg)](https://serverdex.io/install/claude-code/org-datacensus-agents)

One click opens Cursor or VS Code with this server's config prefilled; the Claude Code button lands on the one-line command. The links point at Serverdex, so when your config changes the buttons keep working.

Install buttons — HTML for your docs

<a href="https://serverdex.io/install/cursor/org-datacensus-agents"><img src="https://serverdex.io/brand/install-cursor.svg" alt="Add to Cursor — via Serverdex" width="140" height="28"></a>
<a href="https://serverdex.io/install/vscode/org-datacensus-agents"><img src="https://serverdex.io/brand/install-vscode.svg" alt="Add to VS Code — via Serverdex" width="147" height="28"></a>
<a href="https://serverdex.io/install/claude-code/org-datacensus-agents"><img src="https://serverdex.io/brand/install-claude-code.svg" alt="Open in Claude Code — via Serverdex" width="180" height="28"></a>

Same buttons as plain links for a docs page or website.

Prefer a live badge with the score, stars and live check? Score badge (links to a page that explains the number and back to this listing).