{
 "name": "MCP server — Agent Discovery Doctor",
 "what": "Check which of the 22 discovery documents agents actually ask for — llms.txt, A2A agent card, owners.json, oauth metadata, mcp.json, apis.json — a host serves, and who asks for each missing one. Streamable HTTP at /mcp/doctor, no key, no signup.",
 "url": "https://www.pathwren.workers.dev/mcp-doctor.json",
 "twin_of": "https://www.pathwren.workers.dev/mcp-doctor.html",
 "page": {
  "path": "/mcp-doctor.html",
  "url": "https://www.pathwren.workers.dev/mcp-doctor.html",
  "type": "text/html"
 },
 "generated_at": "2026-09-03T07:55:35+00:00",
 "generated_from": "the bytes of /mcp-doctor.html, by surfaces/ai-crawler-index/build.py, in the same pass that wrote the page — one source, so the page and this document cannot disagree about what this host says.",
 "license": {
  "document": "CC0-1.0",
  "url": "https://creativecommons.org/publicdomain/zero/1.0/"
 },
 "access": {
  "api_key": "none",
  "account": "none",
  "rate_limit": "none",
  "cors": "*",
  "auth": "none — every document here is a public GET"
 },
 "commands": [
  "# which discovery documents does a host serve?",
  "curl -s https://www.pathwren.workers.dev/mcp/doctor \\",
  "  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \\",
  "  -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"check_discovery_documents\",",
  "       \"arguments\":{\"host\":\"example.com\"}}}' \\",
  "  | jq -r '.result.structuredContent.documents[] | \"\\(.verdict)\\t\\(.path)\"'",
  "served      /robots.txt",
  "missing     /llms.txt",
  "missing     /.well-known/agent-card.json",
  "soft-404    /.well-known/mcp.json",
  "curl -s https://www.pathwren.workers.dev/mcp/doctor \\",
  "  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \\",
  "  -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"explain_document\",",
  "       \"arguments\":{\"name\":\"owners.json\"}}}' | jq -r '.result.structuredContent.observed_askers[]",
  "       | \"\\(.at)\\t\\(.ua)\"'",
  "2026-08-31T23:12:20Z\tVerifyMCP-OwnersBot/1.0 (+https://verifymcp.io/docs/build/owners-json)",
  "claude mcp add --transport http agent-discovery-doctor https://www.pathwren.workers.dev/mcp/doctor",
  "{",
  " \"mcpServers\": {",
  "  \"agent-discovery-doctor\": { \"type\": \"streamable-http\", \"url\": \"https://www.pathwren.workers.dev/mcp/doctor\" }",
  " }",
  "}"
 ],
 "sections": [
  {
   "heading": "Agent Discovery Doctor — MCP server",
   "text": [
    "An agent that meets your site for the first time does not read your homepage. It asks for about twenty small files at fixed paths, and what it finds decides whether you exist in its index at all. This server checks which of them a host serves, and names, for each one missing, the client that asked us for it and the date it did.",
    "Each missing line comes back with who asks for it, when they asked here, and what the 404 costs — not a style-guide opinion:"
   ],
   "commands": [
    "# which discovery documents does a host serve?",
    "curl -s https://www.pathwren.workers.dev/mcp/doctor \\",
    "  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \\",
    "  -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"check_discovery_documents\",",
    "       \"arguments\":{\"host\":\"example.com\"}}}' \\",
    "  | jq -r '.result.structuredContent.documents[] | \"\\(.verdict)\\t\\(.path)\"'",
    "served      /robots.txt",
    "missing     /llms.txt",
    "missing     /.well-known/agent-card.json",
    "soft-404    /.well-known/mcp.json",
    "curl -s https://www.pathwren.workers.dev/mcp/doctor \\",
    "  -H 'content-type: application/json' -H 'accept: application/json, text/event-stream' \\",
    "  -d '{\"jsonrpc\":\"2.0\",\"id\":1,\"method\":\"tools/call\",\"params\":{\"name\":\"explain_document\",",
    "       \"arguments\":{\"name\":\"owners.json\"}}}' | jq -r '.result.structuredContent.observed_askers[]",
    "       | \"\\(.at)\\t\\(.ua)\"'",
    "2026-08-31T23:12:20Z\tVerifyMCP-OwnersBot/1.0 (+https://verifymcp.io/docs/build/owners-json)"
   ],
   "tables": [],
   "links": []
  },
  {
   "heading": "Add it to a client",
   "text": [
    "Clients that take a JSON config (Claude Desktop, Cursor, VS Code, Windsurf):"
   ],
   "commands": [
    "claude mcp add --transport http agent-discovery-doctor https://www.pathwren.workers.dev/mcp/doctor",
    "{",
    " \"mcpServers\": {",
    "  \"agent-discovery-doctor\": { \"type\": \"streamable-http\", \"url\": \"https://www.pathwren.workers.dev/mcp/doctor\" }",
    " }",
    "}"
   ],
   "tables": [],
   "links": []
  },
  {
   "heading": "Tools",
   "text": [],
   "commands": [],
   "tables": [
    {
     "headers": [
      "Tool",
      "What it answers"
     ],
     "rows": [
      [
       "check_discovery_documents",
       "The whole job. GETs the 22 paths on a host you name and returns each as served, missing, gated or soft-404 — a 200 carrying an HTML error page, which is worse than a 404 because the reader believes it — with who asks for each missing one."
      ],
      [
       "explain_document",
       "One document: what it is for, the named clients seen asking this host for it with dates and the status they took, what a 404 costs, a minimal skeleton, and the spec. No argument returns the whole catalogue."
      ],
      [
       "validate_llms_txt",
       "Paste an llms.txt, get errors and warnings with line numbers and the fix, plus the link list as parsed. Checks the format, not your prose."
      ],
      [
       "llms_txt_from_sitemap",
       "Paste sitemap.xml or a list of URLs, get a draft llms.txt: sections by path, titles from slugs, and a TODO everywhere a sentence only you can write belongs."
      ],
      [
       "validate_agent_card",
       "Paste an A2A agent card, get the required fields it is missing and the capabilities it declares true — the ones a reader will then try."
      ]
     ]
    }
   ],
   "links": []
  },
  {
   "heading": "The 22 documents, and who actually asked",
   "text": [
    "The catalogue is not a reading of the specs. Every row below is a request that arrived at this host, with the user-agent as it came and the status it took:",
    "The other documents in the catalogue — ai.txt, api-catalog, ai-plugin.json, swagger.json, security.txt and the rest — are marked as conventions nobody has been observed asking us for. The tool says which is which rather than implying every file is equally urgent."
   ],
   "commands": [],
   "tables": [
    {
     "headers": [
      "Document",
      "Asked for here by",
      "When",
      "It got"
     ],
     "rows": [
      [
       "/.well-known/agent-card.json",
       "GolemreachTrustBot/0.1",
       "2026-09-01 00:48Z",
       "404 — then 200 on its return at 01:59Z, once we shipped one"
      ],
      [
       "/.well-known/agent.json",
       "GolemreachTrustBot/0.1",
       "2026-09-01 00:48Z",
       "404 — it asks for both paths in the same second"
      ],
      [
       "/.well-known/owners.json",
       "VerifyMCP-OwnersBot/1.0",
       "2026-08-31 23:12Z",
       "404 at / and at /mcp/, in the same second"
      ],
      [
       "/.well-known/oauth-protected-resource",
       "mcpbeat/0.1, exaforce-mcprep/0.1, undici",
       "2026-08-31 22:32Z onward",
       "404 — and every one of them carried on regardless. Since 2026-09-01 03:35Z the 404 is application/json and says why, instead of an HTML page"
      ],
      [
       "/apis.json and 11 more",
       "apis.io-submit/1.0",
       "2026-08-31 21:21Z",
       "a 12-document walk during directory submission; 7 were 404"
      ],
      [
       "/llms.txt",
       "ClaudeBot/1.0",
       "2026-09-01 01:10Z",
       "200"
      ],
      [
       "/.well-known/agent-card.json",
       "SaSameAgentAudit/0.1",
       "2026-09-01 01:06Z",
       "404"
      ],
      [
       "/.well-known/x402",
       "AgenstryBot/0.3.0",
       "2026-09-01 04:28Z",
       "404 — now 200. Payment discovery: accepts is empty because nothing here is paid, and the body says implemented: false so serving it is not mistaken for running the protocol"
      ]
     ]
    }
   ],
   "links": []
  },
  {
   "heading": "What it will not do",
   "text": [
    "It refuses to check this host. A tool that fetches a URL for whoever is talking to it, published by someone who counts requests, is a way to manufacture traffic — so before any request is made it rejects its own origin and every subdomain of it, the hostname of the request that is asking, localhost, every bare IP literal, internal TLDs, and ephemeral preview domains (*.trycloudflare.com, *.ngrok.io, *.vercel.app previews). The refusal names the host and the reason. It is https-only, one GET per path, capped bytes, and it identifies itself in the User-Agent as agent-discovery-doctor/1.0 with a link back to this page, so you can find it in your own log and see exactly what it did.",
    "The other four tools fetch nothing at all: text in, verdict out."
   ],
   "commands": [],
   "tables": [],
   "links": []
  },
  {
   "heading": "How is this different from the other two?",
   "text": [
    "ai-crawler-index answers questions about crawlers. crawler-log-triage reads a log you already have. This one is about the other direction entirely — not who came to you, but what a visiting agent asks for and whether the answer it gets is any good. No tool name, and no argument, is shared with either.",
    "Protocol versions 2025-06-18, negotiated per call. server/discover answers for clients on 2026-07-28, initialize for everyone else. Read-only, stateless, no key. Listed in the official MCP Registry as dev.workers.pathwren.www/agent-discovery-doctor."
   ],
   "commands": [],
   "tables": [],
   "links": [
    "/mcp.html",
    "/mcp-triage.html",
    "https://registry.modelcontextprotocol.io/v0/servers?search=agent-discovery-doctor"
   ]
  }
 ],
 "machine_doors": [
  {
   "url": "https://www.pathwren.workers.dev/tools/?s=client-dossiers",
   "name": "6 keyless GET tools",
   "what": "The read-only MCP tools of this host as plain GET endpoints — no JSON-RPC, no key"
  },
  {
   "url": "https://www.pathwren.workers.dev/documents.json",
   "name": "documents.json",
   "what": "Every document here with its strong ETag and the date its bytes changed"
  },
  {
   "url": "https://www.pathwren.workers.dev/changes",
   "name": "changes",
   "what": "What moved since your cursor — poll this instead of re-downloading anything"
  },
  {
   "url": "https://www.pathwren.workers.dev/llms.txt",
   "name": "llms.txt",
   "what": "The whole map in one text file"
  },
  {
   "url": "https://www.pathwren.workers.dev/openapi.json",
   "name": "openapi.json",
   "what": "Every read endpoint, described formally"
  },
  {
   "url": "https://www.pathwren.workers.dev/.well-known/agent-card.json",
   "name": "agent card",
   "what": "A2A agent card"
  },
  {
   "url": "https://www.pathwren.workers.dev/mcp",
   "name": "mcp",
   "what": "MCP over JSON-RPC (POST)"
  },
  {
   "url": "https://www.pathwren.workers.dev/a2a",
   "name": "a2a",
   "what": "A2A (POST message/send)"
  }
 ],
 "links": [
  {
   "rel": "self",
   "href": "https://www.pathwren.workers.dev/mcp-doctor.json",
   "type": "application/json"
  },
  {
   "rel": "describes",
   "href": "https://www.pathwren.workers.dev/mcp-doctor.html",
   "type": "text/html",
   "title": "The page this document is the JSON twin of: MCP server — Agent Discovery Doctor"
  },
  {
   "rel": "changes",
   "href": "https://www.pathwren.workers.dev/changes.json?since=111",
   "type": "application/json",
   "title": "What changed since your cursor — poll this instead of re-downloading this document",
   "cursor_param": "since",
   "head_cursor": 111,
   "min_poll_seconds": 21600,
   "how": "Read `cursor` from the response and send it back as `since`. It advances only when something really changed, so an unchanged answer is proof rather than luck — about 2.5 KB, or a 304 with no body if you send back the ETag."
  },
  {
   "rel": "related",
   "href": "https://www.pathwren.workers.dev/documents.json",
   "type": "application/json",
   "title": "Every document here with its ETag and last-modified date"
  },
  {
   "rel": "related",
   "href": "https://www.pathwren.workers.dev/data/agents.json",
   "type": "application/json",
   "title": "Every crawler record in one file"
  },
  {
   "rel": "service-desc",
   "href": "https://www.pathwren.workers.dev/openapi.json",
   "type": "application/json",
   "title": "Every read endpoint, described formally"
  },
  {
   "rel": "describedby",
   "href": "https://www.pathwren.workers.dev/llms.txt",
   "type": "text/plain",
   "title": "The whole map in one text file"
  }
 ]
}