curl -s https://www.pathwren.workers.dev/crawler/dotbot.json # this page, as JSON
No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a
Moz · SEO and backlink crawlers · json
User-agent: dotbot Disallow: /
| robots.txt token | dotbot |
| User-agent contains | dotbot |
| Operator | Moz |
| Category | SEO and backlink crawlers |
| robots.txt | obeys robots.txt (documented) |
| Verify by | no published verification method |
Moz's crawler for Link Explorer. Moz documents that it respects robots.txt and that dotbot is the token to name.
You leave Moz's link index, so Domain Authority and link reports about your site get thinner. Nothing a reader or an assistant sees changes.
Mozilla/5.0 (compatible; DotBot/1.2; +https://opensiteexplorer.org/dotbot; help@moz.com)
User-agent: dotbot Allow: /
Operator documentation: https://moz.com/help/moz-procedures/crawlers/dotbot
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · block-seo-tools