curl -s https://www.pathwren.workers.dev/crawler/google-cloudvertexbot.json # this page, as JSON
No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a
Google · AI search crawlers · json
User-agent: Google-CloudVertexBot Disallow: /
| robots.txt token | Google-CloudVertexBot |
| User-agent contains | Google-CloudVertexBot |
| Operator | |
| Category | AI search crawlers |
| robots.txt | obeys robots.txt (documented) |
| Verify by | published IP ranges |
| Published ranges | 136 IPv4 + 136 IPv6 · json · source |
Crawls a site on behalf of a Vertex AI Agent Builder customer who is building an agent over that site. It only visits sites the customer has asked it to.
Third parties can no longer build Vertex AI agents that read your site. Irrelevant to Google Search.
Mozilla/5.0 (compatible; Google-CloudVertexBot/1.0; +https://developers.google.com/search/docs/crawling-indexing/google-common-crawlers)
User-agent: Google-CloudVertexBot Allow: /
Operator documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · block-all-ai · allow-ai-search-only · maximum-ai-visibility