curl -s https://www.pathwren.workers.dev/operator/google.json   # this page, as JSON

No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a

Google

26 crawler(s) · json

CrawlerTokenCategoryrobots.txt
AdsBot-GoogleAdsBot-GoogleTools and frameworksignores the * group; obeys rules named for its own token
AdsBot-Google-MobileAdsBot-Google-MobileTools and frameworksignores the * group; obeys rules named for its own token
AdsBot-Google-Mobile-AppsAdsBot-Google-Mobile-AppsTools and frameworksignores the * group; obeys rules named for its own token
APIs-GoogleAPIs-GoogleTools and frameworksignores the * group; obeys rules named for its own token
FeedFetcher-GoogleFeedFetcher-GoogleTools and frameworksnot governed by robots.txt (user-initiated, by operator policy)
Google-AgentGoogle-AgentUser-triggered fetchersnot governed by robots.txt (user-initiated, by operator policy)
Google-CloudVertexBotGoogle-CloudVertexBotAI search crawlersobeys robots.txt (documented)
Google-CWSGoogle-CWSTools and frameworksnot governed by robots.txt (user-initiated, by operator policy)
Google-ExtendedGoogle-ExtendedAI training crawlerscontrol token only — no crawler
Google-GeminiNotebookGoogle-GeminiNotebookUser-triggered fetchersnot governed by robots.txt (user-initiated, by operator policy)
Google-InspectionToolGoogle-InspectionToolTools and frameworksobeys robots.txt (documented)
Google-PinpointGoogle-PinpointUser-triggered fetchersnot governed by robots.txt (user-initiated, by operator policy)
Google-Read-AloudGoogle-Read-AloudUser-triggered fetchersnot governed by robots.txt (user-initiated, by operator policy)
Google-SafetyGoogle-SafetyTools and frameworksnot governed by robots.txt (user-initiated, by operator policy)
Google-Site-VerificationGoogle-Site-VerificationTools and frameworksnot governed by robots.txt (user-initiated, by operator policy)
GooglebotGooglebotSearch enginesobeys robots.txt (documented)
Googlebot-ImageGooglebot-ImageSearch enginesobeys robots.txt (documented)
Googlebot-NewsGooglebot-NewsSearch enginesobeys robots.txt (documented)
Googlebot-VideoGooglebot-VideoSearch enginesobeys robots.txt (documented)
GoogleMessagesGoogleMessagesLink preview fetchersnot governed by robots.txt (user-initiated, by operator policy)
GoogleOtherGoogleOtherAI training crawlersobeys robots.txt (documented)
GoogleOther-ImageGoogleOther-ImageAI training crawlersobeys robots.txt (documented)
GoogleOther-VideoGoogleOther-VideoAI training crawlersobeys robots.txt (documented)
GoogleProducerGoogleProducerTools and frameworksnot governed by robots.txt (user-initiated, by operator policy)
Mediapartners-GoogleMediapartners-GoogleTools and frameworksignores the * group; obeys rules named for its own token
Storebot-GoogleStorebot-GoogleSearch enginesobeys robots.txt (documented)

Published IP ranges

EndpointIPv4IPv6Operator timestamp
Googlebot1701472026-09-02T14:45:50.000000
Google special-purpose crawlers1361362026-09-02T14:45:54.000000
Google user-triggered fetchers5295292026-09-02T14:45:44.000000
Google user-triggered fetchers (Google-owned ranges)2482482026-09-02T14:45:50.000000
Google-Agent (agents on Google infrastructure)312026-03-03T10:00:00.000000

Documentation: https://developers.google.com/search/docs/crawling-indexing/overview-google-crawlers