curl -s https://www.pathwren.workers.dev/crawler/facebookexternalhit.json # this page, as JSON
No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a
Meta · Link preview fetchers · json
User-agent: facebookexternalhit Disallow: /
| robots.txt token | facebookexternalhit |
| User-agent contains | facebookexternalhit |
| Operator | Meta |
| Category | Link preview fetchers |
| robots.txt | obeys robots.txt (documented) |
| Verify by | no published verification method |
The link unfurler: it reads your Open Graph tags when somebody shares your URL on a Meta property.
Severe and usually accidental. Your links share as bare grey boxes with no title, image or description across Facebook, Instagram, Messenger and WhatsApp. Almost nobody means to block this.
facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)
User-agent: facebookexternalhit Allow: /
Operator documentation: https://developers.facebook.com/docs/sharing/webmasters/web-crawlers
Machine copies: json ·
markdown
Policies that name this crawler:
allow-all · allow-ai-search-only · maximum-ai-visibility