curl -s https://www.pathwren.workers.dev/category/archive.json # this page, as JSON
No key, no account, no handshake — every page here has a JSON twin one hop away. Machine doors: 6 keyless GET tools · documents.json · changes · llms.txt · openapi.json · agent card · mcp · a2a
Preservation crawlers. Their output is public and permanent, which makes them a separate decision from the AI one.
| Crawler | Token | Operator | Cost of blocking |
|---|---|---|---|
| archive.org_bot | archive.org_bot | Internet Archive | Your site stops being preserved. When it dies, it is gone. Consider this one separately fr… |
| ia_archiver | ia_archiver | Internet Archive | Negligible today; retain for tidiness.… |