robots.txt policy: Maximum AI visibility

Allow every AI crawler and every search engine; refuse only SEO scrapers. For sites whose goal is to be found and cited by machines. If your content exists to be read by assistants — documentation, reference data, an API — every block costs you and none of them protect anything. Pair this with an llms.txt, a sitemap, and per-item JSON, and the crawlers can actually use what they find. curl -s https://www.pathwren.workers.dev/robots/maximum-ai-visibility.txt