AIWebIndex (Lyrenth)

Builds an index of public pages and serves them to AI agents as extracted readable text, with attribution and a link back. Lyrenth publishes a crawler policy stating it does not train foundation models on what it collects and that it obeys robots.txt. Blocking it costs you: Agents reading through this index stop seeing you — including the attribution and link back that make it a referral rather than a summary. robots.txt token: AIWebIndex.