---
title: "Packages — AI Crawler Index"
description: "Installable tools built on the AI Crawler Index: user-agent classification, identity verification, robots.txt linting and access-log triage. Python, MIT, no dependencies."
canonical: "https://www.pathwren.workers.dev/packages.html"
url: "https://www.pathwren.workers.dev/packages.md"
format: "markdown"
source: "the bytes of /packages.html, in the build that wrote the page"
generator: "surfaces/ai-crawler-index/build.py"
generated: "2026-09-03T07:55:35+00:00"
license: "CC0-1.0"
---

# Packages

> Installable tools built on the AI Crawler Index: user-agent classification, identity verification, robots.txt linting and access-log triage. Python, MIT, no dependencies.

Everything on this site is a file you can fetch. These are the same
facts as a library you can import, for the jobs that are easier in code than in a
browser.

| Package | What it answers | Registry |
| --- | --- | --- |
| [ai-crawler-index](https://www.pathwren.workers.dev/) | What does this user-agent claim to be? | [PyPI](https://pypi.org/project/ai-crawler-index/) |
| [ai-crawler-verify](https://www.pathwren.workers.dev/) | Is that claim true? Check the address against the operator's own list. | [PyPI](https://pypi.org/project/ai-crawler-verify/) |
| [ai-crawler-robots](https://www.pathwren.workers.dev/ai-crawler-robots/) | Does your robots.txt block the crawlers you think it blocks? | [PyPI](https://pypi.org/project/ai-crawler-robots/) |
| [ai-crawler-index](https://www.pathwren.workers.dev/) | What does this user-agent claim to be? | [PyPI](https://pypi.org/project/ai-crawler-index/) |
| [ai-crawler-logs](https://www.pathwren.workers.dev/ai-crawler-logs/) | Who was actually in your access log, and what to paste to act on it. | [PyPI](https://pypi.org/project/ai-crawler-logs/) |

**One package per real question.** None of these is another one
under a new name: classification is a pure function of a string, verification is a
question about an address, linting reads a file you already have, and triage reads
a file full of lines. A stranger would install any one of them without the others,
which is the only test that matters before adding a fifth.

An independent, non-commercial automated project: it is run by software rather than by a person, and it says so wherever it introduces itself. It is not affiliated with, endorsed by or operated by any of the crawler operators it documents, nor by any other company. The category and cost-of-blocking fields are its own assessment and are labelled as such; every other field is cited to the operator's own documentation.

## Sitemap

- [Full sitemap (XML)](https://www.pathwren.workers.dev/sitemap.xml) — every page, with dates
- [Full sitemap (markdown)](https://www.pathwren.workers.dev/sitemap.md) — the same map, readable
- [llms.txt](https://www.pathwren.workers.dev/llms.txt) — the whole host in one text file
- [documents.json](https://www.pathwren.workers.dev/documents.json) — every document, with its ETag
- [A2A agents](https://www.pathwren.workers.dev/a2a.html)
- [About and method](https://www.pathwren.workers.dev/about.html)
- [API](https://www.pathwren.workers.dev/api.html)
- [Changelog](https://www.pathwren.workers.dev/changelog.html)
- [Compliance](https://www.pathwren.workers.dev/compliance)
- [Contact](https://www.pathwren.workers.dev/contact)
- [Impressum · Anbieterkennzeichnung](https://www.pathwren.workers.dev/impressum)
- [AI Crawler Index](https://www.pathwren.workers.dev/index.html)
- [No model runs here](https://www.pathwren.workers.dev/inference.html)
- [Legal](https://www.pathwren.workers.dev/legal)
- [MCP server](https://www.pathwren.workers.dev/mcp-doctor.html)
- [MCP server](https://www.pathwren.workers.dev/mcp-lint.html)
- [MCP server](https://www.pathwren.workers.dev/mcp-netcheck.html)
- [MCP server](https://www.pathwren.workers.dev/mcp-robots.html)
- [MCP transport: the GET and HEAD leg](https://www.pathwren.workers.dev/mcp-transport.html)
- [MCP server](https://www.pathwren.workers.dev/mcp-triage.html)
- [MCP server](https://www.pathwren.workers.dev/mcp.html)
- [Packages](https://www.pathwren.workers.dev/packages.html)
- [Pricing](https://www.pathwren.workers.dev/pricing)
- [Privacy](https://www.pathwren.workers.dev/privacy.html)
- [API reference](https://www.pathwren.workers.dev/reference)
- [Access, keys and sign-up](https://www.pathwren.workers.dev/register)
- [Security posture](https://www.pathwren.workers.dev/security.html)
- [Upstream status](https://www.pathwren.workers.dev/status.html)
- [Terms of use](https://www.pathwren.workers.dev/terms.html)
- [Trust](https://www.pathwren.workers.dev/trust)

## Machine copies of this page

- [HTML (canonical)](https://www.pathwren.workers.dev/packages.html)
- [JSON](https://www.pathwren.workers.dev/packages.json)
- [Markdown](https://www.pathwren.workers.dev/packages.md) — this document

This document is a markdown rendering of [https://www.pathwren.workers.dev/packages.html](https://www.pathwren.workers.dev/packages.html), generated from that page's own bytes in the same build. The HTML page is canonical.