About this bot

MetrynScanner

If you see a request in your logs identifying itself as MetrynScanner/1.0 (+https://getmetryn.com/scanner-info), it came from Metryn — a tool that runs technical AI-search-readiness diagnostics against a single web page at the explicit request of one of our users.

What it does

It fetches the exact URL a user submitted and its robots.txt and sitemap.xml (and, when that file is a sitemap index, up to three of the sitemaps it lists, to confirm they are valid), to check crawler access, indexability, and structured data. It does not crawl your site beyond that one URL, does not follow internal links, and does not run on a schedule — every request is a one-off, human-triggered check.

The free tools

The free tools at /tools use the same user-agent. The robots.txt checker requests one file, /robots.txt, from the site a visitor enters. The llms.txt validator requests /llms.txt and /llms-full.txt, then makes up to 25 HEAD or GET requests, three at a time, to addresses listed in the file, on the same host by default. Each visitor is limited to 20 tool checks per hour.

Rate limits we respect on our side

Requests are bounded per user: at most 10 scans per hour per visitor, and at most 3 scans per hour for any single target domain, regardless of how many visitors submit it.

Block it if you want

Before fetching the page, the scanner reads your robots.txt. If a group that names MetrynScanner disallows the URL, the page is not fetched, no score is produced, and the user who submitted it is told that the site’s robots.txt disallows MetrynScanner. A wildcard (User-agent: *) group is not applied to this user-initiated check, so that site owners can still see how their own site looks to AI crawlers.

If a firewall or bot-protection service blocks the scanner, the user sees a message saying so. To let Metryn through, allow the user agent MetrynScanner in your firewall. We do not try to get around bot protection.

Contact

Questions or concerns about this traffic: support@getmetryn.com.