Get started
The free parity scan
What the free scan fetches, what it compares, and how long it takes.
The free scan shows, before you install anything, whether search engine and AI crawlers receive the same page a visitor does.
What it fetches
From the URL or sitemap you enter, Intercrawl finds up to 100 pages: from the sitemaps named in your robots.txt, then /sitemap.xml, otherwise from the links on your homepage. The homepage always comes first. Each page is fetched four ways:
| View | How |
|---|---|
| Human | A browser with JavaScript on |
| Googlebot | A browser with JavaScript on, with Googlebot's user agent |
| GPTBot | A plain request with GPTBot's user agent: the HTML your server sends, no JavaScript |
| ClaudeBot | A plain request with ClaudeBot's user agent: the HTML your server sends, no JavaScript |
robots.txt is checked for each crawler first; a page it disallows is marked Blocked and not fetched. Every request carries X-Intercrawl-Scan: 1, so you can recognise the scan in your logs.
What it compares
For each crawler, the scan compares the crawler's page with the human view: the status code, the visible words, the internal links, the title and the canonical. Reading results explains each status.
Time and limits
A scan of 100 pages takes about 10 to 15 minutes. Each domain can be scanned once every 24 hours.
Who can see the results
A scan's link is unlisted: anyone with the link can open it, and it is not indexed. Signed out, the page shows the summary; the page-by-page results need an account. The browser that started the scan can make it public or delete it. Scans are deleted after 30 days.