Help centre
How to install Intercrawl on your site, what it does for each crawler request, and how to read what it shows you. Questions this does not answer: hello@intercrawl.com.
Get started
- Getting startedFrom a free scan to a site whose crawlers get rendered pages, in five steps.
- How serving worksWhat happens when a crawler asks for one of your pages, from the first request to the fallback.
- Verify your siteProve that a site is yours, so Intercrawl serves rendered pages for it.
- The free parity scanWhat the free scan fetches, what it compares, and how long it takes.
Install guides
- Choose an installWhich install fits your site, and what each one asks of you.
- Install on Next.jsOne middleware file for a Next.js app, wherever it is hosted.
- Install on Vercel (other frameworks)One middleware file for a site on Vercel built with anything but Next.js.
- Install on NetlifyOne edge function file for a site on Netlify.
- Install on Cloudflare PagesOne middleware file for a Cloudflare Pages project.
- Install as a Cloudflare WorkerA small Worker in front of any site whose domain is proxied by Cloudflare.
- Prerender-compatible APICall the render API from your own server or middleware, in the format Prerender's middlewares use.
- DNS modePoint a hostname at Intercrawl with one DNS record. No code, any host.
- Add to VercelOne routing rule in your Vercel project, added for you. No code and no deploy.
- Add to CloudflareApprove Intercrawl once on Cloudflare, and a small Worker is placed in front of your site.
- Install with AIA prompt for your AI coding tool that adds the right install file to your project.
Move to Intercrawl
- Move from Prerender.ioKeep your Prerender middleware and point it at Intercrawl: a new service address and token, then the re-cache calls.
- Move from self-hosted prerender or RendertronReplace your own rendering server with Intercrawl: a new service address for Prerender's middlewares, or an Intercrawl install in place of Rendertron's.
Using Intercrawl
- Reading resultsWhat each status means, for a page and a crawler, in scans and audits.
- Audits and alertsThe daily check of your installed site, and the messages Intercrawl sends when something changes.
- ReportsA month of one site's audits as a report, a PDF or a share link, with or without Intercrawl's branding.
- Crawler policyAllow, block or charge each AI crawler per path, on your own Cloudflare zone.
- Share cardsA 1200 x 630 preview image for every page, made from its own title and image, for link previews.
- Crawlers Intercrawl recognisesThe crawlers that get a rendered page, and how each one's requests are checked.
Help
- Response headersThe X-Intercrawl headers on a crawler's answer, and what each value means.
- TroubleshootingWhat Check install's answers mean, and what to do about each.
- Questions and answersShort answers about what Intercrawl does, and does not do.
- SupportHow to reach Intercrawl, and what to send so the answer comes quickly.