Get started
Verify your site
Prove that a site is yours, so Intercrawl serves rendered pages for it.
Intercrawl renders pages only for a site whose owner has proved control of it. Until then, the install answers crawlers with your own pages. Each site on the Sites page shows its verification token and these ways to use it.
A DNS record
Add a TXT record at your DNS provider:
| Field | Value |
|---|---|
| Type | TXT |
| Host | _intercrawl (or the root of your domain) |
| Value | intercrawl-site-verification= followed by your token |
Then press Verify. DNS changes can take a few minutes to be seen.
A meta tag on your homepage
Add this to the <head> of your homepage, with your token, deploy, then press Verify:
<meta name="intercrawl-site-verification" content="YOUR_TOKEN">
The tag must be in the HTML your server sends, not added later by JavaScript.
Through Vercel or Cloudflare
Add to Vercel and Add to Cloudflare verify the site as part of connecting it: Intercrawl confirms that the Vercel project or Cloudflare zone you approved really serves the domain. No record or tag is needed.
A site is one domain. www. is the same site as the domain without it; another subdomain (app.example.com) is a site of its own.