Install guides

Install on Cloudflare Pages

One middleware file for a Cloudflare Pages project.

Steps

  1. Create an API token on the Account page and copy it.
  2. In your Pages project, add INTERCRAWL_TOKEN as a secret under Settings, Variables and Secrets.
  3. Save the install file in your project:
Save asfunctions/_middleware.tsintercrawl.com/install/v1/pages.ts
  1. Deploy, then press Check install on the Sites page.

Cloudflare gives the function each visitor's real IP address, so every crawler request is checked against its operator's published addresses.