Move to Intercrawl

Move from self-hosted prerender or Rendertron

Replace your own rendering server with Intercrawl: a new service address for Prerender's middlewares, or an Intercrawl install in place of Rendertron's.

If you run your own Prerender server or Rendertron, Intercrawl can take over the rendering, and you can turn the server off afterwards. For how the options differ, see Intercrawl and self-hosted prerendering.

Before you switch

Create an account and add your site on the Sites page.
Verify your site. The render API renders only verified sites.
Create an API token on the Account page, under API tokens. It starts with ic_live_ and is shown once.

Keep your server running until the switch is checked.

From your own Prerender server

Your middleware (prerender-node, prerender_rails, nginx, Apache or a community one) already speaks the format Intercrawl answers in. Point it at Intercrawl and add the token, which your own server did not need:

PRERENDER_SERVICE_URL=https://render.intercrawl.com/
PRERENDER_TOKEN=ic_live_...

For nginx, and for each setting in more detail, follow Move from Prerender.io: the steps are the same.

What your server's plugins did, Intercrawl does itself:

Prerender server plugin	With Intercrawl
removeScriptTags	Scripts are removed from every render, except structured data (JSON-LD)
httpHeaders	`<meta name="prerender-status-code">` and `<meta name="prerender-header">` work the same way
A cache plugin	Renders are kept 7 days and renewed when a page changes
whitelist, basicAuth	A token renders only the verified sites of its own account

From Rendertron

Rendertron's middleware sends crawlers' requests with no token, so it cannot be pointed at Intercrawl. Replace it:

On Next.js, Vercel, Netlify or Cloudflare: remove rendertron-middleware and add the Intercrawl install for your site. Choose an install lists them; most are one file.
On Express or another Node.js server: replace rendertron-middleware with Prerender's prerender-node middleware (MIT licence) and set it as above:

// Before: app.use(rendertron.makeMiddleware({ proxyUrl: "https://your-rendertron/render" }));
app.use(
  require("prerender-node")
    .set("prerenderServiceUrl", "https://render.intercrawl.com/")
    .set("prerenderToken", process.env.PRERENDER_TOKEN),
);

Anywhere else: DNS mode needs no code, and the Prerender-compatible API describes the requests if you write your own middleware.

Calls to Rendertron's /invalidate/<url> or refreshCache=true become re-cache requests, up to 1,000 URLs at a time.

Check the switch, then turn the server off

Deploy, then ask for a page as a crawler:

curl -sI -A "Mozilla/5.0 (compatible; Twitterbot/1.0)" https://example.com/

The answer should carry X-Intercrawl-Source: fresh the first time and cache after that. See Response headers. Run an audit from the dashboard to see what each crawler receives, then turn your rendering server off.

The cache starts empty: each page is rendered the first time a crawler asks for it, within 10 seconds, and a crawler whose render is not ready in time gets your own page. To fill the cache before crawlers arrive, send your most important URLs to the re-cache API.

Not answered here? Write to hello@intercrawl.com.