Connect a Vercel site to AI Crawlers

This guide connects a site hosted on Vercel to AI Crawlers with a Vercel log drain, so you can see which AI bots read it. There is no code to deploy and no DNS to change: Vercel sends us your project's request logs, and we keep the bot requests only. Read it all, including the cost, before you start.

Before you start

• Vercel plan: Vercel Pro or Enterprise only (not a Pro Trial). On Hobby? Use Cloudflare if your domain is on it.
• You need: access to the team's settings in Vercel, and your site key from AI Peekaboo. It is shown once.
• Where: every step after the key is done in your Vercel team settings.
• If someone else manages your Vercel team, choose Copy steps for a teammate on the setup screen, and send them the key over a private channel such as a password manager, never email or chat.

What it costs

Vercel bills you per GB of log records, visitors included. It is $0.50 per GB on Vercel Pro; Enterprise has custom pricing. Check your Usage page in Vercel after a day.

A drain carries every request to the project, not only bot requests, so a busy site sends more. Pick only the project that serves your site, as in step 3: choosing all projects sends us the requests of every project, and Vercel bills you for all of them.

What is sent

We receive every request for this project and keep bots only:

• Every request to this project, visitors included
• Address with query, status, time, user agent, linking site, visitor IP unless hidden, and more
• Your functions' log output (not kept)
• We keep bot requests only, never IPs or query strings

Visitor IPs are sent unless you hide them; we never read them. To hide visitor IPs, use IP Address Visibility in your team Settings, under Security & Privacy (it applies to your whole Vercel team).

1. Choose Vercel and create your key

1. Open AI Crawlers in the sidebar and pick your brand in the header. If the page says Set your site URL first, add your site address in Settings.
2. Under "How is your site served?", choose Vercel.
3. Choose Create key and start setup. The key window shows a Custom header line to copy. Copy it right away: we can't show the key again.

Wait at least a minute before step 5. A new key takes about a minute to start working, and Vercel tests the drain as soon as you create it.

2. Add a Logs drain in Vercel

Open Drains in your Team Settings, choose Add Drain, then Logs.

3. Pick this site's project

Name the drain, then select only the project that serves your site.

4. Narrow what it sends

In Additional configuration for logs: production only, every source except build. Keep the firewall source on: it is how blocked bots show.

5. Send it to AI Peekaboo

1. Choose Custom Endpoint, then Format NDJSON (JSON works too).
2. Paste the Endpoint URL shown on your setup screen.
3. In Custom Headers, paste the header line from the key window. It is one line: x-crawler-drain-key: then your key.
4. Leave the Signature Verification Secret as Vercel made it. We check the header, not the signature.

6. Create Drain, then check

1. Select Create Drain. If Vercel's test fails, wait a minute and try again.
2. Open your site in a browser, then press Check now in AI Crawlers.

Your own visit shows that the drain is working, but it is never stored as a bot's read. AI Crawlers explains what happens once real bots arrive.

Nothing arriving?

• The drain's project must be the one that serves your site.
• Custom Headers: paste the header line from the key window.
• Environment production, and sources other than build.
• If you added sampling rules, unmatched requests are dropped.
• A new key takes about a minute.

Lost your key? Use New key on the AI Crawlers page. Your drain sends nothing we accept until its header holds the new key: edit the drain's header, or delete the drain and add it again.

To stop it

Under Drains in your Team Settings, open the drain's menu and select Pause or Delete.

Disconnecting in AI Crawlers revokes your key, but it does not stop the drain. The drain keeps sending, and Vercel keeps billing it, until you pause or delete it. Vercel e-mails your team once more than 80% of the drain's deliveries, or more than 50 of them, fail in an hour.

Need help?

Not sure what a screen means, or something looks wrong? Stop there and write to team@aipeekaboo.com or see Contact support. Tell us which step you were on.