Next.js

Middleware Setup#

Install the package:

npm install @crawlready/middleware
# or
pnpm add @crawlready/middleware

Create middleware.ts in your project root:

// middleware.ts
import { withCrawlReady } from '@crawlready/middleware/next';

export default withCrawlReady({
  siteKey: process.env.CRAWLREADY_SITE_KEY!,
});

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
};

Add your site key to .env.local:

CRAWLREADY_SITE_KEY=cr_live_xxxxx

For Vercel deployments, add CRAWLREADY_SITE_KEY in your project's Environment Variables settings.

Composing with existing middleware#

If you already have middleware (e.g. Clerk), pass it as the second argument:

// middleware.ts
import { withCrawlReady } from '@crawlready/middleware/next';
import { clerkMiddleware } from '@clerk/nextjs/server';

export default withCrawlReady(
  { siteKey: process.env.CRAWLREADY_SITE_KEY! },
  clerkMiddleware(),
);

export const config = {
  matcher: ['/((?!_next/static|_next/image|favicon.ico).*)'],
};

withCrawlReady detects AI bots, fires an analytics beacon via event.waitUntil() (so it never blocks the response), then hands off to your existing middleware.

For all configuration options, see Core Concepts — Middleware Configuration.

Instant indexing (optional)#

Add indexNowKey to publish new and changed pages to Bing — and through Bing's index, ChatGPT search and Microsoft Copilot — within minutes instead of waiting for a crawl. Free on every plan, no daily limit.

export default withCrawlReady({
  siteKey: process.env.CRAWLREADY_SITE_KEY!,
  indexNowKey: process.env.CRAWLREADY_INDEXNOW_KEY!,
});

Get your key from Indexing → Settings in the dashboard. withCrawlReady answers the verification request itself once the key is set — nothing to deploy separately. See Get Indexed by Bing, ChatGPT & Copilot for the full setup.

Verify Your Integration#

After deploying, use the Verify Integration button in your CrawlReady dashboard. It sends a synthetic GPTBot request to your site and confirms that a visit is recorded within 15 seconds.

If the check times out:

• Confirm middleware.ts is in your project root (or src/ directory if you use that layout).
• Confirm the environment variable is set in your deployment environment, not just .env.local.
• Check that your matcher pattern covers the paths being tested.