# CrawlReady > AI readiness scoring, monitoring, and optimization for websites. Scans a URL and > reports whether AI crawlers (GPTBot, OAI-SearchBot, PerplexityBot, ClaudeBot, > Google-Extended) can actually see and act on the content. - [Free AI Readiness Scan](https://www.crawlready.app/scan): Run a scan and get an AI Readiness Score with sub-scores and fix recommendations. - [AI Readiness Leaderboard](https://www.crawlready.app/leaderboard): Public rankings of scanned sites by AI Readiness Score. - [Schema Markup Analyzer](https://www.crawlready.app/schema-checker): Free structured-data checker for JSON-LD, Microdata, and RDFa. - [MCP Server](https://www.crawlready.app/mcp): Query CrawlReady's scan, diff, recommend, and indexing tools from Claude, Cursor, ChatGPT, or any MCP client. - [Indexing](https://www.crawlready.app/indexing): Push new and changed pages straight into the Bing index (ChatGPT search, Copilot) and the Google index (Search, AI Overviews) in minutes instead of waiting on a crawl schedule. - [AI Visibility](https://www.crawlready.app/ai-visibility): AI crawler analytics — see which AI bots visit a site, what they access, and how often. ## Packages > Open source, published to npm under the @crawlready scope (plus the crawlready CLI). - [crawlready](https://www.npmjs.com/package/crawlready): AI readiness CLI — scan, diagnose, and fix a site for AI crawlers. - [@crawlready/core](https://www.npmjs.com/package/@crawlready/core): AI bot detection engine — identify GPTBot, ClaudeBot, PerplexityBot, and 80+ other crawlers from User-Agent strings. - [@crawlready/middleware](https://www.npmjs.com/package/@crawlready/middleware): AI crawler analytics and optimized content serving — framework adapters for Next.js, Express, Cloudflare Workers, and Node.js. - [@crawlready/mcp](https://www.npmjs.com/package/@crawlready/mcp): MCP server — scan and optimize sites for AI crawlers from Claude, Cursor, or ChatGPT. See https://www.crawlready.app/mcp. ## Docs ### Getting Started - [What is CrawlReady](https://www.crawlready.app/docs/introduction): CrawlReady scores your website's AI readiness and tracks which AI crawlers visit, what they access, and how often. - [Core Concepts](https://www.crawlready.app/docs/core-concepts): Understand the core concepts behind CrawlReady: AI readiness scores, site keys, API keys, and integration types. - [Quickstart](https://www.crawlready.app/docs/quickstart): Get CrawlReady integrated and tracking AI crawler visits to your site in under 5 minutes, no rebuild required. ### Integrations - [Next.js](https://www.crawlready.app/docs/integrations/nextjs): Integrate CrawlReady with your Next.js application using middleware to track AI crawler visits automatically, with no extra setup. - [Express](https://www.crawlready.app/docs/integrations/express): Integrate CrawlReady with your Express.js application using middleware to track and analyze AI crawler traffic. - [Cloudflare Workers](https://www.crawlready.app/docs/integrations/cloudflare-workers): Integrate CrawlReady with Cloudflare Workers to track AI crawler traffic and serve optimized content at the edge. - [Vercel Edge Middleware](https://www.crawlready.app/docs/integrations/vercel-edge): Integrate CrawlReady with Vercel Edge Middleware to track AI crawler traffic without adding latency to your app. - [Script Tag & Client-Side Sites](https://www.crawlready.app/docs/integrations/script-tag): Add CrawlReady analytics tracking to client-side sites and SPAs with a single script tag — no server changes required. ### Analytics - [Analytics Overview](https://www.crawlready.app/docs/analytics/overview): Track which AI crawlers visit your site, what content they access, and how your visibility changes over time, in one dashboard. - [Analytics Dashboard](https://www.crawlready.app/docs/analytics/dashboard): Read and interpret your AI crawler analytics dashboard, including crawl volume, bot breakdown, and page-level visibility trends. - [Alerts](https://www.crawlready.app/docs/analytics/alerts): Get notified by email or webhook when AI crawler activity changes on your site, including new bots, spikes, and blocked requests. - [Export](https://www.crawlready.app/docs/analytics/export): Download your AI crawler analytics data as CSV or JSON for offline analysis, reporting, or integration with other tools. ### Scores - [How Scoring Works](https://www.crawlready.app/docs/scores/how-scoring-works): Understand CrawlReady's AI Readiness Score, a unified 0-100 metric combining crawlability, agent readiness, and interaction. - [Crawlability](https://www.crawlready.app/docs/scores/crawlability): What the Crawlability score measures, why it matters for AI search visibility and citations, and how to improve it. - [Agent Readiness](https://www.crawlready.app/docs/scores/agent-readiness): What the Agent Readiness score measures, why it matters for autonomous AI agents acting on your site, and how to improve it. - [Agent Interaction](https://www.crawlready.app/docs/scores/agent-interaction): What the Agent Interaction score measures, why it matters for autonomous AI agents, and how to improve it on your site. - [Public Score Pages](https://www.crawlready.app/docs/scores/public-score-pages): Share your AI readiness score with a public, shareable URL and an embeddable badge to show visitors your site is AI-ready. ### Indexing - [How AI Assistants Index Your Website](https://www.crawlready.app/docs/indexing/overview): See which AI assistants read your pages, which search index each one uses, and how to get new content indexed in minutes instead of weeks. - [Get Indexed by Bing, ChatGPT & Copilot](https://www.crawlready.app/docs/indexing/bing-and-chatgpt): Push new and changed pages into the Bing index in minutes via IndexNow — what ChatGPT search and Copilot both read from. Free, unlimited, on every plan. - [Google Indexing API & AI Overviews](https://www.crawlready.app/docs/indexing/google-and-gemini): Use the Google Indexing API to push pages straight into Google's index, see what actually got indexed, and scale past the 200-a-day cap. - [Automate Indexing via CI & Webhooks](https://www.crawlready.app/docs/indexing/sitemaps-and-automation): Submit URLs for indexing from your sitemap, CI pipeline, deploy webhook, or a single curl command — every route hits the same guarded pipeline. - [Fix a Page That Won't Get Indexed](https://www.crawlready.app/docs/indexing/troubleshooting): Why a page's submission got skipped, what each coverage status means, and what to do when a search engine keeps declining to index it. ### Reference - [API Reference](https://www.crawlready.app/docs/api-reference): Interactive API documentation for the CrawlReady REST API, covering scans, sites, analytics, and API key management. ## Learn - [How to Rank on Perplexity, Copilot, Gemini, and Other AI Search Engines](https://www.crawlready.app/learn/ai-search-engines): Perplexity, Copilot, Gemini, and Claude each source citations differently — some read a shared index, some crawl independently. Here's what's documented and what to check for each. - [How ChatGPT Search Works](https://www.crawlready.app/learn/chatgpt-search): ChatGPT search reads from the Bing index, crawls separately via OAI-SearchBot, and — per practitioner crawl telemetry — still fetches HTML most of the time even when a markdown alternate exists. - [What Is Generative Engine Optimization (GEO)?](https://www.crawlready.app/learn/generative-engine-optimization): Generative Engine Optimization is making content usable as source material for AI-generated answers like Google AI Overviews and ChatGPT search — grounded in the same crawlability and indexing fundamentals as SEO. - [AI Search Glossary](https://www.crawlready.app/learn/glossary): Plain-language definitions for the terms that come up in AI search optimization — SEO, AEO, GEO, the major AI crawlers, and the mechanics behind AI-generated answers. - [What Is an AI Overview on Google, and Can You Trust It?](https://www.crawlready.app/learn/google-ai-overviews): AI Overview is a Google Search feature that generates a written summary above the normal results, drawn from Google's search index. Here's how it works and how reliable it is. - [SEO vs. AEO vs. GEO: What's the Difference?](https://www.crawlready.app/learn/seo-vs-aeo-vs-geo): SEO, AEO, and GEO target search rankings, direct answers, and AI overviews respectively — but Google says optimizing for any of them is still SEO. - [What Is Answer Engine Optimization (AEO)?](https://www.crawlready.app/learn/answer-engine-optimization): Answer Engine Optimization is structuring content so a system can extract one direct, correct answer from it — for featured snippets, voice assistants, and AI chat answers. - [How to Appear in Google AI Overviews](https://www.crawlready.app/learn/how-to-appear-in-google-ai-overviews): Appearing in Google AI Overviews requires the same crawlability and indexing fundamentals as ranking in Search, plus confirmed inclusion in Search generative AI features. - [How to Rank in ChatGPT](https://www.crawlready.app/learn/how-to-rank-in-chatgpt): Ranking in ChatGPT search answers requires allowing OAI-SearchBot, being indexed by Bing, and structuring content for extraction — not switching to markdown-only serving. - [What Is llms.txt?](https://www.crawlready.app/learn/llms-txt): llms.txt is a proposed convention file that points AI systems to a curated, machine-readable summary of a site's content. Google says it's not required for AI Overviews; practitioner crawl data shows OpenAI and Anthropic bots fetch it anyway. - [What Is SEO, and What Changed With AI Search?](https://www.crawlready.app/learn/seo): SEO is optimizing a site to be found and ranked by search engines. AI Overviews and AI chat answers changed where that work shows up — not what the work is. ## Blog - [Why ChatGPT Can't See Your JavaScript Site](https://www.crawlready.app/blog/why-chatgpt-cant-see-your-javascript-site): 98.9% of websites use JavaScript, but only 31% of AI crawlers can render it. Here's why your React, Vue, or Angular site is invisible to ChatGPT and what you can do about it. - [How We Built the AI Crawler Checker](https://www.crawlready.app/blog/how-we-built-the-ai-crawler-checker): A deep dive into the technical architecture behind our free AI crawler compatibility tool. Learn how we simulate different AI crawlers and analyze JavaScript rendering.