Read the score as a compressed audit#
A readiness score compresses many checks into a number. The number is only as useful as its evidence: which URL was requested, when it was requested, which redirects occurred, the response status and headers, what appeared in the raw response, what appeared after rendering, and how each check contributed. CrawlReady documents its three components—Crawlability, Agent Readiness, and Agent Interaction—on how scoring works.
Do not use a score as a claim that a named bot saw a page. A scan’s raw and rendered outputs are observed artifacts from its test environment. They are evidence about the tested delivery path, not a universal contract for all crawler implementations, locations, cache states, credentials, or future deployments.
Create an evidence packet before changing anything#
For every material finding, retain a packet a colleague can inspect. Include canonical URL and test timestamp; redirect chain; final status; robots response used by the checker; raw HTML capture or hash; rendered capture or hash; extracted text; detected structured-data blocks; screenshots when interaction is relevant; and the rule version. Redact credentials and personal data.
| Finding | Supporting observation | Sensible next check |
|---|---|---|
| Main answer absent from raw HTML | raw text lacks the heading and paragraph | inspect SSR or prerender output |
| Important script blocked | robots response disallows a needed resource | confirm whether rendering degrades |
| Bot appears blocked | directive matches an identified agent token | compare to the operator’s current documentation |
| Schema parse error | validator names an invalid property | fix the page markup and retest |
Google explains that robots.txt manages crawler access and is not a reliable mechanism for hiding pages from Search. Its robots guide is the source to consult before interpreting a directive as an indexing result.
Separate observation, interpretation, and action#
Write each recommendation in three parts. Observation: “The scanner received 200 HTML; the primary answer was absent from the captured raw text.” Interpretation: “A non-rendering client would not receive that answer in this observed response.” Action: “Render the answer server-side or expose an equivalent accessible HTML representation, then retest.” This wording is specific without inventing a named bot behavior.
Use a confidence label as well: direct response artifact, configuration inference, or third-party verification. For identity claims, do not rely on user-agent text. Google documents reverse DNS followed by forward DNS, or matching against published IP ranges, for Google crawler verification; see Verify requests from Google. Keep the original IP and lookup result under appropriate retention controls rather than storing only “Googlebot: yes.”
Prioritize fixes by reversibility and blast radius#
Start with issues that prevent a page from being delivered or understood: server errors, accidental authentication, incorrect canonical or robots configuration, missing raw content, and broken structured data. Then test interaction paths such as navigation, forms, and key links. Change one deployment variable at a time when feasible, record the release ID, and rescan the same URL under the same conditions.
For OpenAI-specific configuration, use the current OpenAI crawler documentation. It distinguishes OAI-SearchBot from GPTBot and says their robots controls are independent. That supports a configuration check; it does not support a claim that any page will appear in a ChatGPT answer. Use the Agent Readiness documentation to understand what CrawlReady tests and a free scan to collect a current URL-level report.
Scope and limits. This procedure creates an audit trail, not a promise of crawler access, inclusion, indexing, citations, referral traffic, rankings, or conversions. User-agent text is easy to spoof. Treat the records as observed request artifacts unless the requester is verified by the relevant operator method. Keep the raw event, the verification result, and the interpretation separate so a later reviewer can reproduce the conclusion.
A short retest protocol#
Use the same canonical URL and record the initial report identifier. Make one scoped change, deploy it, then collect a second report after the deployment is serving. Compare only the affected observations: status, raw-text presence, rendered-text presence, directives, markup, and interaction evidence. If a result changed unexpectedly, inspect cache keys, redirects, experiment flags, locale, and authentication before crediting the code change. Keep an “unknown” result where the available artifact cannot answer the question.
Ask a reviewer to read the evidence without the score first. If they cannot tell why a check passed or failed, improve the evidence label or capture. That discipline makes a score usable in engineering triage instead of a vague executive signal.
Reporting language that stays accurate#
Use “the scan observed” for captures, “the configuration indicates” for directives and settings, and “the operator documents” for a crawler’s stated behavior. Avoid “the bot could not see” unless a verified request and matching response supply that evidence. State the test limitation when a URL requires a locale, cookie, account, or a particular cache state. This wording is not cosmetic: it prevents a team from applying a generic scan result to a different path or audience.
A high-severity result should also include a rollback-safe fix and the condition that will prove it resolved. For example, “raw capture contains the main answer after release” is testable; “improve AI visibility” is not.
FAQ#
What should an AI Readiness Score include as evidence?#
At minimum, retain the URL, time, redirects, response status, raw and rendered artifacts, check results, and rule version. That lets another reviewer reproduce the finding.
Does a higher score guarantee citations or rankings?#
No. A score describes observed technical conditions and configured signals. It does not guarantee indexing, citations, rankings, traffic, or conversion.
Why keep raw and rendered artifacts separately?#
They answer different questions. Raw HTML shows the response artifact without client execution; rendered output shows what the test renderer observed after execution.