Privacy Policy

Effective Date: January 13, 2026

Last Updated: January 13, 2026

1. Introduction

CrawlReady ("we," "our," or "us"), provides an AI readiness platform that scores, monitors, and optimizes websites for artificial intelligence crawlers and agents ("Services"). This Privacy Policy describes how we collect, use, disclose, and protect personal information when you use our website at crawlready.app ("Site"), our application programming interface ("API"), our middleware integrations, and related services.

By accessing or using our Services, you acknowledge that you have read, understood, and agree to be bound by this Privacy Policy. If you do not agree with this Privacy Policy, you must not use our Services.

2. Information We Collect

2.1 Information You Provide Directly

We collect information that you provide to us when you use our Services:

  • Account Information: When you create an account through Clerk (our authentication provider), we collect your email address, name, and optional organization details. You may also authenticate using third-party services (e.g., Google, GitHub), in which case we receive limited profile information from those services.
  • Email Capture for Diagnostic Reports: If you use our free AI readiness diagnostic tool, you may voluntarily provide your email address to receive detailed diagnostic reports. This is optional and you may use the tool without providing an email address.
  • Site Registration Information: When you register a website with our Services, we collect the URL, domain name, and sitemap location (if provided).
  • Payment Information: Payment processing is handled by Stripe, Inc. ("Stripe"). We do not store complete credit card numbers or sensitive payment credentials. Stripe collects billing name, address, and payment method details in accordance with its own privacy policy.
  • Communications: If you contact us via email, support channels, or social media, we collect the content of your communications, including any attachments you provide.
  • Integration Configuration: If you deploy our middleware snippet, CDN integration, or DNS proxy service, we collect configuration parameters such as site key, deployment method, and framework type.

2.2 Information Collected Automatically

When you use our Services, we automatically collect certain information:

  • Usage Data: We collect information about how you interact with our Services, including pages visited, features accessed, scan requests initiated, dashboard activity, and API calls made.
  • Device and Browser Information: We collect device type, operating system, browser type and version, screen resolution, and IP address for fraud detection and security purposes.
  • Log Data: Our servers automatically log standard HTTP request data, including timestamps, request methods, response codes, and referrer URLs.
  • Analytics Data: We use PostHog for product analytics to understand usage patterns, feature adoption, and user experience. PostHog collects anonymized event data including page views, button clicks, and feature interactions.

2.3 Information Collected from Your Website Visitors (AI Crawler Analytics)

If you deploy our AI Crawler Analytics feature via middleware integration or script tag, we collect limited information about AI crawler visits to your website:

  • AI Crawler Identification: User-Agent strings of verified AI crawlers (e.g., ChatGPT-User, OAI-SearchBot, ClaudeBot, PerplexityBot) and their IP addresses for verification purposes.
  • Request Metadata: Timestamp of crawler visit, URL path accessed, HTTP status code, and page response time.
  • Critical Distinction: We do not collect any information about human visitors to your website. Our analytics feature only tracks verified AI crawler requests. We do not collect, process, or store personally identifiable information (PII) of end users visiting your website. No cookies are set on end user browsers. No tracking occurs for human visitors.

2.4 Information Collected Through Content Processing

When you use our diagnostic, optimization, or content transformation services, we process the following information from your website:

  • Crawled Content: HTML content, page structure, DOM elements, JavaScript code, CSS stylesheets, and metadata from pages you submit for analysis or optimization.
  • Rendered Page Data: Screenshots, rendered DOM trees, and client-side rendered content obtained through headless browser rendering via our crawling service provider.
  • Technical Metrics: Page load time, JavaScript execution time, content availability metrics, Schema.org markup presence, robots.txt directives, and sitemap data.
  • Derived Scoring Data: AI readiness scores, crawlability assessments, agent readiness metrics, interaction scores, and detailed diagnostic findings.
  • Extracted Structured Data: Content patterns, FAQ structures, product information, and organizational data extracted for Schema.org generation purposes.

3. How We Use Your Information

3.1 Service Provision and Operation

We use your information to:

  • Provide, operate, maintain, and improve our Services;
  • Perform AI readiness scans and generate diagnostic reports;
  • Process and render JavaScript-heavy pages for AI crawler visibility;
  • Generate dynamic Schema.org markup from your website content;
  • Serve optimized content formats (Markdown, enriched HTML, structured JSON) to AI agents;
  • Monitor and track AI crawler visits to your registered websites;
  • Verify AI crawler authenticity through User-Agent and IP address validation;
  • Cache and deliver optimized content via our content delivery network;
  • Provide real-time analytics dashboards and reporting;
  • Enable synthetic bot verification during integration onboarding.

3.2 Account Management and Communications

We use your information to:

  • Create and manage your account;
  • Authenticate your identity and authorize access to Services;
  • Send transactional emails, including diagnostic reports, scan completion notifications, integration verification results, and alert notifications;
  • Provide customer support and respond to your inquiries;
  • Send service announcements, security alerts, and policy updates;
  • Communicate about new features, upgrades, and upsell opportunities (you may opt out of promotional communications).

3.3 Billing and Payment Processing

We use your information to:

  • Process subscription payments and manage billing cycles;
  • Track usage-based pricing metrics (fresh crawls, cached responses, API calls);
  • Issue invoices, receipts, and refunds;
  • Enforce usage limits and tier-specific restrictions;
  • Detect and prevent fraudulent transactions.

3.4 Analytics, Improvement, and Research

We use your information to:

  • Analyze usage patterns, feature adoption, and user behavior;
  • Conduct A/B testing and product experiments;
  • Develop new features and improve existing functionality;
  • Optimize scoring algorithms and recommendation engines;
  • Generate aggregated, anonymized industry benchmarks and statistics;
  • Improve AI crawler detection accuracy and bot verification methods.

3.5 Security, Compliance, and Legal Obligations

We use your information to:

  • Detect, investigate, and prevent fraudulent activity, abuse, and security incidents;
  • Enforce our Terms of Service and acceptable use policies;
  • Implement rate limiting and abuse prevention measures;
  • Comply with applicable laws, regulations, and legal processes;
  • Respond to lawful requests from public authorities, including national security and law enforcement;
  • Protect the rights, property, and safety of CrawlReady, our users, and third parties.

4. Legal Basis for Processing (GDPR)

If you are located in the European Economic Area (EEA), United Kingdom, or Switzerland, we process your personal data based on the following legal grounds:

  • Contract Performance: Processing is necessary to provide the Services you requested and perform our contractual obligations (GDPR Article 6(1)(b)).
  • Legitimate Interests: We process data to pursue our legitimate interests in operating, improving, and securing our Services, provided these interests are not overridden by your data protection rights (GDPR Article 6(1)(f)). Legitimate interests include fraud prevention, network security, product development, and customer support.
  • Consent: Where required by law, we obtain your explicit consent before processing personal data, such as for marketing communications or optional diagnostic email capture (GDPR Article 6(1)(a)). You may withdraw consent at any time.
  • Legal Obligations: Processing is necessary to comply with legal obligations, including tax reporting, financial record-keeping, and responses to lawful requests (GDPR Article 6(1)(c)).

5. Information Sharing and Disclosure

5.1 Third-Party Service Providers

We share information with trusted third-party vendors who perform services on our behalf under written agreements:

  • Clerk, Inc.: Authentication and user identity management. Clerk processes account credentials, email addresses, and authentication tokens in accordance with its privacy policy.
  • Supabase, Inc.: Database hosting (PostgreSQL) for account data, site registrations, scan results, crawler visit logs, and application state. Supabase operates data centers in the United States and Europe.
  • Crawling Service Provider (Firecrawl or equivalent): Web crawling and headless browser rendering services. We transmit URLs and receive rendered HTML, screenshots, and technical metrics. Crawling providers process content temporarily for rendering purposes only.
  • Vercel Inc.: Web hosting, serverless function execution, and content delivery. Vercel processes HTTP requests, API calls, and serves cached content.
  • Cloudflare, Inc.: Content delivery network (CDN), DDoS protection, and edge compute for DNS proxy and optimization features. Cloudflare processes HTTP requests and caches optimized content.
  • Upstash, Inc.: Redis-based rate limiting and API request throttling. Upstash processes IP addresses and request metadata for abuse prevention.
  • Stripe, Inc.: Payment processing and subscription billing. Stripe collects and processes payment information in accordance with its privacy policy and PCI DSS standards.
  • PostHog, Inc.: Product analytics and feature usage tracking. PostHog processes anonymized event data and session replays (if enabled).
  • Sentry, Inc.: Error tracking, performance monitoring, and crash reporting. Sentry collects stack traces, error messages, and anonymized request context.

These service providers are contractually obligated to use your information only to provide services to us and may not use it for their own purposes. We conduct vendor due diligence and require appropriate data protection safeguards.

5.2 Aggregated and Anonymized Data

We may share aggregated, anonymized, or de-identified data that cannot reasonably identify you or your website with third parties for industry research, benchmarking, marketing purposes, and public reporting. Examples include:

  • Average AI readiness scores by industry vertical;
  • Trends in AI crawler activity across geographic regions;
  • Adoption rates of Schema.org markup types;
  • Framework-specific JavaScript rendering performance benchmarks.

5.3 Business Transfers

If CrawlReady is involved in a merger, acquisition, reorganization, sale of assets, bankruptcy, or similar corporate transaction, your information may be transferred as part of that transaction. We will provide notice before your information is transferred and becomes subject to a different privacy policy.

5.4 Legal Requirements and Protection of Rights

We may disclose your information if required to do so by law or if we believe in good faith that such disclosure is necessary to:

  • Comply with legal obligations, court orders, subpoenas, or governmental requests;
  • Enforce our Terms of Service and other agreements;
  • Detect, prevent, or address fraud, security issues, or technical problems;
  • Protect the rights, property, or safety of CrawlReady, our users, or the public as required or permitted by law.

5.5 With Your Consent

We may share information with third parties when you provide explicit consent or direct us to do so, such as when you authorize integration with third-party platforms or share diagnostic reports publicly.

6. Data Retention

6.1 Account Data

We retain your account information for as long as your account is active or as necessary to provide Services. If you close your account, we will delete or anonymize your account data within 90 days, except where retention is required for legal, tax, audit, or security purposes.

6.2 Scan Results and Diagnostic Data

Scan results, AI readiness scores, and diagnostic reports are retained for the duration of your subscription plus 90 days. You may delete specific scan results at any time through your dashboard. Free-tier diagnostic results may be retained indefinitely for public shareable score URLs unless you request deletion.

6.3 Crawler Visit Logs

AI crawler analytics data (visit logs, bot activity) is retained for 90 days by default. You may configure shorter retention periods in your account settings. Aggregated, anonymized analytics may be retained indefinitely for product improvement purposes.

6.4 Cached Content

Cached optimized content and rendered pages are stored according to tier-specific time-to-live (TTL) policies ranging from 24 hours (Enterprise tier) to 14 days (Starter tier). You may configure custom TTL settings within tier-specific minimum and maximum limits. Content cache is automatically purged upon TTL expiration or manual invalidation.

6.5 Backup and Disaster Recovery

We maintain encrypted backups of data for disaster recovery purposes. Backup data is retained for 30 days and is used solely for system restoration in the event of data loss or catastrophic failure.

7. Data Security

We implement industry-standard technical and organizational security measures to protect your information from unauthorized access, disclosure, alteration, and destruction:

  • Encryption: All data in transit is encrypted using TLS 1.3 or higher. Data at rest in our database is encrypted using AES-256 encryption.
  • Access Controls: We implement role-based access controls (RBAC) and multi-factor authentication for internal systems. Access to personal data is restricted to authorized personnel on a need-to-know basis.
  • Multi-Tenancy Isolation: Customer data is logically isolated using Row-Level Security (RLS) in our database to prevent unauthorized cross-tenant access.
  • Infrastructure Security: Our infrastructure providers (Vercel, Supabase, Cloudflare) maintain SOC 2 Type II compliance and implement physical and network security controls.
  • Vulnerability Management: We conduct regular security assessments, dependency scanning, and patch management to address known vulnerabilities.
  • Incident Response: We maintain an incident response plan and will notify affected users of any data breach in accordance with applicable laws.

However, no method of electronic transmission or storage is 100% secure. While we strive to protect your information, we cannot guarantee absolute security. You are responsible for maintaining the confidentiality of your account credentials.

8. Your Privacy Rights

8.1 Access and Portability

You have the right to access your personal data and receive a copy in a structured, commonly used, machine-readable format (data portability). You can export your scan results, crawler analytics, and account data through your dashboard or by contacting us.

8.2 Correction and Update

You have the right to correct inaccurate or incomplete personal data. You can update your account information, site registrations, and preferences through your account settings at any time.

8.3 Deletion and Erasure

You have the right to request deletion of your personal data ("right to be forgotten"), subject to certain legal exceptions. You may delete your account, individual scan results, or specific websites through your dashboard. To request full account deletion, contact us at privacy@crawlready.app. We will process deletion requests within 30 days.

8.4 Restriction and Objection

You have the right to restrict or object to certain processing activities, including:

  • Objecting to marketing communications (opt-out links are provided in all promotional emails);
  • Restricting automated decision-making or profiling;
  • Objecting to processing based on legitimate interests.

8.5 Withdrawal of Consent

Where processing is based on consent, you have the right to withdraw consent at any time without affecting the lawfulness of processing before withdrawal. To withdraw consent for email diagnostic reports, unsubscribe via the link in any email or contact us.

8.6 Lodge a Complaint

If you are located in the EEA, UK, or Switzerland, you have the right to lodge a complaint with a supervisory authority if you believe our processing of your personal data violates applicable data protection laws. Contact information for EEA data protection authorities is available at https://edpb.europa.eu/about-edpb/board/members_en.

8.7 Exercising Your Rights

To exercise any of these rights, please contact us at privacy@crawlready.app with the subject line "Privacy Rights Request." We will respond to verified requests within 30 days (or as otherwise required by applicable law). We may request additional information to verify your identity before processing your request.

9. International Data Transfers

CrawlReady is based in France. Your information may be transferred to, stored, and processed in the United States, the European Union, or other countries where our service providers operate. These countries may have data protection laws different from those in your country of residence.

When we transfer personal data from the EEA, UK, or Switzerland to countries without an adequacy decision from the European Commission, we implement appropriate safeguards, including:

  • Standard Contractual Clauses (SCCs) approved by the European Commission;
  • Data Processing Agreements (DPAs) with sub-processors;
  • Technical measures such as encryption and pseudonymization.

For more information about our data transfer mechanisms, contact us at privacy@crawlready.app.

10. Cookies and Tracking Technologies

10.1 Essential Cookies

We use essential cookies necessary for authentication, security, and basic functionality. These cookies include:

  • Session cookies for maintaining logged-in state (Clerk authentication);
  • CSRF protection tokens;
  • Rate limiting and abuse prevention identifiers.

Essential cookies cannot be disabled without impacting core functionality.

10.2 Analytics Cookies

We use PostHog for product analytics, which sets cookies to track usage patterns and feature adoption. PostHog data is anonymized and does not identify individual users. You can opt out of analytics tracking through your browser settings or by disabling third-party cookies.

10.3 Your Cookie Choices

Most web browsers automatically accept cookies, but you can modify your browser settings to decline cookies. However, disabling cookies may limit your ability to use certain features of our Services. To learn more about cookies and how to manage them, visit www.allaboutcookies.org.

11. Do Not Track Signals

Some browsers support "Do Not Track" (DNT) signals. Currently, there is no industry standard for responding to DNT signals, and our Services do not respond to DNT browser settings. If a standard is established, we will reassess our approach to DNT compliance.

12. Children's Privacy

Our Services are not directed to individuals under the age of 16. We do not knowingly collect personal information from children under 16. If you are a parent or guardian and believe your child has provided us with personal information, please contact us at privacy@crawlready.app, and we will delete such information within 30 days.

13. California Privacy Rights (CCPA/CPRA)

If you are a California resident, the California Consumer Privacy Act (CCPA) and California Privacy Rights Act (CPRA) grant you specific rights regarding your personal information:

13.1 Categories of Personal Information Collected

In the preceding 12 months, we have collected the following categories of personal information:

  • Identifiers (email address, name, IP address, account ID);
  • Commercial information (subscription type, payment history, usage metrics);
  • Internet or network activity (browsing behavior, API requests, feature usage);
  • Professional or employment-related information (organization name, job title if provided).

13.2 Categories of Sources

We collect personal information directly from you, automatically through your use of Services, and from third-party authentication providers (Clerk).

13.3 Business or Commercial Purposes

We use personal information for the purposes described in Section 3 of this Privacy Policy, including service provision, analytics, security, and communications.

13.4 Categories of Third Parties

We share personal information with service providers (Clerk, Supabase, Stripe, Vercel, Cloudflare, PostHog, Sentry), payment processors, and crawling service providers as described in Section 5.

13.5 Sale or Sharing of Personal Information

We do not sell your personal information as defined under CCPA/CPRA. We do not share personal information for cross-context behavioral advertising.

13.6 Your CCPA/CPRA Rights

California residents have the right to:

  • Know: Request disclosure of categories and specific pieces of personal information collected, sources, purposes, and third parties with whom information is shared.
  • Delete: Request deletion of personal information, subject to certain exceptions.
  • Correct: Request correction of inaccurate personal information.
  • Opt-Out: Opt out of the sale or sharing of personal information (not applicable as we do not sell or share).
  • Limit Use of Sensitive Personal Information: Request limitation of use of sensitive personal information (we do not collect sensitive personal information as defined under CPRA).
  • Non-Discrimination: You have the right not to receive discriminatory treatment for exercising your CCPA/CPRA rights.

To exercise these rights, submit a verifiable consumer request to privacy@crawlready.app. We will verify your identity and respond within 45 days.

13.7 Authorized Agent

You may designate an authorized agent to submit requests on your behalf. The agent must provide written authorization signed by you, and we may require you to verify your identity directly with us.

14. Nevada Privacy Rights

Nevada residents have the right to opt out of the sale of covered personal information. We do not sell personal information as defined under Nevada law (NRS 603A). If you are a Nevada resident and have questions, contact us at privacy@crawlready.app.

15. Changes to This Privacy Policy

We may update this Privacy Policy from time to time to reflect changes in our practices, technology, legal requirements, or other factors. We will notify you of material changes by:

  • Posting the updated Privacy Policy on this page with a revised "Last Updated" date;
  • Sending an email notification to the address associated with your account (if you have an account);
  • Displaying a prominent notice on our Site or within our Services.

Your continued use of our Services after the effective date of the updated Privacy Policy constitutes your acceptance of the changes. If you do not agree to the updated Privacy Policy, you must stop using our Services and may close your account.

16. Contact Us

If you have questions, concerns, or requests regarding this Privacy Policy or our data practices, please contact us:

We will respond to inquiries within 30 days (or as otherwise required by applicable law). For urgent security or privacy concerns, please use the subject line "URGENT: Privacy Matter."

This Privacy Policy was last updated on January 13, 2026. Previous versions are available upon request.