Export

Export raw crawler visit rows for a site as CSV or JSON.

Endpoint

GET /api/v1/analytics/{siteId}/export

Authentication: Bearer token (session or API key).

Query parameters

| Parameter | Type | Default | Description | |-----------|------|---------|-------------| | format | csv | json | json | Output format | | days | integer | 30 | Look-back window (rows with visited_at in the last N days) |

Rate limit

1 export per 5 minutes per user. Exceeding this returns 429 Too Many Requests.

Row limit

10,000 rows maximum per export. Rows are ordered oldest-first by visited_at. If your site has more than 10,000 visits in the selected window, only the oldest 10,000 are returned. Reduce the days parameter to narrow the window.

CSV export

curl -H "Authorization: Bearer $API_KEY" \
  "https://crawlready.app/api/v1/analytics/$SITE_ID/export?format=csv&days=30" \
  -o crawlready-export.csv

The response is Content-Type: text/csv with a Content-Disposition header that sets a filename. The file includes a header row:

path,bot,visited_at,source
/blog/ai-readiness,GPTBot,2026-05-17T14:32:00.000Z,middleware
/,ClaudeBot,2026-05-17T14:31:42.000Z,middleware

Fields that contain commas, double-quotes, or newlines are wrapped in double-quotes with internal quotes escaped as "".

JSON export

curl -H "Authorization: Bearer $API_KEY" \
  "https://crawlready.app/api/v1/analytics/$SITE_ID/export?format=json&days=7"

Response:

{
  "rows": [
    {
      "path": "/blog/ai-readiness",
      "bot": "GPTBot",
      "visited_at": "2026-05-17T14:32:00.000Z",
      "source": "middleware"
    }
  ]
}

Fields

| Field | Type | Description | |-------|------|-------------| | path | string | URL path requested by the crawler | | bot | string | Crawler name as identified from the User-Agent | | visited_at | ISO 8601 | Timestamp of the visit (UTC) | | source | string | How the visit was captured: middleware or script |