Export gives you raw crawler visit data for analysis in spreadsheets, BI tools, or custom scripts. Each row represents a single crawler visit with the path, bot name, timestamp, and data source.
Formats
Two formats are available:
- CSV — for spreadsheets and data tools. Includes a header row (
path,bot,visited_at,source). Fields containing commas, quotes, or newlines are properly escaped. - JSON — for programmatic use. Returns an array of visit objects with the same four fields.
Constraints
- 10,000 rows maximum per export. Rows are ordered oldest-first. If your site has more visits than the limit in the selected window, reduce the date range to narrow the result set.
- 1 export per 5 minutes per user. Exceeding this returns a rate limit error.
- Up to 90 days of data available.
Example
curl -H "Authorization: Bearer $API_KEY" \
"https://crawlready.app/api/v1/analytics/$SITE_ID/export?format=csv&days=30" \
-o crawlready-export.csv
API access
See the Export endpoint in the API Reference for the full specification.