API reference

Batches

Scrape thousands of URLs in one background job.

POSThttps://api.scrapeflow.dev/v1/batches1 credit per URL

Submit a large list of URLs and one set of formats. Runs through the SNS → SQS fan-out and returns a single dataset when complete. Poll or use a webhook.

Request

curl -X POST ${SCRAPEFLOW_API_URL}/v1/batches \
  -H "Authorization: Bearer $SCRAPEFLOW_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{ "urls": ["https://a.com", "https://b.com"], "formats": { "markdown": true } }'

Parameters

Parameter	Type	Description
urlsrequired	string[]	Up to 25,000 URLs.
formats	object	Formats applied to every URL.
webhook	string	URL to notify on completion.

Response

json

{ "success": true, "batch_id": "batch_9f2a", "status": "running", "total": 25000 }

← Brand Monitors →