/batch-scrape
POST/batch-scrape
Start an async job over a list of URLs; poll GET /batch-scrape/{id}; optional webhooks.
Request
Query Parameters
Possible values: [datacenter, residential]
Possible values: [0, 1, false, true]
Possible values: [0, 1, false, true]
Body
- Schema
- Example (auto)
Public HTTP(S) URLs to scrape. Limited by your plan before deduplication.
Rejected by the handler with guidance to use top-level scrape options.
Skip and report invalid URLs.
trueOutput formats.
Possible values: [html, markdown, rawText]
["markdown"]Extract only main content.
trueHTML selectors to include.
[]HTML selectors to exclude.
[]Wait after load in milliseconds.
0headers object
Customer HTTP headers for page requests.
Navigation timeout in milliseconds.
150000Proxy network.
Possible values: [datacenter, residential]
residentialwebhook object
Public HTTPS receiver.
Events to deliver.
Possible values: [completed, failed, page]
["completed"]{
"urls": [
"string"
],
"ignoreInvalidURLs": true,
"formats": [
"html"
],
"onlyMainContent": true,
"includeTags": [
"string"
],
"excludeTags": [
"string"
],
"waitFor": 0,
"headers": {},
"timeout": 150000,
"proxy": "residential",
"webhook": {
"url": "string",
"events": [
"completed"
]
}
}
Responses
- 200
- 400
- 401
- 403
- 404
- 408
- 429
- 500
- 503
Schema
- Fields
- Example (auto)
{
"success": true,
"id": "string",
"url": "string",
"invalidURLs": [
"string"
]
}
The request contains errors or didn't properly encode content.
The request is missing, or contains bad, authorization credentials.
The request targets a destination that is not allowed.
Resource couldn't be found.
The request took has taken too long to process.
Too many requests are currently being processed.
An internal error occurred when handling the request.
Service is unavailable.