For AI agents: a documentation index is available at /llms.txt
Skip to main content

/batch-scrape

POST 

/batch-scrape

Start an async job over a list of URLs; poll GET /batch-scrape/{id}; optional webhooks.

Request​

Query Parameters

    profile string
    token string
    allowedDomains string[]
    credentials object
    externalProxyServer string
    integrationId string
    proxy ProxyType

    Possible values: [datacenter, residential]

    proxyCity string
    proxyCountry string
    proxyLocaleMatch string

    Possible values: [0, 1, false, true]

    proxyPreset string
    proxyState string
    proxySticky string

    Possible values: [0, 1, false, true]

Body

    urlsstring[]required

    Public HTTP(S) URLs to scrape. Limited by your plan before deduplication.

    scrapeOptions

    Rejected by the handler with guidance to use top-level scrape options.

    ignoreInvalidURLsboolean

    Skip and report invalid URLs.

    Default value: true
    formatsstring[]

    Output formats.

    Possible values: [html, markdown, rawText]

    Default value: ["markdown"]
    onlyMainContentboolean

    Extract only main content.

    Default value: true
    includeTagsstring[]

    HTML selectors to include.

    Default value: []
    excludeTagsstring[]

    HTML selectors to exclude.

    Default value: []
    waitFornumber

    Wait after load in milliseconds.

    Default value: 0
    headers object

    Customer HTTP headers for page requests.

    property name*string
    timeoutnumber

    Navigation timeout in milliseconds.

    Default value: 150000
    proxystring

    Proxy network.

    Possible values: [datacenter, residential]

    Default value: residential
    webhook object
    urlstringrequired

    Public HTTPS receiver.

    eventsstring[]

    Events to deliver.

    Possible values: [completed, failed, page]

    Default value: ["completed"]

Responses​

Schema
    successbooleanrequired
    idstringrequired
    urlstringrequired
    invalidURLsstring[]required
Was this page helpful?