For AI agents: a documentation index is available at /llms.txt
Skip to main content

/scrapers

GET 

/scrapers

Lists the scrapers and APIs available from the loaded scraper bundle, with the bundle version and each API's output schema. Call POST /scrapers/{scraper}/{api} to run one.

Request​

Query Parameters

    token string
    allowedDomains string[]
    credentials object
    externalProxyServer string
    integrationId string
    proxy ProxyType

    Possible values: [datacenter, residential]

    proxyCity string
    proxyCountry string
    proxyLocaleMatch string

    Possible values: [0, 1, false, true]

    proxyPreset string
    proxyState string
    proxySticky string

    Possible values: [0, 1, false, true]

Responses​

Schema
    versionstringrequired

    Bundle version (semver only). The build's git sha is never part of a response; operators find it in the logs and the info metric.

    contractVersionnumberrequired

    Contract version the loaded bundle was built against: one of the versions this host supports, not necessarily its own CONTRACT_VERSION.

    previousVersionstringrequired

    Version that was live before the last hot swap (semver only); null until the first swap on this instance.

    swap objectrequired

    Hot-swap state: whether a replacement bundle is being verified or handshaked right now, which version it is, and how the last swap ended.

    inProgressbooleanrequired
    targetVersionstringrequired
    startedAtstringrequired
    lastOutcomestringrequired
    lastAtstringrequired
    scrapers objectrequired

    Catalog keyed by scraper name (an organisation slug such as zillow, never a hostname). Each scraper lists the DNS hosts it handles and its APIs; each API declares its result-cache TTL and the JSON Schema its data is validated against on every result.

    property name* object
    hostnamesstring[]required

    Every DNS host this scraper handles — one organisation may span several.

    apis objectrequired
    property name* object
    ttlSecondsnumberrequired

    Seconds a result for this API stays fresh in the result cache.

    unitsnumberrequired

    Units one call costs. Charged when the call returns HTTP 200 with ok: true or error code not-found; every other outcome is free.

    unitVariants object

    Other prices a call may cost, by name, when the call needs more work (e.g. js: the page rendered in a browser). Absent when the API has a single price.

    property name*number
    outputSchema objectrequired

    JSON Schema (draft 2020-12) the API's data is validated against. The host enforces it on every result.

    property name*any

    JSON Schema (draft 2020-12) the API's data is validated against. The host enforces it on every result.

Was this page helpful?