For AI agents: a documentation index is available at /llms.txt
Skip to main content

/chromium/scrape

POST 

/chromium/scrape

Returns text, HTML, and metadata from a given list of CSS selectors against the rendered DOM. Debugging information is available by sending in the appropriate flags in the "debugOpts" property. Responds with an array of JSON objects.

Note: Also available at /scrape as a convenience alias.

Request​

Query Parameters

    blockAds boolean
    blockAdsInclude string
    launch object
    profile string
    timeout number
    token string
    trackingId string
    allowedDomains string[]
    credentials object
    externalProxyServer string
    integrationId string
    proxy ProxyType

    Possible values: [datacenter, residential]

    proxyCity string
    proxyCountry string
    proxyLocaleMatch string

    Possible values: [0, 1, false, true]

    proxyPreset string
    proxyState string
    proxySticky string

    Possible values: [0, 1, false, true]

Body

    addScriptTag object[]
  • Array [
  • urlstring

    URL of the script to be added.

    contentstring

    JavaScript to be injected into the frame.

    typestring

    Sets the type of the script. Use module in order to load an ES2015 module.

    idstring

    Sets the id of the script.

  • ]
  • addStyleTag object[]
  • Array [
  • urlstring

    the URL of the CSS file to be added.

    contentstring

    Raw CSS content to be injected into the frame.

  • ]
  • authenticate object

    Credentials for HTTP authentication. Contains username and password properties.

    anyOf
    usernamestringrequired
    passwordstringrequired
    bestAttemptboolean

    When bestAttempt is set to true, browserless attempt to proceed when "awaited" events fail or timeout. This includes things like goto, waitForSelector, and more.

    cookies object[]
  • Array [
  • namestringrequired

    Cookie name.

    valuestringrequired

    Cookie value.

    urlstring

    The request-URI to associate with the setting of the cookie. This value can affect the default domain, path, and source scheme values of the created cookie.

    domainstring

    Cookie domain.

    pathstring

    Cookie path.

    secureboolean

    True if cookie is secure.

    httpOnlyboolean

    True if cookie is http-only.

    sameSitestring

    Cookie SameSite type.

    Possible values: [Default, Lax, None, Strict]

    expiresnumber

    Cookie expiration date, session cookie if not set

    prioritystring

    Cookie Priority. Supported only in Chrome.

    Possible values: [High, Low, Medium]

    sourceSchemestring

    Cookie source scheme type. Supported only in Chrome.

    Possible values: [NonSecure, Secure, Unset]

    partitionKey object

    Cookie partition key. In Chrome, it matches the top-level site the partitioned cookie is available in. In Firefox, it matches the source origin in the {@link https://w3c.github.io/webdriver-bidi/#type-storage-PartitionKey PartitionKey}.

    anyOf

    Represents a cookie partition key in Chrome.

    sourceOriginstringrequired

    The site of the top-level URL the browser was visiting at the start of the request to the endpoint that set the cookie.

    In Chrome, maps to the CDP's topLevelSite partition key.

    hasCrossSiteAncestorboolean

    Indicates if the cookie has any ancestors that are cross-site to the topLevelSite.

    Supported only in Chrome.

  • ]
  • debugOpts object
    consoleboolean
    cookiesboolean
    htmlboolean
    networkboolean
    screenshotboolean
    elements object[]required
  • Array [
  • selectorstringrequired
    timeoutnumber
  • ]
  • emulateMediaTypestring
    gotoOptions object

    Options to configure the page navigation, such as timeout and waitUntil.

    refererstring

    If provided, it will take preference over the referer header value set by {@link Page.setExtraHTTPHeaderspage.setExtraHTTPHeaders()}.

    referrerPolicystring

    If provided, it will take preference over the referer-policy header value set by {@link Page.setExtraHTTPHeaderspage.setExtraHTTPHeaders()}.

    timeoutnumber

    Maximum wait time in milliseconds. Pass 0 to disable the timeout.

    The default value can be changed by using the {@link Page.setDefaultTimeout} or {@link Page.setDefaultNavigationTimeout} methods.

    waitUntil object

    When to consider waiting succeeds. Given an array of event strings, waiting is considered to be successful after all events have been fired.

    anyOf
  • Array [
  • PuppeteerLifeCycleEvent

    Possible values: [domcontentloaded, load, networkidle0, networkidle2]

  • ]
  • signal object

    A signal object that allows you to cancel the call.

    abortedbooleanrequired
    onabort objectrequired
    anyOf
    object
    reasonrequired
    htmlstring
    rejectRequestPatternstring[]
    rejectResourceTypesstring[]

    Possible values: [cspviolationreport, document, eventsource, fedcm, fetch, font, image, manifest, media, other, ping, prefetch, preflight, script, signedexchange, stylesheet, texttrack, websocket, xhr]

    requestInterceptors object[]
  • Array [
  • patternstringrequired

    An array of patterns (using req.url().match(r.pattern) to match) and their corresponding responses to use in order to fulfill those requests.

    response objectrequired
    headers object

    Optional response headers. Either a {@link https://developer.mozilla.org/en-US/docs/Web/API/Headers Headers} instance or a plain record.

    The record values will be converted to string following: Arrays' values will be mapped to String (Used when you need multiple headers with the same name). Non-arrays will be converted to String.

    anyOf
    object
    statusnumber
    contentTypestring
    bodystring

    A string representation of the body to return. Can be a base64-encoded string but please omit any leading content-type data (eg "data:image/png;base64,").

  • ]
  • setExtraHTTPHeaders object
    property name*string
    setJavaScriptEnabledboolean
    urlstring
    userAgent object
    userAgentstring
    userAgentMetadata object

    Used to specify User Agent Client Hints to emulate. See https://wicg.github.io/ua-client-hints Missing optional values will be filled in by the target with what it would normally use.

    brands object[]

    Brands appearing in Sec-CH-UA.

  • Array [
  • brandstringrequired
    versionstringrequired
  • ]
  • fullVersionList object[]

    Brands appearing in Sec-CH-UA-Full-Version-List.

  • Array [
  • brandstringrequired
    versionstringrequired
  • ]
  • fullVersionstring
    platformstringrequired
    platformVersionstringrequired
    architecturestringrequired
    modelstringrequired
    mobilebooleanrequired
    bitnessstring
    wow64boolean
    formFactorsstring[]

    Used to specify User Agent form-factor values. See https://wicg.github.io/ua-client-hints/#sec-ch-ua-form-factors

    platformstring
    viewport object

    The viewport dimensions and settings for the page. Includes properties like width, height, deviceScaleFactor, etc.

    anyOf
    widthnumberrequired

    The page width in CSS pixels.

    heightnumberrequired

    The page height in CSS pixels.

    deviceScaleFactornumber

    Specify device scale factor. See {@link https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio devicePixelRatio} for more info.

    isMobileboolean

    Whether the meta viewport tag is taken into account.

    isLandscapeboolean

    Specifies if the viewport is in landscape mode.

    hasTouchboolean

    Specify if the viewport supports touch events.

    waitForEvent object
    eventstringrequired
    timeoutnumber
    waitForFunction object
    fnstringrequired

    The function, or statement, to be evaluated in browser context

    pollingstring | number

    An interval at which the pageFunction is executed, defaults to raf. If polling is a number, then it is treated as an interval in milliseconds at which the function would be executed. If polling is a string, then it can be one of the following values: "raf" or "mutation"

    timeoutnumber

    Maximum time to wait for in milliseconds. Defaults to 30000 (30 seconds). Pass 0 to disable timeout.

    waitForSelector object
    hiddenboolean
    selectorstringrequired
    timeoutnumber
    visibleboolean
    waitForTimeoutnumber

Responses​

The JSON response body

Schema
    data objectrequired
    anyOf
  • Array [
  • results object[]required
  • Array [
  • attributes object[]required

    A list of HTML attributes of the element

  • Array [
  • namestringrequired

    The name of the HTML attribute for the element

    valuestringrequired

    The value of the HTML attribute for the element

  • ]
  • heightnumberrequired

    The height the element

    htmlstringrequired

    The HTML the element

    leftnumberrequired

    The amount of pixels from the left of the page

    textstringrequired

    The text the element

    topnumberrequired

    The amount of pixels from the top of the page

    widthnumberrequired

    The width the element

  • ]
  • selectorstringrequired

    The DOM selector of the element

  • ]
  • debug objectrequired

    When debugOpts options are present, results are here

    anyOf
    consolestring[]required

    A list of console messages from the browser

    cookies objectrequired

    List of cookies for the site or null

    anyOf
  • Array [
  • pathstringrequired

    Cookie path.

    expiresnumberrequired

    Cookie expiration date as the number of seconds since the UNIX epoch. Set to -1 for session cookies

    sizenumberrequired

    Cookie size.

    securebooleanrequired

    True if cookie is secure.

    sessionbooleanrequired

    True in case of session cookie.

    partitionKeyOpaqueboolean

    True if cookie partition key is opaque. Supported only in Chrome.

    namestringrequired

    Cookie name.

    valuestringrequired

    Cookie value.

    domainstringrequired

    Cookie domain.

    httpOnlyboolean

    True if cookie is http-only.

    sameSitestring

    Cookie SameSite type.

    Possible values: [Default, Lax, None, Strict]

    prioritystring

    Cookie Priority. Supported only in Chrome.

    Possible values: [High, Low, Medium]

    sourceSchemestring

    Cookie source scheme type. Supported only in Chrome.

    Possible values: [NonSecure, Secure, Unset]

    partitionKey object

    Cookie partition key. In Chrome, it matches the top-level site the partitioned cookie is available in. In Firefox, it matches the source origin in the {@link https://w3c.github.io/webdriver-bidi/#type-storage-PartitionKey PartitionKey}.

    anyOf

    Represents a cookie partition key in Chrome.

    sourceOriginstringrequired

    The site of the top-level URL the browser was visiting at the start of the request to the endpoint that set the cookie.

    In Chrome, maps to the CDP's topLevelSite partition key.

    hasCrossSiteAncestorboolean

    Indicates if the cookie has any ancestors that are cross-site to the topLevelSite.

    Supported only in Chrome.

  • ]
  • htmlnull | stringnullablerequired

    The HTML string of the website or null

    network objectrequired
    inbound object[]required
  • Array [
  • headersrequired
    statusnumberrequired
    urlstringrequired
  • ]
  • outbound object[]required
  • Array [
  • headersrequired
    methodstringrequired
    urlstringrequired
  • ]
  • screenshotnull | stringnullablerequired

    A base64-encoded string of the site or null

Was this page helpful?