/edge/scrape
POST/edge/scrape
Returns text, HTML, and metadata from a given list of CSS selectors against the rendered DOM (Edge). Debugging information is available by sending in the appropriate flags in the "debugOpts" property. Responds with an array of JSON objects.
Request
Query Parameters
Possible values: [datacenter, residential]
Possible values: [0, 1, false, true]
Possible values: [0, 1, false, true]
Body
- Schema
- JSON
- Array [
- ]
- Array [
- ]
- Credentials
- null
- Array [
- CookiePartitionKey
- string
- ]
- Array [
- ]
- PuppeteerLifeCycleEvent[]
- string
- Array [
- ]
- object
- null
- Array [
- Headers
- AnyRecord
- ]
- Array [
- ]
- Array [
- ]
- Viewport
- null
addScriptTag object[]
URL of the script to be added.
JavaScript to be injected into the frame.
Sets the type of the script. Use module in order to load an ES2015 module.
Sets the id of the script.
addStyleTag object[]
the URL of the CSS file to be added.
Raw CSS content to be injected into the frame.
authenticate object
Credentials for HTTP authentication. Contains username and password properties.
When bestAttempt is set to true, browserless attempt to proceed when "awaited" events fail or timeout. This includes things like goto, waitForSelector, and more.
cookies object[]
Cookie name.
Cookie value.
The request-URI to associate with the setting of the cookie. This value can affect the default domain, path, and source scheme values of the created cookie.
Cookie domain.
Cookie path.
True if cookie is secure.
True if cookie is http-only.
Cookie SameSite type.
Possible values: [Default, Lax, None, Strict]
Cookie expiration date, session cookie if not set
Cookie Priority. Supported only in Chrome.
Possible values: [High, Low, Medium]
Cookie source scheme type. Supported only in Chrome.
Possible values: [NonSecure, Secure, Unset]
partitionKey object
Cookie partition key. In Chrome, it matches the top-level site the partitioned cookie is available in. In Firefox, it matches the source origin in the {@link https://w3c.github.io/webdriver-bidi/#type-storage-PartitionKey PartitionKey}.
Represents a cookie partition key in Chrome.
The site of the top-level URL the browser was visiting at the start of the request to the endpoint that set the cookie.
In Chrome, maps to the CDP's topLevelSite partition key.
Indicates if the cookie has any ancestors that are cross-site to the topLevelSite.
Supported only in Chrome.
debugOpts object
elements object[]required
gotoOptions object
Options to configure the page navigation, such as timeout and waitUntil.
If provided, it will take preference over the referer header value set by {@link Page.setExtraHTTPHeaderspage.setExtraHTTPHeaders()}.
If provided, it will take preference over the referer-policy header value set by {@link Page.setExtraHTTPHeaderspage.setExtraHTTPHeaders()}.
Maximum wait time in milliseconds. Pass 0 to disable the timeout.
The default value can be changed by using the {@link Page.setDefaultTimeout} or {@link Page.setDefaultNavigationTimeout} methods.
waitUntil object
When to consider waiting succeeds. Given an array of event strings, waiting is considered to be successful after all events have been fired.
Possible values: [domcontentloaded, load, networkidle0, networkidle2]
signal object
A signal object that allows you to cancel the call.
onabort objectrequired
Possible values: [cspviolationreport, document, eventsource, fedcm, fetch, font, image, manifest, media, other, ping, prefetch, preflight, script, signedexchange, stylesheet, texttrack, websocket, xhr]
requestInterceptors object[]
An array of patterns (using req.url().match(r.pattern) to match) and their
corresponding responses to use in order to fulfill those requests.
response objectrequired
headers object
Optional response headers. Either a {@link https://developer.mozilla.org/en-US/docs/Web/API/Headers Headers} instance or a plain record.
The record values will be converted to string following: Arrays' values will be mapped to String (Used when you need multiple headers with the same name). Non-arrays will be converted to String.
A string representation of the body to return. Can be a base64-encoded string but please omit any leading content-type data (eg "data:image/png;base64,").
setExtraHTTPHeaders object
userAgent object
userAgentMetadata object
Used to specify User Agent Client Hints to emulate. See https://wicg.github.io/ua-client-hints Missing optional values will be filled in by the target with what it would normally use.
brands object[]
Brands appearing in Sec-CH-UA.
fullVersionList object[]
Brands appearing in Sec-CH-UA-Full-Version-List.
Used to specify User Agent form-factor values. See https://wicg.github.io/ua-client-hints/#sec-ch-ua-form-factors
viewport object
The viewport dimensions and settings for the page.
Includes properties like width, height, deviceScaleFactor, etc.
The page width in CSS pixels.
The page height in CSS pixels.
Specify device scale factor. See {@link https://developer.mozilla.org/en-US/docs/Web/API/Window/devicePixelRatio devicePixelRatio} for more info.
Whether the meta viewport tag is taken into account.
Specifies if the viewport is in landscape mode.
Specify if the viewport supports touch events.
waitForEvent object
waitForFunction object
The function, or statement, to be evaluated in browser context
An interval at which the pageFunction is executed, defaults to raf. If polling is a number, then it is treated as an interval in milliseconds at which the function would be executed. If polling is a string, then it can be one of the following values: "raf" or "mutation"
Maximum time to wait for in milliseconds. Defaults to 30000 (30 seconds). Pass 0 to disable timeout.
waitForSelector object
{
"addScriptTag": [
{
"url": "string",
"content": "string",
"type": "string",
"id": "string"
}
],
"addStyleTag": [
{
"url": "string",
"content": "string"
}
],
"authenticate": {
"username": "string",
"password": "string"
},
"bestAttempt": true,
"cookies": [
{
"name": "string",
"value": "string",
"url": "string",
"domain": "string",
"path": "string",
"secure": true,
"httpOnly": true,
"sameSite": "Default",
"expires": 0,
"priority": "High",
"sourceScheme": "NonSecure",
"partitionKey": {
"sourceOrigin": "string",
"hasCrossSiteAncestor": true
}
}
],
"debugOpts": {
"console": true,
"cookies": true,
"html": true,
"network": true,
"screenshot": true
},
"elements": [
{
"selector": "string",
"timeout": 0
}
],
"emulateMediaType": "string",
"gotoOptions": {
"referer": "string",
"referrerPolicy": "string",
"timeout": 0,
"waitUntil": [
"domcontentloaded"
],
"signal": {
"aborted": true,
"onabort": {},
"reason": {}
}
},
"html": "string",
"rejectRequestPattern": [
"string"
],
"rejectResourceTypes": [
"cspviolationreport"
],
"requestInterceptors": [
{
"pattern": "string",
"response": {
"headers": {},
"status": 0,
"contentType": "string",
"body": "string"
}
}
],
"setExtraHTTPHeaders": {},
"setJavaScriptEnabled": true,
"url": "string",
"userAgent": {
"userAgent": "string",
"userAgentMetadata": {
"brands": [
{
"brand": "string",
"version": "string"
}
],
"fullVersionList": [
{
"brand": "string",
"version": "string"
}
],
"fullVersion": "string",
"platform": "string",
"platformVersion": "string",
"architecture": "string",
"model": "string",
"mobile": true,
"bitness": "string",
"wow64": true,
"formFactors": [
"string"
]
},
"platform": "string"
},
"viewport": {
"width": 0,
"height": 0,
"deviceScaleFactor": 0,
"isMobile": true,
"isLandscape": true,
"hasTouch": true
},
"waitForEvent": {
"event": "string",
"timeout": 0
},
"waitForFunction": {
"fn": "string",
"polling": "string",
"timeout": 0
},
"waitForSelector": {
"hidden": true,
"selector": "string",
"timeout": 0,
"visible": true
},
"waitForTimeout": 0
}
Responses
- 200
- 400
- 401
- 403
- 404
- 408
- 429
- 500
- 503
The JSON response body
Schema
- Fields
- Example (auto)
- object[]
- null
- Array [
- Array [
- Array [
- ]
- ]
- ]
- object
- null
- object[]
- null
- Array [
- CookiePartitionKey
- string
- ]
- Array [
- ]
- Array [
- ]
data objectrequired
results object[]required
attributes object[]required
A list of HTML attributes of the element
The name of the HTML attribute for the element
The value of the HTML attribute for the element
The height the element
The HTML the element
The amount of pixels from the left of the page
The text the element
The amount of pixels from the top of the page
The width the element
The DOM selector of the element
debug objectrequired
When debugOpts options are present, results are here
A list of console messages from the browser
cookies objectrequired
List of cookies for the site or null
Cookie path.
Cookie expiration date as the number of seconds since the UNIX epoch. Set to -1 for
session cookies
Cookie size.
True if cookie is secure.
True in case of session cookie.
True if cookie partition key is opaque. Supported only in Chrome.
Cookie name.
Cookie value.
Cookie domain.
True if cookie is http-only.
Cookie SameSite type.
Possible values: [Default, Lax, None, Strict]
Cookie Priority. Supported only in Chrome.
Possible values: [High, Low, Medium]
Cookie source scheme type. Supported only in Chrome.
Possible values: [NonSecure, Secure, Unset]
partitionKey object
Cookie partition key. In Chrome, it matches the top-level site the partitioned cookie is available in. In Firefox, it matches the source origin in the {@link https://w3c.github.io/webdriver-bidi/#type-storage-PartitionKey PartitionKey}.
Represents a cookie partition key in Chrome.
The site of the top-level URL the browser was visiting at the start of the request to the endpoint that set the cookie.
In Chrome, maps to the CDP's topLevelSite partition key.
Indicates if the cookie has any ancestors that are cross-site to the topLevelSite.
Supported only in Chrome.
The HTML string of the website or null
network objectrequired
inbound object[]required
outbound object[]required
A base64-encoded string of the site or null
{
"data": [
{
"results": [
{
"attributes": [
{
"name": "string",
"value": "string"
}
],
"height": 0,
"html": "string",
"left": 0,
"text": "string",
"top": 0,
"width": 0
}
],
"selector": "string"
}
],
"debug": {
"console": [
"string"
],
"cookies": [
{
"path": "string",
"expires": 0,
"size": 0,
"secure": true,
"session": true,
"partitionKeyOpaque": true,
"name": "string",
"value": "string",
"domain": "string",
"httpOnly": true,
"sameSite": "Default",
"priority": "High",
"sourceScheme": "NonSecure",
"partitionKey": {
"sourceOrigin": "string",
"hasCrossSiteAncestor": true
}
}
],
"html": "null",
"network": {
"inbound": [
{
"headers": {},
"status": 0,
"url": "string"
}
],
"outbound": [
{
"headers": {},
"method": "string",
"url": "string"
}
]
},
"screenshot": "null"
}
}
The request contains errors or didn't properly encode content.
The request is missing, or contains bad, authorization credentials.
The request targets a destination that is not allowed.
Resource couldn't be found.
The request took has taken too long to process.
Too many requests are currently being processed.
An internal error occurred when handling the request.
Service is unavailable.