For AI agents: a documentation index is available at /llms.txt
Skip to main content

Running Puppeteer and Playwright on AWS Lambda

Run Puppeteer or Playwright from an AWS Lambda function by connecting to a Browserless browser instead of bundling Chromium in the function. This guide covers the handler code for Node.js and Python, packaging and deployment, Lambda timeout and memory settings, region choice, and how to handle bot-protected sites.

Prerequisites

Why not run Chromium inside Lambda​

Running a browser inside the function itself means fighting the platform:

  • Package size. A Lambda .zip deployment package, including layers, can't exceed 250 MB unzipped. A Chromium build takes a large share of that before your own code and dependencies, which is why many teams end up on container images.
  • Cold starts. Each new execution environment has to load the larger package and launch Chromium before your code does any work.
  • Operating system compatibility. Current Lambda runtimes run on Amazon Linux 2023, which doesn't ship the shared libraries a desktop Chromium build expects. You need a Lambda-specific Chromium build and have to keep it in step with your Puppeteer or Playwright version.
  • Blocking. Requests leave from AWS IP ranges, which AWS publishes and many bot-detection systems treat as datacenter traffic.

With Browserless, the browser runs on our infrastructure and your function only ships the client library. puppeteer-core and playwright-core don't download a browser, so the deployment package stays small, there's no Chromium to launch on a cold start, and you get stealth routes, residential proxies, and CAPTCHA solving through the connection URL.

If you don't need page-level control at all, skip the browser library and call a REST API from Lambda instead.

Write the handler​

Each handler reads the API token and endpoint from environment variables, connects to Browserless, loads the URL from the event, and returns the page title. The browser is closed in a finally block so the Browserless session ends even when navigation throws.

The connection URL also sets the Browserless timeout query parameter to 60 seconds. That caps the session length on our side, which matters on Lambda: if the function hits its own timeout, Lambda stops your code and the finally block never runs. The timeout parameter makes sure the session still ends.

Save this as index.mjs:

import puppeteer from "puppeteer-core";

// Build the endpoint from env vars so you can switch region, stealth route, or proxy without a redeploy.
const endpoint = new URL(process.env.BROWSERLESS_URL ?? "wss://production-sfo.browserless.io");
endpoint.searchParams.set("token", process.env.BROWSERLESS_TOKEN);
// Keep this at or below the Lambda timeout so a timed-out invocation can't leave a session running.
endpoint.searchParams.set("timeout", "60000");

export const handler = async (event) => {
const url = event.url ?? "https://example.com/";

// Connect per invocation. A connection held across invocations stays open while Lambda freezes the environment.
const browser = await puppeteer.connect({ browserWSEndpoint: endpoint.toString() });

try {
const page = await browser.newPage();
await page.goto(url, { waitUntil: "domcontentloaded", timeout: 30000 });
return { url, title: await page.title() };
} finally {
// Ends the Browserless session even when navigation fails.
await browser.close();
}
};

Response

{
"url": "https://example.com/",
"title": "Example Domain"
}

BROWSERLESS_URL holds everything except the token, so the same code can point at another region or a stealth route with a proxy by changing one environment variable.

Store the API token​

An environment variable is fine to get started. Lambda encrypts environment variables at rest, but anyone who can view the function configuration can read them. For production, keep the token in AWS Secrets Manager or Parameter Store and read it at runtime, either through the AWS Parameters and Secrets Lambda Extension or the Powertools for AWS Lambda parameters utility. Both cache the value, so you don't call Secrets Manager on every invocation. The execution role needs secretsmanager:GetSecretValue on the secret.

Deploy the function​

Pick the AWS CLI for a quick test or AWS SAM for a repeatable deployment. Both deploy a .zip package, not a container image. With no browser in the package there's no reason to use a container image.

  1. Install dependencies and build the package

    mkdir browserless-lambda && cd browserless-lambda
    npm init -y
    npm install puppeteer-core
    # Add index.mjs from the previous section, then zip the handler and node_modules at the archive root.
    zip -r function.zip index.mjs package.json node_modules
  2. Create the function

    aws lambda create-function \
    --function-name browserless-scraper \
    --runtime nodejs24.x \
    --handler index.handler \
    --role arn:aws:iam::111122223333:role/your-lambda-role \
    --memory-size 512 \
    --timeout 90 \
    --environment '{"Variables":{"BROWSERLESS_TOKEN":"YOUR_API_TOKEN_HERE","BROWSERLESS_URL":"wss://production-sfo.browserless.io"}}' \
    --zip-file fileb://function.zip

    Replace the role ARN with your function's execution role. To ship a code change later, rebuild function.zip and run aws lambda update-function-code --function-name browserless-scraper --zip-file fileb://function.zip.

  3. Invoke it

    aws lambda invoke \
    --function-name browserless-scraper \
    --cli-binary-format raw-in-base64-out \
    --payload '{"url": "https://example.com/"}' \
    response.json && cat response.json

    Response

    {"url": "https://example.com/", "title": "Example Domain"}

Deploy with AWS SAM​

The same Node.js function as an AWS SAM template. Put index.mjs and package.json in src/; sam build installs the dependencies listed in package.json. The token is a NoEcho parameter, so it isn't printed in CloudFormation output.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Parameters:
BrowserlessToken:
Type: String
NoEcho: true
Resources:
BrowserlessScraper:
Type: AWS::Serverless::Function
Properties:
CodeUri: src/
Handler: index.handler
Runtime: nodejs24.x
MemorySize: 512
Timeout: 90
Environment:
Variables:
BROWSERLESS_TOKEN: !Ref BrowserlessToken
BROWSERLESS_URL: wss://production-sfo.browserless.io
sam build
sam deploy --guided

sam deploy --guided prompts for the BrowserlessToken value and saves the rest of your answers to samconfig.toml.

Timeout and memory settings​

Timeout: Lambda's default timeout is 3 seconds, which is too short to connect and load a page. Set the Lambda timeout above the longest job you expect, and keep the Browserless timeout query parameter below it with a margin for the time it takes to connect. The Lambda clock starts with the invocation, but the Browserless session starts only when the handler connects, so equal values would let a session run briefly after Lambda stops the function. In the examples, Browserless ends the session at 60 seconds and Lambda stops the function at 90, which leaves a 30-second margin. Treat the Browserless timeout as the cleanup bound for sessions that finally never closes. The Lambda maximum is 900 seconds (15 minutes).

Memory: The browser's CPU and memory use happens on Browserless, so the function only needs enough for the client library, the WebSocket connection, and whatever data you return. We recommend starting at 512 MB rather than the 128 MB default. Lambda allocates CPU in proportion to memory, and a little more CPU shortens the time spent loading the library on a cold start. The Python handler needs more headroom than the Node.js ones because Playwright for Python runs a Node.js driver as a subprocess. Check memory use and duration in CloudWatch and adjust from there; you don't need the multi-gigabyte settings that in-function Chromium usually requires.

Concurrency: Every concurrent invocation opens its own Browserless session, and sessions over your plan's concurrency limit get a 429. If an event source can fan out to many invocations at once, cap the function with reserved concurrency below your plan's limit.

Pick the closest region​

Browserless runs in three regions. Set BROWSERLESS_URL to the one closest to your Lambda region to keep the WebSocket round trips short. Every Puppeteer or Playwright call is a round trip over that connection, so latency between the function and the browser adds up over a script.

Lambda regionBrowserless endpoint
US regions, such as us-west-1, us-west-2, us-east-1wss://production-sfo.browserless.io
eu-west-2 (London), eu-west-1 (Ireland)wss://production-lon.browserless.io
eu-central-1 (Frankfurt) and other continental European regionswss://production-ams.browserless.io

These are starting points, not measurements. If the sites you load are far from your function, the distance between the browser and the target site matters too. Measure from your own function and pick the fastest. There's no Asia-Pacific region today. If you need traffic to exit from a specific country, use the residential proxy with proxyCountry rather than a closer region. See Regional endpoints for the full list.

Handle bot-protected sites​

Because BROWSERLESS_URL is read from the environment, you can turn on bot-detection features without touching the handler. Add the path and query parameters to the URL; the handler appends the token and timeout.

NeedBROWSERLESS_URL valueDocs
Stealth browserwss://production-sfo.browserless.io/stealthStealth routes
Stealth plus residential proxywss://production-sfo.browserless.io/stealth?proxy=residential&proxyCountry=usProxies
Stealth, proxy, and CAPTCHA solvingwss://production-sfo.browserless.io/stealth?proxy=residential&proxyCountry=us&solveCaptchas=trueCAPTCHA solving

Start with the stealth route alone and add the proxy only if you're still blocked: residential proxy traffic is billed per MB. Stealth routes keep the same proxy IP for the whole session by default. CAPTCHA solving can take a while, so raise the timeout value in the handler and the Lambda timeout together when you use it.

If a site still blocks you, use the /unblock API or BrowserQL, which are built for the strictest bot detectors.

Call a REST API instead of driving a browser​

If the function only needs a page's content, a screenshot, or a PDF, call a Browserless REST API with fetch. There's no browser library to package at all: the handler below has no dependencies, so the .zip contains only index.mjs.

/smart-scrape returns LLM-ready markdown and escalates from a plain HTTP fetch to a proxied fetch, a stealth browser, and CAPTCHA solving only when the page needs it:

// Reuse the region from BROWSERLESS_URL: switch to https and drop any path such as /stealth.
const browserlessUrl = new URL(process.env.BROWSERLESS_URL ?? "wss://production-sfo.browserless.io");
browserlessUrl.protocol = "https:";
const BASE_URL = process.env.BROWSERLESS_HTTP_URL ?? browserlessUrl.origin;

export const handler = async (event) => {
const response = await fetch(`${BASE_URL}/smart-scrape?token=${process.env.BROWSERLESS_TOKEN}`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ url: event.url, formats: ["markdown"] }),
});

if (!response.ok) {
throw new Error(`smart-scrape failed: ${response.status} ${await response.text()}`);
}

const { ok, message, strategy, markdown, metadata } = await response.json();
// Smart Scrape can answer 200 with ok: false, so check the body too.
if (!ok) {
throw new Error(`smart-scrape failed: ${message}`);
}
return { ok, strategy, title: metadata?.title, markdown };
};

Response

{
"ok": true,
"strategy": "http-fetch",
"title": "Example Domain",
"markdown": "Example Domain\n\nThis domain is for use in documentation examples..."
}

/screenshot and /pdf work the same way but return binary data. A synchronous Lambda response is limited to 6 MB, so write large screenshots and PDFs to S3 and return the object key instead of the file.

Jobs that run longer than one invocation​

A Lambda invocation can run for at most 15 minutes, and a Browserless session is also capped by your plan:

PlanMaximum Session Duration
Free2 minutes
Prototyping (20k)15 minutes
Starter (180k)30 minutes
Scale (500k)60 minutes
Enterprise (self-hosted)Custom

The effective limit for one invocation is the lower of the two. For longer workflows, split the work across several invocations and keep browser state between them with the Session API, which persists cookies, localStorage, and cache across connections and works with both Puppeteer and Playwright. See Session management to choose between that and short-lived reconnects.

Self-packaged Chromium compared with Browserless​

Chromium in the functionBrowserless
Deployment packageClient library plus a Lambda-specific Chromium build; close to the 250 MB .zip limit or a container imageClient library only (puppeteer-core, playwright-core, or playwright), or no dependencies with the REST APIs
Cold startLoad a large package, then launch ChromiumLoad the client library, then open a WebSocket
Lambda memorySized for the browserSized for your code
MaintenanceKeep the Chromium build, the library version, and Amazon Linux libraries compatibleWe update the browsers; match versions using version compatibility
BlockingTraffic exits from AWS IP rangesStealth routes, residential proxies, and CAPTCHA solving through the connection URL
Session lengthUp to the 15-minute Lambda maximumThe lower of the Lambda timeout and your plan's session limit

FAQ & Troubleshooting​

The function fails with Task timed out after 3.00 seconds

Raise the function timeout. Lambda's default is 3 seconds, which isn't enough to connect to a browser and load a page. Set it with --timeout on create-function or update-function-configuration, or Timeout in SAM, and keep it above the Browserless timeout query parameter with a margin for connecting. See Timeout and memory settings.

If the function is attached to a VPC, the same error can mean it has no route to the internet. See the next question.

The WebSocket never connects and the function times out

If the function is attached to a VPC, it can't reach the internet unless it runs in private subnets that route through a NAT gateway. Putting it in a public subnet doesn't give it internet access. Either remove the VPC configuration if the function doesn't need private resources, or add the NAT gateway route. Also check that the security group allows outbound traffic on port 443.

Why am I getting 401 Unauthorized or Unexpected server response: 401?

The token in the connection URL is missing or wrong. Playwright reports 401 Unauthorized with Invalid API key. Puppeteer rejects with Unexpected server response: 401, which Lambda can log only as [object Object] because the rejection isn't an Error. Check that BROWSERLESS_TOKEN is set on the function, not only in your local shell. In Node.js, an unset variable turns into the literal string undefined in the URL. Copy the current token from your account dashboard and update the function configuration.

I'm getting 429 Too Many Requests when traffic spikes

More invocations are running at once than your plan's concurrency allows, since each one opens its own session. Cap the function with reserved concurrency below your plan's limit, and make sure every handler closes the browser in finally.

Node.js fails with Runtime.ImportModuleError or Cannot find module 'puppeteer-core'

The node_modules folder isn't at the root of function.zip, or the handler setting doesn't match the file name. Zip from inside the project folder so index.mjs and node_modules sit at the archive root, and use index.handler for a file named index.mjs. If you name the file index.js and use import, add "type": "module" to package.json.

Python fails with Runtime.ImportModuleError mentioning greenlet

The dependencies were installed for your local OS instead of Lambda's. greenlet is compiled, so a wheel built for macOS or Windows won't load on Amazon Linux. Rebuild the package with the pip install --platform manylinux2014_x86_64 --only-binary=:all: command from the deploy step (or manylinux2014_aarch64 for arm64), matching --python-version to the function's runtime.

The script is slow even though each page loads quickly in the dashboard

The function and the browser are probably far apart. Every Puppeteer or Playwright call crosses the WebSocket, so a function in Europe talking to production-sfo pays that round trip many times per script. Switch BROWSERLESS_URL to the closest region, or move work that needs many small calls into a single REST API request.

Next steps​

Was this page helpful?