Skip to main content

Puppeteer & Playwright

The reconnect mutation returns a WebSocket endpoint you can pass to Puppeteer or Playwright. This lets you use BQL for unblocking (solving CAPTCHAs, bypassing bot detection) and then hand off control to your library for everything else.

Reconnecting with More BQL

You can also reconnect to run more BQL queries instead of connecting a library. See the Reconnect to Browserless guide.

The Reconnect Mutation​

Run your BQL mutations (navigation, unblocking, etc.) and call reconnect to get a browserWSEndpoint. This is the WebSocket URL you pass to Puppeteer's connect() or Playwright's connectOverCDP().

The timeout argument controls how long (in milliseconds) the browser waits for a library connection. If nothing connects within this window, the browser shuts down and returns a 404.

mutation Reconnect {
  goto(url: "https://www.browserless.io/practice-form/", waitUntil: networkIdle) {
    status
  }

  # Uncomment to solve Cloudflare before handing off:
  # solve(type: cloudflare) {
  #   found
  #   solved
  #   time
  # }

  reconnect(timeout: 30000) {
    browserWSEndpoint
  }
}

Connecting with Libraries​

The browserWSEndpoint from the response connects to any library that supports the Chrome DevTools Protocol (CDP). This includes Puppeteer (puppeteer.connect()) and Playwright (chromium.connectOverCDP()).

Execute your BQL query first, extract the browserWSEndpoint, then pass it to your library. The examples below navigate to a page with BQL, reconnect, and take a screenshot.

Connecting to Puppeteer​

import puppeteer from 'puppeteer-core';

const url = 'https://www.browserless.io/practice-form';
const token = 'YOUR_API_TOKEN_HERE';
const timeout = 5 * 60 * 1000;

const queryParams = new URLSearchParams({
  timeout,
  token,
}).toString();

const query = `
  mutation Reconnect($url: String!) {
    goto(url: $url, waitUntil: networkIdle) {
      status
    }
    reconnect(timeout: 30000) {
      browserWSEndpoint
    }
  }
`;

const variables = { url };

const endpoint = `https://production-sfo.browserless.io/chromium/bql?${queryParams}`;

const options = {
  method: 'POST',
  headers: {
    'content-type': 'application/json'
  },
  body: JSON.stringify({
    query,
    variables,
  }),
};

try {
  console.log(`Running BQL Query: ${url}`);

  const response = await fetch(endpoint, options);

  if (!response.ok) {
    throw new Error(`Got non-ok response:\n` + (await response.text()));
  }

  const { data } = await response.json();
  const browserWSEndpoint = data.reconnect.browserWSEndpoint
  console.log(`Got OK response! Connecting puppeteer to ${browserWSEndpoint}`);
  const browser = await puppeteer.connect({
    browserWSEndpoint,
  });
  console.log(`Connected to ${await browser.version()}`);
  const pages = await browser.pages();
  const page = pages.find((p) => p.url().includes(url));
  await page.screenshot({ fullPage: true, path: 'temp.png' });
  await browser.close();
} catch (error) {
  console.error(error);
}

Connecting to Playwright​

import playwright from 'playwright-core';

const url = 'https://www.browserless.io/practice-form';
const token = 'YOUR_API_TOKEN_HERE';
const timeout = 5 * 60 * 1000;

const queryParams = new URLSearchParams({
  timeout,
  token,
}).toString();

const query = `
  mutation Reconnect($url: String!) {
    goto(url: $url, waitUntil: networkIdle) {
      status
    }
    reconnect(timeout: 30000) {
      browserWSEndpoint
    }
  }
`;

const variables = { url };

const endpoint =
  `https://production-sfo.browserless.io/chromium/bql?${queryParams}`;

const options = {
  method: 'POST',
  headers: {
    'content-type': 'application/json',
  },
  body: JSON.stringify({
    query,
    variables,
  }),
};

(async () => {
  try {
    console.log(`Running BQL Query: ${url}`);

    const response = await fetch(endpoint, options);

    if (!response.ok) {
      throw new Error(`Got non-ok response:\n` + (await response.text()));
    }

    const { data } = await response.json();
    const browserWSEndpoint = data.reconnect.browserWSEndpoint;

    console.log(`Got OK response! Connecting Playwright to ${browserWSEndpoint}`);

    const browser = await playwright.chromium.connectOverCDP(browserWSEndpoint);
    const context = browser.contexts()[0];
    const page = context.pages().find((p) => p.url().includes(url));

    if (!page) {
      throw new Error(`Could not find a page matching ${url}`);
    }

    await page.screenshot({ fullPage: true, path: 'temp.png' });

    await browser.close();
  } catch (error) {
    console.error(error);
  }
})();