Automate Google Search

Navigate to Google, submit a search query, and extract result titles and links.

Prerequisites

• A Browserless API token from your account dashboard

Steps

REST API

Use the /scrape REST endpoint to extract Google search results. No WebSocket connection needed.

note

Google actively blocks automated requests. The /scrape endpoint has no stealth capabilities, so results may be empty or return a CAPTCHA page. The BQL tab uses BrowserQL's built-in stealth and CAPTCHA solving. Use it for more reliable results.

cURL

View Full Code on GitHub

1. Build the request

Pass the query directly in the URL and use the /scrape endpoint to extract headings from the results page:

https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE

2. Send the request

curl -X POST \
  "https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE" \
  -H "Content-Type: application/json" \
  -d '{
    "url": "https://www.google.com/search?q=Browserless+headless+browser",
    "elements": [{ "selector": "h3" }]
  }'

3. Check the output

Each entry in results includes the text, HTML, dimensions, and position of the matched element:

{
  "data": [
    {
      "results": [
        {
          "attributes": [{ "name": "class", "value": "..." }],
          "height": 48,
          "html": "Browserless | Headless Browser Automation & Scraping",
          "left": 180,
          "text": "Browserless | Headless Browser Automation & Scraping",
          "top": 320,
          "width": 600
        },
        {
          "attributes": [{ "name": "class", "value": "..." }],
          "height": 48,
          "html": "Getting Started | Browserless",
          "left": 180,
          "text": "Getting Started | Browserless",
          "top": 420,
          "width": 600
        }
      ],
      "selector": "h3"
    }
  ]
}

JavaScript

View Full Code on GitHub

1. Send the request

const response = await fetch(
  'https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE',
  {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
      url: 'https://www.google.com/search?q=Browserless+headless+browser',
      elements: [{ selector: 'h3' }],
    }),
  }
);

const { data } = await response.json();
const titles = data[0].results.map((r) => r.text);
console.log(titles);

2. Check the output

Run with node search.mjs. The array contains the title of each search result.

Python

View Full Code on GitHub

1. Install dependencies

pip install requests

2. Send the request

import requests

response = requests.post(
    'https://production-sfo.browserless.io/scrape?token=YOUR_API_TOKEN_HERE',
    json={
        'url': 'https://www.google.com/search?q=Browserless+headless+browser',
        'elements': [{'selector': 'h3'}],
    },
)

data = response.json()['data']
titles = [item['text'] for item in data[0]['results']]
print(titles)

3. Check the output

Run with python search.py. The list contains the title of each search result.

Java

View Full Code on GitHub

1. Add dependencies

<!-- https://kong.github.io/unirest-java/ -->
<dependency>
  <groupId>com.konghq</groupId>
  <artifactId>unirest-java</artifactId>
  <version>3.14.5</version>
</dependency>
<!-- Add Gson for JSON parsing -->
<dependency>
  <groupId>com.google.code.gson</groupId>
  <artifactId>gson</artifactId>
  <version>2.10.1</version>
</dependency>

2. Send the request

import com.google.gson.*;
import kong.unirest.HttpResponse;
import kong.unirest.Unirest;
import java.util.*;

String url = "https://production-sfo.browserless.io/scrape";
String token = "YOUR_API_TOKEN_HERE";
String endpoint = String.format("%s?token=%s", url, token);

HttpResponse<String> response = Unirest.post(endpoint)
    .header("Content-Type", "application/json")
    .body("{\"url\": \"https://www.google.com/search?q=Browserless+headless+browser\", \"elements\": [{\"selector\": \"h3\"}]}")
    .asString();

JsonArray results = JsonParser.parseString(response.getBody())
    .getAsJsonObject()
    .getAsJsonArray("data")
    .get(0).getAsJsonObject()
    .getAsJsonArray("results");

List<String> titles = new ArrayList<>();
for (JsonElement r : results) {
    titles.add(r.getAsJsonObject().get("text").getAsString());
}
System.out.println(titles);

3. Check the output

Run the class. The list contains the title of each search result.

C#

View Full Code on GitHub

1. Send the request

using System.Net.Http;
using System.Text;
using System.Text.Json;
using System.Linq;

string url = "https://production-sfo.browserless.io/scrape";
string token = "YOUR_API_TOKEN_HERE";
string endpoint = $"{url}?token={token}";

var payload = new
{
    url = "https://www.google.com/search?q=Browserless+headless+browser",
    elements = new[] { new { selector = "h3" } },
};

using (HttpClient httpClient = new HttpClient())
{
    var jsonPayload = JsonSerializer.Serialize(payload);
    var content = new StringContent(jsonPayload, Encoding.UTF8, "application/json");
    var response = await httpClient.PostAsync(endpoint, content);
    string responseBody = await response.Content.ReadAsStringAsync();

    var json = JsonDocument.Parse(responseBody);
    var results = json.RootElement.GetProperty("data")[0].GetProperty("results");
    var titles = results.EnumerateArray()
        .Select(r => r.GetProperty("text").GetString())
        .ToList();
    Console.WriteLine(string.Join(", ", titles));
}

2. Check the output

Run the program. The list contains the title of each search result.

Frameworks

Use a browser connection to navigate to Google and extract results from the rendered DOM.

Puppeteer

View Full Code on GitHub

1. Install dependencies

npm install puppeteer-core

2. Connect and search

import puppeteer from 'puppeteer-core';

const browser = await puppeteer.connect({
  browserWSEndpoint: 'wss://production-sfo.browserless.io?token=YOUR_API_TOKEN_HERE',
});

try {
  const page = await browser.newPage();
  await page.goto('https://www.google.com', { waitUntil: 'networkidle2' });

  await page.type('textarea[name="q"]', 'Browserless headless browser');
  await page.keyboard.press('Enter');
  await page.waitForSelector('#search');

  const results = await page.evaluate(() =>
    Array.from(document.querySelectorAll('h3')).map((h) => ({
      title: h.innerText,
      url: h.closest('a')?.href ?? null,
    }))
  );

  console.log(results);
} finally {
  // Always close to release the session even on error.
  await browser.close();
}

3. Check the output

Run with node search.mjs. Each object in the array has a title and url.

Playwright

View Full Code on GitHub

1. Install dependencies

npm install playwright-core

2. Connect and search

import { chromium } from 'playwright-core';

const browser = await chromium.connect(
  'wss://production-sfo.browserless.io/chromium/playwright?token=YOUR_API_TOKEN_HERE'
);

try {
  const page = await browser.newPage();
  await page.goto('https://www.google.com', { waitUntil: 'networkidle' });

  await page.fill('textarea[name="q"]', 'Browserless headless browser');
  await page.keyboard.press('Enter');
  await page.waitForSelector('#search');

  const results = await page.evaluate(() =>
    Array.from(document.querySelectorAll('h3')).map((h) => ({
      title: h.innerText,
      url: h.closest('a')?.href ?? null,
    }))
  );

  console.log(results);
} finally {
  // Always close to release the session even on error.
  await browser.close();
}

3. Check the output

Run with node search.mjs. Each object in the array has a title and url.

BQL

View Full Code on GitHub

1. Write the mutation

Pass the query in the URL to skip typing, then use mapSelector to extract all result headings:

mutation GoogleSearch {
  goto(url: "https://www.google.com/search?q=Browserless+headless+browser") {
    status
  }
  results: mapSelector(selector: "h3", wait: true) {
    innerText
  }
}

2. Run it

Paste into the BQL IDE and click Run.

3. Check the output

{
  "data": {
    "goto": { "status": 200 },
    "results": [
      { "innerText": "Browserless | Headless Browser Automation & Scraping" },
      { "innerText": "Getting Started | Browserless" }
    ]
  }
}

Next steps

• Scrape Structured Data — extract more complex data with CSS selectors
• Extract Content from a URL — retrieve the full rendered HTML of any page