---
title: Engines
description: When to pick HTML or Browser — latency, fidelity, and cost trade-offs.
---

import { Tabs, TabItem } from '@astrojs/starlight/components';

The `engine` field on every scrape request decides how the page is fetched and how the DSL is executed.

## At a glance

| Engine | Fetch | JS execution | Latency | Cost | Use when |
|---|---|---|---|---|---|
| `html` | `fetch()` | No | ~100–400 ms | 1 credit | Static pages, SSR'd HTML, RSS, sitemaps, anything that renders without JS. |
| `browser` | Headless Chromium | Yes | ~2–5 s | 5 credits | SPAs, anti-bot defenses, login flows, anything that needs JS or interaction. |

Cost is per URL — see [Pricing & quotas](/docs/pricing) for the full table and how multi-URL requests are charged.

## Decision tree

1. **Try `html` first.** Cheapest (1 credit), fastest. Static / SSR pages usually have the fields in the raw HTML.
2. **Escalate to `browser`** when `html` returns null or empty fields, the page is JS-rendered, or you need click / fill / wait / navigation.
3. **Use `analyse`** (`POST /analyse`) when you want interpretation — outline, summary, “is X present?” — not exact CSS fields. Analyse is one LLM pass over page markdown; it does not emit CSS actions.

Prove the split on the fixture pages: [`/fixtures/article`](https://scrapesilo.com/fixtures/article) is static (html works); [`/fixtures/js-list`](https://scrapesilo.com/fixtures/js-list) is empty until JS runs.

## Options matrix

| Option | `html` | `browser` |
|---|---|---|
| `options.waitFor` | **Ignored** | Lifecycle (`load` / `domcontentloaded` / `networkidle` / `commit`), ms delay, or CSS selector. Default goto lifecycle when `waitFor` is ms/selector: `domcontentloaded`. |
| `options.timeoutMs` | Honoured — AbortController on the fetch. Default 30s, max 120s. | Honoured — navigation + action timeouts. Default 30s, max 120s. |
| `resolution` | Ignored | Viewport: `"desktop"` (1280×800), `"mobile"` (390×844 + mobile UA), or `{ width, height }`. |
| `headless` | Ignored | Default `true`. `false` renders into a real display — better fingerprint stealth, higher CPU/RAM. |
| `blockAds` | Ignored | Default `true`. `false` stops refusing known ad/tracker hosts. |
| `solveCaptcha` | Ignored | Opt-in 2captcha on detected reCAPTCHA; outcome on `result.captcha`. |
| `sessionId` | Matching cookies only (`Cookie` header). `localStorage` ignored. | Full storageState: cookies + per-origin `localStorage`. |

## Worked examples

<Tabs>
<TabItem label="html">

Static article. Raw HTML already has `h1`, `p`, and `a.sf-cta`.

```json
{
  "url": "https://scrapesilo.com/fixtures/article",
  "engine": "html",
  "actions": {
    "title": "h1",
    "body": "p",
    "cta": "a.sf-cta@href"
  }
}
```

</TabItem>
<TabItem label="browser">

Form needs interaction: fill the query, click submit, then extract the results.

```json
{
  "url": "https://scrapesilo.com/fixtures/form",
  "engine": "browser",
  "actions": {
    "type": { "fn": "fill", "selector": "#sf-q", "args": "widget" },
    "submit": { "fn": "click", "selector": "#sf-submit" },
    "title": ".sf-result-title",
    "count": ".sf-result-count"
  }
}
```

</TabItem>
<TabItem label="why html is empty">

[`/fixtures/js-list`](https://scrapesilo.com/fixtures/js-list) has no `.sf-item` nodes in the raw HTML — they appear after client JS. An html scrape returns empty `items`; browser does not.

```json
{
  "url": "https://scrapesilo.com/fixtures/js-list",
  "engine": "html",
  "actions": {
    "items": { "selector": ".sf-item", "many": true, "output": "text" }
  }
}
```

```json
{
  "url": "https://scrapesilo.com/fixtures/js-list",
  "engine": "browser",
  "actions": {
    "items": { "selector": ".sf-item", "many": true, "output": "text" }
  }
}
```

</TabItem>
</Tabs>

## `html` — plain fetch

`html` fetches the page with undici (`fetch`, follow redirects) and runs the action tree over the static DOM. No JS execution.

`fn` actions `wait`, `goto`, `click`, `fill`, `selectOption`, and `2fa` are **rejected** (warning: use `browser`). `evaluate` is allowed as **sync JSDOM `eval` only** — no async / Promise. Full fn matrix: [Actions DSL](/docs/actions-dsl).

### Non-2xx

A non-2xx response is a failure for `html`: that URL’s result has `data: null`, `error` set, and the captured HTTP `status`. Sibling URLs in the same request still return. Browser does not fail the run on a non-2xx page — it may still run actions and reports the real page `status`.

## `browser` — real Chromium

Full browser runtime: navigation, click / fill / select, evaluate (with async / timeout race), 2FA via `otpauth`. Pages run in Chromium with stealth and ad-blocking on by default.

`options.waitFor` controls when navigation is considered done — a lifecycle keyword (`networkidle` is safest for modern SPAs; `domcontentloaded` is fastest), a number of ms, or a CSS selector to wait for. Ignored on `html`. `options.timeoutMs` caps the per-page run at up to 120 s on both engines.

### Authenticated sessions

Capture a logged-in browser session once via the dashboard's live recorder, then replay it by passing the captured id as top-level `sessionId`. Browser restores cookies + per-origin `localStorage`. Html sends only the matching cookies on the initial request (`localStorage` is unreachable from a plain HTTP fetch). List saved sessions with `GET /sessions`.

## Multiple URLs per request

`url` accepts either a single URL or an array. The same `actions` run against every URL; the engine fans them out through a bounded pool and returns one result item per URL, so one bad URL fails on its own without sinking the batch. Each URL is charged independently (cost = per-engine credit × URL count).

| Engine | Max URLs | Run concurrency |
|---|---|---|
| `html` | 50 | 10 |
| `browser` | 10 | 3 |

```json
{
  "url": [
    "https://scrapesilo.com/fixtures/article",
    "https://scrapesilo.com/fixtures/listing"
  ],
  "engine": "html",
  "actions": { "title": "h1" }
}
```

## Proxies

Any scrape accepts these optional top-level fields:

| Field | What it does |
|---|---|
| `useProxy: true` | Route the request through the built-in residential pool, when no BYO fields are set. |
| `useProxy: "us"` | Same, but an ISO 3166-1 alpha-2 country code geo-targets the built-in pool. Ignored when BYO is set. |
| `myProxyUrl` | BYO proxy as a single URL with embedded creds, e.g. `http://user:pass@host:port`. |
| `myProxyConfig` | BYO proxy as `{ server, username?, password? }`. |

The proxy applies to both engines transparently.

```json
{
  "url": "https://scrapesilo.com/fixtures/article",
  "engine": "html",
  "actions": { "title": "h1" },
  "useProxy": "de"
}
```

## Copy for an agent

```
ScrapeSilo engines: html (1 credit, plain fetch) or browser (5 credits, Chromium).
Try html first. Escalate to browser when fields are empty, the page needs JS, or you need click/fill/wait. Use POST /analyse for interpretation, not scrape.

Rules:
- engine is only "html" | "browser". actions required. No query on /scrape.
- options.waitFor is ignored on html. timeoutMs honoured on both (default 30s, max 120s).
- resolution, headless, blockAds, solveCaptcha are browser-only.
- sessionId: html = cookies only; browser = cookies + localStorage.
- html rejects fn wait/goto/click/fill/selectOption/2fa. evaluate is sync JSDOM only.
- html non-2xx → data null + error + status. Caps: html 50 URLs, browser 10.

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer $SCRAPESILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/article",
        "engine": "html",
        "actions": { "title": "h1", "body": "p", "cta": "a.sf-cta@href" }
      }'

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer $SCRAPESILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/form",
        "engine": "browser",
        "actions": {
          "type": { "fn": "fill", "selector": "#sf-q", "args": "widget" },
          "submit": { "fn": "click", "selector": "#sf-submit" },
          "title": ".sf-result-title",
          "count": ".sf-result-count"
        }
      }'
```