Engines

import { Tabs, TabItem } from ‘@astrojs/starlight/components’;

The engine field on every scrape request decides how the page is fetched and how the DSL is executed.

At a glance

Engine	Fetch	JS execution	Latency	Cost	Use when
html	fetch()	No	~100–400 ms	1 credit	Static pages, SSR’d HTML, RSS, sitemaps, anything that renders without JS.
browser	Headless Chromium	Yes	~2–5 s	5 credits	SPAs, anti-bot defenses, login flows, anything that needs JS or interaction.

Cost is per URL — see Pricing & quotas for the full table and how multi-URL requests are charged.

Decision tree

1. Try html first. Cheapest (1 credit), fastest. Static / SSR pages usually have the fields in the raw HTML.
2. Escalate to browser when html returns null or empty fields, the page is JS-rendered, or you need click / fill / wait / navigation.
3. Use analyse (POST /analyse) when you want interpretation — outline, summary, “is X present?” — not exact CSS fields. Analyse is one LLM pass over page markdown; it does not emit CSS actions.

Prove the split on the fixture pages: /fixtures/article is static (html works); /fixtures/js-list is empty until JS runs.

Options matrix

Option	html	browser
options.waitFor	Ignored	Lifecycle (load / domcontentloaded / networkidle / commit), ms delay, or CSS selector. Default goto lifecycle when waitFor is ms/selector: domcontentloaded.
options.timeoutMs	Honoured — AbortController on the fetch. Default 30s, max 120s.	Honoured — navigation + action timeouts. Default 30s, max 120s.
resolution	Ignored	Viewport: "desktop" (1280×800), "mobile" (390×844 + mobile UA), or { width, height }.
headless	Ignored	Default true. false renders into a real display — better fingerprint stealth, higher CPU/RAM.
blockAds	Ignored	Default true. false stops refusing known ad/tracker hosts.
solveCaptcha	Ignored	Opt-in 2captcha on detected reCAPTCHA; outcome on result.captcha.
sessionId	Matching cookies only (Cookie header). localStorage ignored.	Full storageState: cookies + per-origin localStorage.

Worked examples

Static article. Raw HTML already has h1, p, and a.sf-cta.

{
  "url": "https://scrapesilo.com/fixtures/article",
  "engine": "html",
  "actions": {
    "title": "h1",
    "body": "p",
    "cta": "a.sf-cta@href"
  }
}

Form needs interaction: fill the query, click submit, then extract the results.

{
  "url": "https://scrapesilo.com/fixtures/form",
  "engine": "browser",
  "actions": {
    "type": { "fn": "fill", "selector": "#sf-q", "args": "widget" },
    "submit": { "fn": "click", "selector": "#sf-submit" },
    "title": ".sf-result-title",
    "count": ".sf-result-count"
  }
}

/fixtures/js-list has no .sf-item nodes in the raw HTML — they appear after client JS. An html scrape returns empty items; browser does not.

{
  "url": "https://scrapesilo.com/fixtures/js-list",
  "engine": "html",
  "actions": {
    "items": { "selector": ".sf-item", "many": true, "output": "text" }
  }
}

{
  "url": "https://scrapesilo.com/fixtures/js-list",
  "engine": "browser",
  "actions": {
    "items": { "selector": ".sf-item", "many": true, "output": "text" }
  }
}

html — plain fetch

html fetches the page with undici (fetch, follow redirects) and runs the action tree over the static DOM. No JS execution.

fn actions wait, goto, click, fill, selectOption, and 2fa are rejected (warning: use browser). evaluate is allowed as sync JSDOM eval only — no async / Promise. Full fn matrix: Actions DSL.

Non-2xx

A non-2xx response is a failure for html: that URL’s result has data: null, error set, and the captured HTTP status. Sibling URLs in the same request still return. Browser does not fail the run on a non-2xx page — it may still run actions and reports the real page status.

browser — real Chromium

Full browser runtime: navigation, click / fill / select, evaluate (with async / timeout race), 2FA via otpauth. Pages run in Chromium with stealth and ad-blocking on by default.

options.waitFor controls when navigation is considered done — a lifecycle keyword (networkidle is safest for modern SPAs; domcontentloaded is fastest), a number of ms, or a CSS selector to wait for. Ignored on html. options.timeoutMs caps the per-page run at up to 120 s on both engines.

Authenticated sessions

Capture a logged-in browser session once via the dashboard’s live recorder, then replay it by passing the captured id as top-level sessionId. Browser restores cookies + per-origin localStorage. Html sends only the matching cookies on the initial request (localStorage is unreachable from a plain HTTP fetch). List saved sessions with GET /sessions.

Multiple URLs per request

url accepts either a single URL or an array. The same actions run against every URL; the engine fans them out through a bounded pool and returns one result item per URL, so one bad URL fails on its own without sinking the batch. Each URL is charged independently (cost = per-engine credit × URL count).

Engine	Max URLs	Run concurrency
html	50	10
browser	10	3

{
  "url": [
    "https://scrapesilo.com/fixtures/article",
    "https://scrapesilo.com/fixtures/listing"
  ],
  "engine": "html",
  "actions": { "title": "h1" }
}

Proxies

Any scrape accepts these optional top-level fields:

Field	What it does
useProxy: true	Route the request through the built-in residential pool, when no BYO fields are set.
useProxy: "us"	Same, but an ISO 3166-1 alpha-2 country code geo-targets the built-in pool. Ignored when BYO is set.
myProxyUrl	BYO proxy as a single URL with embedded creds, e.g. http://user:pass@host:port.
myProxyConfig	BYO proxy as { server, username?, password? }.

The proxy applies to both engines transparently.

{
  "url": "https://scrapesilo.com/fixtures/article",
  "engine": "html",
  "actions": { "title": "h1" },
  "useProxy": "de"
}

Copy for an agent

ScrapeSilo engines: html (1 credit, plain fetch) or browser (5 credits, Chromium).
Try html first. Escalate to browser when fields are empty, the page needs JS, or you need click/fill/wait. Use POST /analyse for interpretation, not scrape.

Rules:
- engine is only "html" | "browser". actions required. No query on /scrape.
- options.waitFor is ignored on html. timeoutMs honoured on both (default 30s, max 120s).
- resolution, headless, blockAds, solveCaptcha are browser-only.
- sessionId: html = cookies only; browser = cookies + localStorage.
- html rejects fn wait/goto/click/fill/selectOption/2fa. evaluate is sync JSDOM only.
- html non-2xx → data null + error + status. Caps: html 50 URLs, browser 10.

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer $SCRAPESILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/article",
        "engine": "html",
        "actions": { "title": "h1", "body": "p", "cta": "a.sf-cta@href" }
      }'

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer $SCRAPESILO_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/form",
        "engine": "browser",
        "actions": {
          "type": { "fn": "fill", "selector": "#sf-q", "args": "widget" },
          "submit": { "fn": "click", "selector": "#sf-submit" },
          "title": ".sf-result-title",
          "count": ".sf-result-count"
        }
      }'