POST /scrape

The main endpoint. Validates the body, runs the scrape on the html or browser engine, records each URL to executions, and returns an array of results once the run completes. There is no async/polling mode.

engine is only 'html' | 'browser'. actions is required. Interpretation of a page belongs on POST /analyse, not here.

Request

POST /scrape
Authorization: Bearer <token>
Content-Type:  application/json

Body (any field marked optional may be omitted):

{
  url: string | string[];   // one http(s) URL, or an array (see Multiple URLs)
  engine: 'html' | 'browser';
  actions: BrowserActions;  // required — see /docs/actions-dsl

  sessionId?: string;       // replay a recorded auth session — /docs/engines#authenticated-sessions
  options?: {
    waitFor?:    'load' | 'domcontentloaded' | 'networkidle' | 'commit' | number | string;
    timeoutMs?:  number;    // capped at 120000
    resolution?: 'desktop' | 'mobile' | { width: number; height: number }; // browser only
    headless?:   boolean;   // browser only, default true
    blockAds?:   boolean;   // browser only, default true
  };

  useProxy?:      boolean | string;  // true = built-in pool; "us" geo-targets it
  myProxyUrl?:    string;            // BYO: "http://user:pass@host:port"
  myProxyConfig?: { server: string; username?: string; password?: string };
  solveCaptcha?:  boolean;           // browser only — opt-in 2captcha on detected reCAPTCHA
}

The server rejects requests where actions is missing. solveCaptcha is ignored on html (no JS runtime to inject a token). Outcome of a solve, when attempted, is captcha on that URL’s result.

Proxies

With useProxy: true and no BYO fields, the request uses the built-in residential pool. A 2-letter ISO country code ("us", "de") geo-targets that pool. Country is ignored when myProxyUrl / myProxyConfig is set. See Engines → Proxies.

Multiple URLs

url may be an array. The same actions run against each URL, fanned out through a bounded per-engine pool. A bad URL fails on its own without sinking the batch. Each URL is charged separately. Per-request caps: html 50, browser 10. See Engines → Multiple URLs.

Response

Always an array — one item per (deduped) URL:

Array<{
  url: string;                          // final URL, after redirects
  status?: number;                      // page HTTP status; absent if no response
  data: Record<string, unknown> | null; // null on failure
  tookMs: number;
  antibot: Array<{                      // empty when the page is clean
    provider: string;
    detection: string;
    blocked: boolean;
    version?: string;
    sitekey?: string;
  }>;
  captcha?: {                           // only if a solve was attempted
    provider: string;
    solved: boolean;
    tookMs: number;
    error?: string;
  };
  error?: { message: string; name: string; stack: string | null; cause: unknown };
  executionId: string;                  // always present — row in /executions
  batchId?: string;                     // only when the request had more than one URL
}>

Ids

Every item includes executionId. Inspect the full row later via the Executions API. batchId is set only when url was an array with more than one entry.

Errors

HTTP-level failures (the whole request is rejected):

Status	Body	Cause
400	{ "error": "invalid scrape request", "issues": [...] }	The body failed validation. issues[] lists field paths.
401	{ "error": "invalid api key" }	Bearer didn’t resolve.
402	{ "error": "credit quota exceeded", "creditsLimit", "creditsUsed", "creditsRemaining" }	Request would exceed your monthly credit pool. See Pricing & quotas.
429	{ "error": "concurrency limit exceeded", "concurrencyLimit" }	Too many scrapes in flight. Retry-After: 5 header set.
5xx	{ "error": "<message>" }	Engine or browser-node failure. The execution row still exists with the error captured.

A 200 can still contain failed URLs. Those items have data: null and error set; siblings in the same array still return.

Example

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer sf_…" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/article",
        "engine": "html",
        "actions": { "title": "h1", "body": "p", "cta": "a.sf-cta@href" }
      }'

[
  {
    "url": "https://scrapesilo.com/fixtures/article",
    "status": 200,
    "data": {
      "title": "How to pin an article field",
      "body": "This lead is the body field. Select the first paragraph after the heading.",
      "cta": "https://scrapesilo.com/docs/quickstart"
    },
    "tookMs": 287,
    "antibot": [],
    "executionId": "ex_…"
  }
]

The same body is the MCP scrape tool args. See MCP.

Copy for an agent

Use this as a self-contained prompt. engine is "html" or "browser" only. actions is required — a CSS tree of output fields (see Actions DSL). Do not send a planner prompt on this endpoint. The HTTP response is an array; every item has executionId.

curl -sS -X POST https://api.scrapesilo.com/scrape \
  -H "Authorization: Bearer sf_…" \
  -H "Content-Type: application/json" \
  -d '{
        "url": "https://scrapesilo.com/fixtures/article",
        "engine": "html",
        "actions": { "title": "h1", "body": "p", "cta": "a.sf-cta@href" }
      }'