---
title: Jobs API
description: Inspect queued scrape, analyse and map operations or retrieve their stored results.
---

Every distinct submitted URL creates one ScrapeJob. Exact duplicate URL strings are removed in first-occurrence order. Multiple jobs share a `batchId`; there is no parent job. Jobs complete independently. Deterministic scrape, AI extraction, analyse and map use the same durable queue. AI admits one seed and one extraction target within a single job.

Identifiers (`job_…`, `batch_…`, `ak_…`) are opaque strings: a prefix and 32 hexadecimal characters. Records created before this format keep their original, differently shaped IDs, so store and compare IDs as whole strings and never parse them.

API-key authentication is sufficient without a user session and only exposes that key's jobs. Clerk users can read jobs across their own keys. Foreign IDs and batches return `404 NOT_FOUND` using the [structured error contract](/docs/api/errors).

## Submission and polling

`POST /scrape`, `/scrape/html`, `/scrape/json`, `/scrape/browser`, `/scrape/ai`, `/analyse` and `/map` accept `?wait=true|false` (default true). HTTP 200 contains completed result arrays for scrape/analyse, or a map aggregate. HTTP 202 contains full jobs in submission order when `wait=false` or the wait deadline expires, including siblings that already completed. Responses include `X-Scrape-Job-Ids` and, for batches, `X-Scrape-Batch-Id`.

Poll the existing IDs or batch; never submit the request again just to wait. Disconnecting detaches the waiter without cancelling jobs. Expired leases fail individually; potentially mutating actions are never automatically replayed. There is no cancellation or retry endpoint.

## `GET /jobs`

```text
GET /jobs?status=failed&engine=html&limit=20
Authorization: Bearer <token>
```

| Filter | Meaning |
|---|---|
| `status` | pending, running, completed, failed, cancelled (historical only) |
| `engine` | html, json, browser, ai, analyse, map |
| `limit` | 1–200; default 50; newest first |
| `batchId` | Complete visible batch in submission order, ignoring history limit |
| `legacyParentId` | Jobs migrated from an old request-level queue parent |
| `apiKeyId` | Clerk history filter for an owned key; API-key callers remain scoped to their own key |

Returns full public ScrapeJob rows:

```ts
Array<{
  id: string;
  apiKeyId: string;
  userId: string | null;
  engine: 'html' | 'json' | 'browser' | 'ai' | 'analyse' | 'map';
  status: 'pending' | 'running' | 'completed' | 'failed' | 'cancelled';
  batchId: string | null;
  batchPosition: number;
  legacyParentId: string | null;
  request: object; // operation input, scalar url, engine; private inputs omitted
  result: object | null; // ScrapeResult | AiScrapeResult | AnalyseResult | MapResult
  error: object | null;
  recordingUrl: string | null;
  replayUrl: string | null;
  createdAt: string;
  startedAt: string | null;
  completedAt: string | null;
}>
```

Public requests omit custom headers and custom proxy inputs. Private credentials and lease fields are not returned. Copying a historical request requires those inputs to be re-entered. Extracted data and normal actions/query/URL remain user content; this is not a universal secret-redaction system.

## `GET /jobs/:id`

Returns one full job or 404. The dashboard's [/jobs](/jobs) page uses these job APIs.

```bash
curl -sS https://api.scrapesilo.com/jobs/job_example \
  -H 'Authorization: Bearer sf_…'
```

## Results only

| Endpoint | Response |
|---|---|
| `GET /jobs/:id/result` | 200 stored result when terminal, including JSON null if historical output is unavailable; 202 JSON null while pending/running |
| `GET /jobs/results?batchId=...` | Ordered result-or-null array; 200 when all visible jobs are terminal, 202 while any are unfinished |

Missing or foreign IDs/batches return 404. Positional nulls preserve the batch order. Read full jobs for status/error details. Map batch results are per-seed values; the synchronous map endpoint separately combines them into its aggregate.

MCP equivalents are `get_job`, `list_jobs`, `get_job_result` and `list_job_results`. MCP wraps successful job reads in `data: { job }` or `data: { jobs }`, and result-only reads in `data: resultOrNull` or `data: results`. MCP result-only nulls have no HTTP status; inspect full jobs to distinguish unfinished work from unavailable history.

## Limits and history

Current user-plan concurrency limits count running URL jobs across all requests, API keys and API replicas, including map. Plan changes affect the next claim. HTML/browser/analyse resource limits and map traversal capacity also apply. Metered credits and all submitted jobs are admitted in one transaction; map is unmetered. Ownerless service keys bypass user quotas and user concurrency, while resource/backlog caps still apply. Browser jobs use separate contexts initialized from their saved profile, without sharing mutations across sibling URLs.

Terminal history is retained for seven days; monthly usage counters remain authoritative for billing. Existing `ex_` IDs survive migration and resolve through `/jobs/:id`. Old parent UUIDs can be found through `legacyParentId`. Historical multi-seed maps retain one terminal row with `request.legacyAggregate: true`; its original aggregate output cannot be reconstructed as per-seed jobs. Historical batch ordering uses deterministic timestamp/ID order when the original order was never recorded.

REST `/executions` and MCP execution tools have been replaced. Dashboard `/executions` links redirect to `/jobs`; this does not restore the old API contract.

## AI results

Jobs can have `engine: "ai"` and request `{ engine: "ai", url, query }`. The stored result retains canonical engine metadata and extracted JSON only in `data`, with tested `recipe`, `extractionStatus: "complete"` and optional model/step/usage metadata alongside it. Numeric `status` retains HTTP meaning. Result reads return it unchanged. The final data is selected from successful executed actions, not written by the model. Failures preserve `data: null` plus `error`. Owner scoping and pending-result behavior match other engines.