sparkcrawlDocs

Scrape one page synchronously or enqueue with async=1

POST
/api/v1/scrape

Authorization

bearerAuth
headerAuthorizationBearer <sc_live_<43 characters>>

Organization API key created at /app/api. Scopes: scrape, crawl, search, research. The secret is shown once.

Query Parameters

async?"1"
Value in"1"

Header Parameters

Idempotency-Key?string

Replays the original job instead of creating a second one when the same key and body are retried.

Match^[A-Za-z0-9._~-]{1,128}$
Lengthlength <= 128

Request Body

application/json
  1. body
url*string
Formaturi
Lengthlength <= 8192
method?string
Default"GET"
Value in"GET""POST"
headers?
Default{}
body?string
Lengthlength <= 65536
cookies?
Default{}
waitFor?string
Length1 <= length <= 512
waitMs?integer
Range0 <= value <= 10000
Default0
actions?array<|||||||>
Itemsitems <= 10
Default[]
screenshot?boolean
Defaultfalse
screenshotFullPage?boolean
Defaultfalse
screenshotSelector?string
Length1 <= length <= 512
pdf?boolean
Defaultfalse
jsonSchema?
cacheTtlSeconds?integer
Range0 <= value <= 3600
Default0
sessionId?string
Match^sess_[a-f0-9]{32}$
maxCustomerCostUsd?number

Optional ceiling in USD on what this request may charge the organization balance. When omitted and billing is enabled, the server reserves exactly the required ceiling: success-page fee × requested pages + maxCostUsd × (1 + upstream markup) — today US$0.005 per page and a 50% markup (GET /api/v1/pricing). The reservation settles down to the measured charge; a value below the required ceiling is rejected.

Range0 <= value <= 100
formats?array<>
Items1 <= items <= 3
Default
[  "html",  "markdown"]
maxTier?||||
Default"auto"
proxy?string
Default"auto"
Value in"auto""none""residential"
country?string
Match^[a-zA-Z]{2}$
timeoutMs?integer
Range1000 <= value <= 180000
Default120000
maxCostUsd?number
Range0 <= value <= 10
Default0.02
selectors?
Default{}
requiredSelectors?array<>
Itemsitems <= 30
Default[]
containsText?array<>
Itemsitems <= 30
Default[]
webhook?

Response Body

Scrape result

application/json
  1. response
id*string
url*string
Formaturi
finalUrl*string
Formaturi
status*string
Value in"success""challenge""empty""validation_failed""error"
httpStatus?integer|null
tierUsed?integer|null
htmlSource?|
Value in"response""dom"null
title?string
html?string
markdown?string
json?
structuredJson?unknown
jsonLd?array<unknown>
screenshotBase64?string
pdfBase64?string
cacheHit?boolean
cacheRevalidated?boolean
etag?string
cacheControl?string
engine?string
Value in"http""impit""lightpanda""chromium""patchright""2captcha""scrapingbee""scrappa"
stage?string
Value in"http_direct""js_direct""residential""captcha""managed"
verification?string
Value in"passed""not_requested""failed"
attempts*array<>
durationMs?integer
bytes?integer
links?array<>
upstreamCostUsd?number|null
pricingStatus?string
Value in"known""estimated""unknown"
customerChargeUsd*number
artifacts?
error?
Example Request
curl -X POST 'https://sparkcrawl.com/api/v1/scrape' \  -H 'Content-Type: application/json' \  -d '{  "url": "http://example.com"}'
Example Response
{  "id": "string",  "url": "http://example.com",  "finalUrl": "http://example.com",  "status": "success",  "httpStatus": 0,  "tierUsed": 0,  "htmlSource": "response",  "title": "string",  "html": "string",  "markdown": "string",  "json": {},  "structuredJson": null,  "jsonLd": [    null  ],  "screenshotBase64": "string",  "pdfBase64": "string",  "cacheHit": true,  "cacheRevalidated": true,  "etag": "string",  "cacheControl": "string",  "engine": "http",  "stage": "http_direct",  "verification": "passed",  "attempts": [    {      "id": "string",      "tier": 0,      "provider": "string",      "proxy": "string",      "status": "string",      "stage": "string",      "engine": "string",      "durationMs": 0,      "bytes": 0,      "upstreamCostUsd": 0,      "pricingStatus": "string"    }  ],  "durationMs": 0,  "bytes": 0,  "links": [    "http://example.com"  ],  "upstreamCostUsd": 0,  "pricingStatus": "known",  "customerChargeUsd": 0,  "artifacts": {    "html": "string",    "screenshot": "string",    "pdf": "string",    "json": "string"  },  "error": {    "code": "string",    "message": "string"  }}