sparkcrawlDocs

From key to first page in two calls.

Sparkcrawl is a web data API — scrape a page, crawl a site, search with cited answers, or run a research report. Every call uses an organization API key and draws from one balance in US dollars.

1. Create an API key

Create an account — every new organization starts with a US$1 welcome balance, no card. Then open /app/api, choose Create key and pick its scopes: scrape, crawl, search, research. The secret starts with sc_live_ and is shown once.

export SPARKCRAWL_API_KEY="sc_live_…"

Keep the key on the server. Owners and admins can revoke it at any time from the same page — authentication covers scopes and errors.

2. Scrape a page

One page, returned synchronously. The cascade starts with a direct request and escalates only when the page needs a browser or a proxy; formats chooses html, markdown or json.

Request
curl -X POST "https://sparkcrawl.com/api/v1/scrape" \
  -H "Authorization: Bearer $SPARKCRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "formats": ["markdown"]}'
Response · trimmed
{
  "id": "…",
  "url": "https://example.com",
  "status": "success",
  "markdown": "# Example Domain\n…",
  "attempts": [ … ],
  "customerChargeUsd": 0.0005
}

Every result reports its attempts and what it cost you in customerChargeUsd. Pages that fail cost nothing.

mode: "answer" reads the sources it judges worth reading and returns an answer whose claims carry citations; mode: "results" returns the judged results without an answer. A search is billed as one page.

Request
curl -X POST "https://sparkcrawl.com/api/v1/search" \
  -H "Authorization: Bearer $SPARKCRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"query": "latest stable PostgreSQL release", "mode": "answer"}'

The response carries answer, citations, sources, completion and customerChargeUsd. Add ?async=1 to get a job back instead of waiting.

4. Crawl and research run as jobs

Crawls, maps, batches and research answer 202 with a job. Read it until status is completed, then page through the results — or pass a webhook and be called when it finishes. Retries with the same Idempotency-Key replay the original job.

Crawl, then poll
curl -X POST "https://sparkcrawl.com/api/v1/crawls" \
  -H "Authorization: Bearer $SPARKCRAWL_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"url": "https://example.com", "maxPages": 20}'

# 202 → { "job": { "id": "…", "status": "queued", … } }
curl "https://sparkcrawl.com/api/v1/jobs/$JOB_ID" \
  -H "Authorization: Bearer $SPARKCRAWL_API_KEY"

Next

  • Billing & errors — the tariff, packs, reservations and every error code.
  • API reference — every /api/v1 operation, generated from the OpenAPI document.
  • MCP — the same API as tools for Claude Code, Cursor and Claude Desktop.