Skip to content
Start free

sparkcrawl · web data instrument

Open the web, thenmake its data your own

Fetch pages, run JavaScript and crawl sites into HTML, Markdown and structured data. Choose how far acquisition can go and inspect every attempt.

Start freeSee it in action →

No card required · every claim checkable

Discover · 01

A URL becomes usable data.

Fetch the page first. Run JavaScript when it is needed. Keep the HTML, read the Markdown or extract exactly the elements your application expects.

Output formats
HTML · Markdown · JSON

Verify · 02

Set the limit before the request.

Allow the whole fallback chain or stop after fetch and JavaScript. Required selectors check the content you need; attempts and costs remain visible.

Acquisition ceiling
your choice

Ship · 03

One page, or a whole crawl.

Call the scraping API for one page. Start a crawl for a site, inspect its progress and export the results. Search and cited answers stay available in the same workspace.

Acquisition trace
every attempt recorded

The acquisition API · beta

Fetch. Render. Extract. Crawl.

Give Sparkcrawl a public URL. Get its HTML, a clean Markdown document and the fields selected by your application. Crawl jobs gather linked pages with progress, cancellation and exports.

Automatic fallback is the default. Set a tier ceiling to stop before CAPTCHA or managed recovery, and a spending ceiling to bound external attempts. The result tells you what worked and what remains unverified.

Open the scraping workspace
  1. 1

    Fetch

    Read the HTTP response and extract HTML, Markdown and selected elements.

  2. 2

    JavaScript

    Execute the page when the data needs a browser.

  3. 3

    Challenge

    Resolve supported CAPTCHA challenges when enabled and within budget.

  4. 4

    Managed recovery

    Use the configured scraping service when local acquisition cannot recover.

Beta usage has no customer charge. Paid acquisition requires configured providers and operational budgets. Search retains its existing credit system.

Also in the workspace

Search, read and cite the web.

The research workflow below remains available when your product needs grounded answers alongside extracted pages.

I — The argument

The web is raw material.

Sparkcrawl does not answer from memory. It searches the open web, opens the sources it judges worth reading, and forges what they actually say into an answer — with every claim still attached to the page it came from.

  1. I

    Search

    One question fans out across the open web and the verticals it implies.

    breadth set per question
  2. II

    Crawl

    Approved pages are read in parallel, so a failed read is replaced instead of leaving a hole.

    parallel reads · gaps backfilled
  3. III

    Judge

    Every candidate is checked for relevance, authority, freshness and source type before anything is downloaded.

    admit only what earns it
  4. IV

    Forge

    Only passages that carry the claim survive, and the answer is written against them — never around them.

    every claim keeps its citation

II — Raw material → forged answer

Nothing is written before it is read.

  1. 01 · Raw page

    As found

    postgresql.orgprimary

    Home · Documentation · Release notes · Downloads

    PostgreSQL 18.0 was released on 2025-09-25. This release contains a variety of fixes and improvements, including asynchronous I/O and an upgraded query planner …

    Navigation, cookies and boilerplate are part of the page. The passage that matters is one line of it.

  2. 02 · Selected evidence

    After the judge

    PostgreSQL 18.0 was released on 2025-09-25.

    selected passage

    Selected from the same page. The rest of the page is context, not evidence.

  3. 03 · Cited claim

    In the answer

    The latest stable major version is PostgreSQL 18, released on September 25, 2025.[1]

    The claim keeps the locator: open citation 1 and you land on the line above.

Worked example — the claim above links back to the line it came from.

III — The run

Seven movements, one run.

Every saved run reports its own timing

  1. 01

    Plan

    Each round starts with the queries the question needs — the run never stalls on its own planning.

    breadth set per question

  2. 02

    Search

    Discovery fans out across the open web and the verticals the question implies.

    multiple independent indexes

  3. 03

    Normalize

    Duplicates are fused across channels before anything is ranked.

    deduplicated before ranking

  4. 04

    Judge

    The gate checks relevance, authority, freshness and source type, and refuses what it cannot justify. A rejected source is never downloaded as filler.

    admit only what earns the download

  5. 05

    Fetch

    Approved pages are read in parallel, so a page that fails is replaced instead of leaving a hole in the evidence.

    failures replaced · gaps reported

  6. 06

    Select

    Only passages that carry the claim survive into the answer — never filler around them.

    only passages that carry the claim

  7. 07

    Synthesize

    One streaming pass writes the answer against the selected passages and marks each claim with the citation that resolves to it.

    streams with [n] citations

Fig. 2 — one run, drawn to scalerelative shares
  • search38%
  • judge6%
  • fetch26%
  • select11%
  • ◈ synthesis19%

Shape of one run, not a promise. Your saved runs report their own timing, sources and cost.

IV — Judgment

A source is admitted, not assumed.

The gate is not a filter bolted on afterwards. It runs before acquisition, on the search results themselves, and it is the only reason a page is ever downloaded.

The gate answers four questions per candidate — how directly it answers, how credible it is, how fresh it is, and what kind of source it is — and refuses what it cannot justify.

  1. relevance

    How directly does this result answer the question?

    unrelated → directly answers

  2. authority

    What is the credibility of this source?

    unknown → authoritative first-party

  3. freshness

    How timely is it for this question?

    stale → recent and time-relevant

  4. type

    What kind of source is it?

    primary · documentation · academic · news · community · blog · commerce

Admit only what earns the download.

The gate runs before a single page is fetched. A rejected source is never downloaded as filler — and the run says so instead of writing around the gap.

Fig. 3 — the gate, worked example
  1. 01

    postgresql.orgprimary

    direct · first-party · fresh

    admit

  2. 02

    wiki.postgresql.orgdocumentation

    direct · maintained · relevant

    admit

  3. 03

    news outletnews

    timely · independent

    admit

  4. 04

    seo mirrorblog

    thin · derivative

    reject

  5. 05

    aggregatorother

    indirect · unsourced

    reject

candidate · why · decision

If judgment fails

Retries are finite and the fallback is explicit — the run always reports which path produced the judgment instead of inventing it.

  1. primary judge
  2. fallback judge
  3. search results

V — Source ecosystem

Reads what the question needs, not what is cheapest.

Standard covers the open web; extended adds video and social when the question needs them.

webWeb pages
standard
The default: sources the gate admits are read in full.
second indexWeb pages
standard
An independent index, so one ranking never decides alone.
newsNews
standard
Recency-scoped discovery for questions that move.
academicAcademic
standard
Papers read as documents, not snippets.
codeCode
standard
Repositories and docs read where the question lives.
communityCommunity
standard
Discussion admitted where it helps, held back where it does not.
videoVideo
extended
Transcripts cited with timestamps.
socialSocial
extended
Posts cited with author and time.

Every channel passes through the same gate and the same evidence path.

Web data you can inspect.

No card required · every claim checkable

Start freeOpen the playground

VI — The workbench

A run, end to end.

  1. Searching
  2. Crawling
  3. Evaluating
  4. Synthesizing
  5. Complete

12 sources ranked · 5 opened · 3 cited

Query

What is the latest stable major version of PostgreSQL and when was it released?

Answer

The latest stable major version of PostgreSQL is 18, released on September 25, 2025. Minor updates such as 18.6 continue under active support.[1][2]

Every claim keeps a resolvable citation: the mark opens the span that produced it, including transcript timestamps for video and post ids for social sources.

Open the playground
  1. [1]primary

    postgresql.org

    PostgreSQL 18 release notes

    PostgreSQL 18.0 was released on 2025-09-25 …

    release notes

  2. [2]documentation

    wiki.postgresql.org

    Version history and support windows

    Each major version is supported for five years from its initial release …

    version history

  3. [3]news

    news outlet

    PostgreSQL 18 lands with asynchronous I/O

    The release ships asynchronous I/O and an upgraded planner …

    coverage

Worked example — open the playground to run your own question

VII — For developers

One key. Scrape, crawl, search, research.

Every capability is one HTTPS call under /api/v1 with an organization key, and every result comes back with its sources, its attempts and what it cost. The same key connects the MCP server, so an agent gets the same tools.

POST
/api/v1/scrape
One page as HTML, Markdown or JSON — rendered only when it needs a browser.
POST
/api/v1/crawls
A bounded same-host crawl as a job, with progress and cancellation.
POST
/api/v1/search
Question in — cited answer, sources and cost out.
POST
/api/v1/research
A multi-question report whose citations are checked against its sources.

Request

curl -X POST https://sparkcrawl.com/api/v1/search \  -H "Authorization: Bearer $SPARKCRAWL_API_KEY" \  -H "Content-Type: application/json" \  -d '{"query": "latest stable PostgreSQL release"}'

Authorization: Bearer sc_live_… — organization keys with scrape, crawl, search and research scopes, created in the app.

Read the API docs →

MCP

claude mcp add --transport http sparkcrawl \  https://sparkcrawl.com/mcp \  --header "Authorization: Bearer sc_live_…"
Measured on a frozen evaluation set
timecostcitations
first runsecondscentsevery claim cited
repeat runfaster — cachedcentsevery claim cited
Order of magnitude, not a promise: seconds per answer, cents per run, every claim cited. Your saved runs report their own numbers.

VIII — The price

One balance. Pay per page.

Every organization opens with a US$1 balance — no card. Scrape, crawl, search, research, the Playground and MCP all spend from it at one tariff, and a page that fails costs nothing.

Balance packs

  • $8

    US$8 balance

    up to 16,000 pages

    Choose →

  • $50

    US$55 balance (+10%)

    up to 110,000 pages

    Choose →

  • $200

    US$240 balance (+20%)

    up to 480,000 pages

    Choose →

  • $1,000

    US$1,300 balance (+30%)

    up to 2,600,000 pages

    Choose →

The tariff

page
US$0.0005 per successful page
search
billed as one page
providers
measured cost × 5

Balance is credited in US dollars, with a bonus on the larger packs and no subscription. Each request reserves its ceiling up front and settles at the measured cost. Top up from Billing once you're in. Create free account →

IX — The return

Search the web.Gather the evidence.Forge the answer.

The metaphor is the process, not the product. Behind the plate is a pipeline with a gate — and every answer it writes can be checked line by line.

Start freeOpen the playground

HTML · Markdown · structured data

Questions

How is this different from a search box?
A search box returns links you still have to open and verify. Sparkcrawl opens the sources it judges worth reading, selects the passages that carry the claim, and writes an answer where every claim links back to the span of evidence behind it — so you audit the answer instead of trusting it.
What does a run cost, and how long does it take?
Seconds per answer and cents per run on the frozen set — and every saved run reports its own timing and cost, so the number you see is yours, not ours.
Which sources can it read?
Web pages, news, academic material, code, community discussion, video transcripts and social posts. The standard profile reads the open web; the extended profile adds video and social — every channel through the same gate and the same evidence path.
How do I call it from my own product?
One HTTPS call with an organization key: POST /api/v1/search returns the cited answer, its sources and its cost; /api/v1/scrape, /api/v1/crawls and /api/v1/research cover pages, sites and reports. The same key connects the MCP server at https://sparkcrawl.com/mcp.
What happens if a source fails or moves?
The run reports it instead of inventing the passage: retries are finite, a failed read is replaced from the approved set, and a stage that cannot recover says so instead of writing around the gap.
How do I cap spending?
You spend a prepaid balance, never an open tab: each request reserves its ceiling before it runs and settles at the measured cost, and maxCostUsd caps what any one request may spend on providers. When the balance runs out, requests answer 402 instead of billing past it.