# Async searches > How do I run a search for up to 10,000 jobs and collect the results? Source: https://docs.betterjobs.cc/platform/async-searches/ A synchronous search returns at most 100 jobs per page. For backfills and market sizing, start an async search instead. It runs in the background, collects up to 10,000 unique jobs, and tells you when it is done. 1. `POST /v1/searches` with your `filters`. You get `202` and a search `id` (`srch_...`). 2. Wait. Poll `GET /v1/searches/{id}`, or pass `webhook_url` and receive `search.completed`. 3. Read the results page by page with `GET /v1/searches/{id}?cursor=...`. Reading is free. ## Start a search The body takes the same `filters` and `waterfall` as [`POST /v1/jobs/search`](https://docs.betterjobs.cc/platform/filters.md). `limit` is the total number of unique jobs to collect: `1` to `10000`, default `1000`. **curl** ```bash curl https://api.betterjobs.cc/v1/searches \ -H "Authorization: Bearer $BETTERJOBS_API_KEY" \ -H "BetterJobs-Version: 2026-10-01" \ -H "Idempotency-Key: 7b8f2c4e-1a3d-4f5b-9c6e-0d2a4b6c8e1f" \ -H "Content-Type: application/json" \ -d '{ "filters": { "title_or": ["Data Engineer", "Analytics Engineer"], "country_code_or": ["DE", "FR", "NL", "ES", "PL"], "posted_within_days": 30 }, "waterfall": { "strategy": "max_coverage", "max_credits": 5000 }, "limit": 5000, "webhook_url": "https://hooks.northwind.example/betterjobs" }' ``` **Python** ```python import os import uuid import requests API = "https://api.betterjobs.cc/v1" HEADERS = { "Authorization": f"Bearer {os.environ['BETTERJOBS_API_KEY']}", "BetterJobs-Version": "2026-10-01", } resp = requests.post( f"{API}/searches", headers={**HEADERS, "Idempotency-Key": str(uuid.uuid4())}, json={ "filters": { "title_or": ["Data Engineer", "Analytics Engineer"], "country_code_or": ["DE", "FR", "NL", "ES", "PL"], "posted_within_days": 30, }, "waterfall": {"strategy": "max_coverage", "max_credits": 5000}, "limit": 5000, }, timeout=30, ) resp.raise_for_status() search_id = resp.json()["id"] # "srch_..." ``` **TypeScript** ```ts const API = "https://api.betterjobs.cc/v1"; const HEADERS = { Authorization: `Bearer ${process.env.BETTERJOBS_API_KEY}`, "BetterJobs-Version": "2026-10-01", }; const res = await fetch(`${API}/searches`, { method: "POST", headers: { ...HEADERS, "Content-Type": "application/json", "Idempotency-Key": crypto.randomUUID() }, body: JSON.stringify({ filters: { title_or: ["Data Engineer", "Analytics Engineer"], country_code_or: ["DE", "FR", "NL", "ES", "PL"], posted_within_days: 30, }, waterfall: { strategy: "max_coverage", max_credits: 5000 }, limit: 5000, }), }); if (res.status !== 202) throw new Error(`BetterJobs ${res.status}: ${await res.text()}`); const { id: searchId } = await res.json(); // "srch_..." ``` The `202` body is the search itself, with `status: queued`, `jobs_found: 0`, empty `data` and `metadata: null`. > Send an Idempotency-Key > > If the create call times out and you retry without a key, you can start the same search twice. Jobs are never charged twice, but you wait for two searches. Reuse one `Idempotency-Key` across retries of the same create. See [Idempotency](https://docs.betterjobs.cc/platform/idempotency.md). ## Status A search moves through six states. Three of them are final. | `status` | Meaning | What to do | | ----------- | -------------------------------------------------------- | ---------------------------------------------------------------------------------------------- | | `queued` | Accepted, not started yet. | Wait. | | `running` | Collecting jobs. `jobs_found` counts unique jobs so far. | Wait. Show `jobs_found` as progress. | | `on_hold` | Out of credits. Collection is paused. | Top up or upgrade. The search resumes on its own. | | `completed` | Finished. Every planned provider answered. | Read results. | | `partial` | Finished, but at least one provider failed or timed out. | Read results. `metadata.providers.failed` names the provider. You pay only for collected jobs. | | `failed` | Finished without results. | Check `metadata` if present, then start a new search. | `completed`, `partial` and `failed` are final. `data` holds results only once the status is `completed` or `partial`. It is empty in every other state. > Branch on status, not on the HTTP code > > `GET /v1/searches/{id}` returns `200` for every state, including `failed` and `on_hold`. A `200` does not mean the search worked. Read `status` and branch on it. Non-2xx codes only mean the read itself failed, for example `404 not_found` for an unknown id or `429 rate_limited`. ## Poll with a deadline Polling is free, but it counts against your [rate limit](https://docs.betterjobs.cc/platform/rate-limits.md). Start with a short interval, back off, and give up at a deadline you choose. Treat `on_hold` as a signal to a human, not as a reason to poll forever. **curl** ```bash curl https://api.betterjobs.cc/v1/searches/srch_2Vd9KqL4mN \ -H "Authorization: Bearer $BETTERJOBS_API_KEY" \ -H "BetterJobs-Version: 2026-10-01" ``` **Python** ```python import time FINAL = {"completed", "partial", "failed"} def wait_for_search(search_id: str, deadline_s: float = 900) -> dict: """Poll until the search reaches a final status. Raises on on_hold or deadline.""" deadline = time.monotonic() + deadline_s delay = 2.0 while True: resp = requests.get(f"{API}/searches/{search_id}", headers=HEADERS, timeout=30) if resp.status_code == 429: time.sleep(int(resp.headers["Retry-After"])) continue resp.raise_for_status() search = resp.json() if search["status"] in FINAL: return search if search["status"] == "on_hold": raise RuntimeError(f"{search_id} is on_hold: out of credits. Top up and it resumes.") if time.monotonic() + delay > deadline: raise TimeoutError(f"{search_id} still {search['status']} after {deadline_s}s") time.sleep(delay) delay = min(delay * 1.5, 30) def collect(search: dict) -> list[dict]: """Read every result page of a finished search. Free.""" jobs = list(search["data"]) cursor = search["next_cursor"] while cursor: resp = requests.get( f"{API}/searches/{search['id']}", headers=HEADERS, params={"cursor": cursor, "limit": 100}, timeout=30, ) resp.raise_for_status() page = resp.json() jobs.extend(page["data"]) cursor = page["next_cursor"] return jobs search = wait_for_search(search_id) if search["status"] == "failed": raise RuntimeError(f"Search {search_id} failed") if search["status"] == "partial": print("Missing providers:", search["metadata"]["providers"]["failed"]) jobs = collect(search) ``` **TypeScript** ```ts const FINAL = new Set(["completed", "partial", "failed"]); const sleep = (ms: number) => new Promise((r) => setTimeout(r, ms)); /** Poll until the search reaches a final status. Throws on on_hold or deadline. */ async function waitForSearch(searchId: string, deadlineMs = 900_000) { const deadline = Date.now() + deadlineMs; let delay = 2_000; while (true) { const res = await fetch(`${API}/searches/${searchId}`, { headers: HEADERS }); if (res.status === 429) { await sleep(Number(res.headers.get("Retry-After")) * 1000); continue; } if (!res.ok) throw new Error(`BetterJobs ${res.status}: ${await res.text()}`); const search = await res.json(); if (FINAL.has(search.status)) return search; if (search.status === "on_hold") throw new Error(`${searchId} is on_hold: out of credits. Top up and it resumes.`); if (Date.now() + delay > deadline) throw new Error(`${searchId} still ${search.status} at deadline`); await sleep(delay); delay = Math.min(delay * 1.5, 30_000); } } /** Read every result page of a finished search. Free. */ async function collect(search: { id: string; data: unknown[]; next_cursor: string | null }) { const jobs = [...search.data]; let cursor = search.next_cursor; while (cursor) { const url = `${API}/searches/${search.id}?${new URLSearchParams({ cursor, limit: "100" })}`; const res = await fetch(url, { headers: HEADERS }); if (!res.ok) throw new Error(`BetterJobs ${res.status}: ${await res.text()}`); const page = await res.json(); jobs.push(...page.data); cursor = page.next_cursor; } return jobs; } const search = await waitForSearch(searchId); if (search.status === "failed") throw new Error(`Search ${searchId} failed`); if (search.status === "partial") console.warn("Missing providers:", search.metadata.providers.failed); const jobs = await collect(search); ``` The Python and TypeScript tabs reuse `API` and `HEADERS` from the create example above. ## Or get a webhook Pass `webhook_url` when you create the search. When the search reaches a final status, BetterJobs POSTs a `search.completed` event to that URL. `watch_id` is `null`, because no watch produced it. ```json { "id": "evt_8Re5TyU1oP", "type": "search.completed", "created_at": "2026-10-11T09:06:45Z", "watch_id": null, "data": { "search": { "id": "srch_2Vd9KqL4mN", "status": "completed", "jobs_found": 3184 } } } ``` The event fires for `completed`, `partial` and `failed`. Branch on `data.search.status`, then read results with `GET /v1/searches/{id}` as in `collect()` above. Verify the signature first. See [Webhooks](https://docs.betterjobs.cc/platform/webhooks.md#verify-the-signature). Use both if you can: the webhook for speed, and a slow poll as a safety net in case a delivery is missed. ## What it costs - Jobs are charged as the search collects them: 1 credit per unique job you have not paid for before. Duplicates and already-paid jobs are free. - Reading results with `GET /v1/searches/{id}` is free, however many times you read them. - `waterfall.max_credits` caps the whole search. Set it to the most you are willing to spend. - If credits run out, the search moves to `on_hold` instead of failing. It resumes after a top-up. `POST /v1/searches` has no `dry_run`, and unknown body fields return `400 invalid_request`. To size a search before you start, send the same `filters` and `waterfall` to `POST /v1/jobs/search` with `"limit": 100` and `"dry_run": true`. It is free. `expected_unique_jobs_range` counts every matching job, across all pages; `credits_range` prices one page of 100 only. Expect the async search to cost at most the smallest of `expected_unique_jobs_range.max`, your `limit` and `waterfall.max_credits`. `max_credits` is the only hard bound. See [Credits and billing](https://docs.betterjobs.cc/concepts/credits-and-billing.md). ## Related - [Pagination](https://docs.betterjobs.cc/platform/pagination.md) - [Webhooks](https://docs.betterjobs.cc/platform/webhooks.md) - [Errors and partial results](https://docs.betterjobs.cc/platform/errors.md#partial-results) - [Create an async search API reference](/api/operations/createsearch/) and [Get an async search](/api/operations/getsearch/)