Batch requests and jobs

Batch#

POST /v1/batch runs up to 100 GET requests in one call. Each item names a path and its parameters, exactly as you would send them alone:

1curl -X POST "https://api.socialscrape.dev/v1/batch" \2  -H "x-api-key: $SOCIALSCRAPE_API_KEY" \3  -H "content-type: application/json" \4  -H "Idempotency-Key: 6f1c2a9e-0b7d-4c55-9a39-2f4f0f6a1c01" \5  -d '{"requests":[6        {"path":"/v1/github/user","params":{"handle":"torvalds"}},7        {"path":"/v1/github/user","params":{"handle":"gaearon"}}8      ]}'

The answer has one full envelope per item, in order, in data.results, each with its own request_id, charge and status:

1{2  "ok": true,3  "credits_charged": 2,4  "data": {5    "object": "batch",6    "results": [7      { "path": "/v1/github/user", "status": 200, "response": { "ok": true, "credits_charged": 1, "data": { } } },8      { "path": "/v1/github/user", "status": 404, "response": { "ok": false, "error": { "code": "not_found" }, "credits_charged": 0 } }9    ]10  }11}

• The batch's credits_charged is the sum of its items'. Each item is priced and cached as if you had called it alone, so failures and cached repeats cost nothing.
• A failing item never fails the others. Check each item's ok.
• Items run concurrently on our side and each counts toward the requests in progress we ask you to stay below; see Rate limits.
• Credits are held up front, so concurrent items cannot overdraw the balance.

See Batch requests in the reference.

Retrying safely: Idempotency-Key#

Send an Idempotency-Key header on a POST you might retry (a network error leaves you not knowing if it ran). The same key with the same request returns the first answer again, with x-idempotent-replay: true, and charges nothing a second time. Keys are remembered for 24 hours.

• The same key with a different request answers 409 conflict.
• A retry that arrives while the first is still running answers a retryable 503; wait and retry.
• Server errors, 429 and 402 answers are not stored, so they are safe to retry with the same key after you fix the cause.

Use a fresh random value (a UUID) per logical operation.

Jobs: requests that answer later#

Most requests answer in a second or two. A request still running after about 25 seconds (AI transcription of a long video, a slow platform) answers 202 instead, with a job:

1{2  "ok": true,3  "credits_charged": 0,4  "credits_remaining": 5000,5  "cached": false,6  "cached_at": null,7  "request_id": "req_0123456789abcdef01234567",8  "data": {9    "object": "job",10    "job_id": "job_0123456789abcdef01234567",11    "status": "running",12    "poll_url": "https://api.socialscrape.dev/v1/jobs/job_0123456789abcdef01234567"13  }14}

Poll data.poll_url with your key. Polling is free. While the job runs you get the same 202; when it finishes you get the final envelope with the result and the charge, once. A failed job returns its error with credits_charged: 0. Job results are kept for 24 hours.

1import os2import time3import requests45headers = {"x-api-key": os.environ["SOCIALSCRAPE_API_KEY"]}6res = requests.get("https://api.socialscrape.dev/v1/transcript", headers=headers, params={"url": "https://www.youtube.com/watch?v=dQw4w9WgXcQ"}, timeout=60)78while res.status_code == 202:9    time.sleep(3)10    res = requests.get(res.json()["data"]["poll_url"], headers=headers, timeout=60)1112body = res.json()

Look up a job by id with Job result.