Back to Home
Recruiter API Data Delivery API
v1 · REST · JSON

Data Delivery API

Pull the results of your data-collection jobs straight into your pipeline: every accepted set with its media files, SHA-256 checksums, participant metadata and signed consents. Incremental, resumable and verifiable — no archives to wait for.

Quick start Ready-made sync script

How delivery works

Participants record and upload their sets on Wadata. Every file is probed on arrival against the media rules of your job, each participant signs the data release themselves, and two independent reviewers must agree before a set is accepted. Only accepted sets are delivered through this API.

1

Collected

Participants upload one file per slot; resolution, frame rate and duration are checked automatically.

2

Consented

Each participant signs the release for your job; a snapshot of the text is stored with the signature.

3

Accepted

Reviewers reach consensus. The set becomes downloadable and participants are paid.

4

Delivered

You poll for new accepted sets, download the files and verify each SHA-256.

Base URL: https://wadata.africa/api/v1/partner — all paths below are relative to it. Responses are JSON (UTF-8); timestamps are ISO 8601 in UTC.

Authentication

Create an API key in your Wadata account under Data API (/client/api). The key is shown once — store it in a secrets manager. Send it on every request:

http — wadata 
Authorization: Bearer wadata_client_3f9c…
Accept: application/json
  • Keys carry the collections:read scope: they can read your own collections and nothing else.
  • Revoke a key at any time from the same page; requests with it then return 401.
  • Use one key per integration (e.g. production ingest, QA sampling) so they can be rotated independently.

Quick start

Three requests take you from a key to verified files on disk.

Terminal 
export WADATA_KEY=wadata_client_…

# 1. Your collections
curl -H "Authorization: Bearer $WADATA_KEY" https://wadata.africa/api/v1/partner/collections

# 2. Accepted sets of collection 25 (participants, consents, files + sha256)
curl -H "Authorization: Bearer $WADATA_KEY" \
  "https://wadata.africa/api/v1/partner/collections/25/sets?status=accepted&per_page=100"

# 3. Download a file (resumable) and verify it
curl -C - -O -J -H "Authorization: Bearer $WADATA_KEY" https://wadata.africa/api/v1/partner/files/108
sha256sum set_16_slot_01.mp4

Endpoints

Identify the key

GET /me

Returns the client account the key belongs to. Handy as a connectivity check.

200 OK 
{
  "client":  { "id": 1423, "name": "Roman Kutsev", "email": "crowd@unidata.pro" },
  "api_key": { "name": "Production ingest", "prefix": "wadata_client_3f9c", "scopes": ["collections:read"] }
}

List your collections

GET /collections

Every data-collection job owned by your account, newest first.

200 OK 
{
  "collections": [{
    "id": 25,
    "title": "Unidata — Webcam Pairs Dataset (Batch 3)",
    "status": "active",
    "price_per_set": "1300",
    "currency": "KES",
    "deadline": null,
    "created_at": "2026-09-13T14:28:19Z"
  }]
}

Collection details, quotas and progress

GET /collections/:id

The specification (slots, media rules, consent version), quota fill per demographic group, and how many sets are in each state.

200 OK (abridged) 
{
  "collection": {
    "id": 25, "title": "…", "status": "active",
    "spec": {
      "modality": "video",
      "participants_per_set": 2,
      "slots": [{ "index": 1, "key": "high_five_handshake", "title": "High-Five and Handshake", "description": "…" }, …],
      "media_rules": { "min_width": 1280, "min_height": 720, "min_fps": 20, "min_duration": 8, "max_duration": 30, "formats": ["mp4","mov","webm","matroska","avi"] },
      "participant_fields": ["age","gender","ethnicity"],
      "quota_field": "ethnicity",
      "consent_version": "1",
      "recruitment_status": "open"
    },
    "quotas": [
      { "group": "ethnicity", "value": "brown", "label": "Brown", "target": 150, "accepted": 1, "reserved": 0, "available": 149, "status": "open" },
      …
    ],
    "sets": { "accepted": 1, "in_review": 3, "in_progress": 12 }
  }
}

List sets (the main feed)

GET /collections/:id/sets

Paginated sets with participants, consent records and files. By default only accepted sets are returned, ordered by acceptance time (oldest first) so a page walk is stable while new sets arrive.

Query parameter Type Description
status string Set status to list. Default accepted. Other values (in_review, rework, rejected…) return metadata only — files are never served for them.
updated_since ISO 8601 Only sets changed after this instant, e.g. 2026-09-13T00:00:00Z. Use for incremental syncs.
page integer 1-based page number. Default 1.
per_page integer Items per page, 1–200. Default 50.
Request 
curl -H "Authorization: Bearer $WADATA_KEY" \
  "https://wadata.africa/api/v1/partner/collections/25/sets?status=accepted&page=1&per_page=100&updated_since=2026-09-13T00:00:00Z"
200 OK 
{
  "sets": [{
    "id": 16,
    "status": "accepted",
    "submitted_at": "2026-09-13T15:18:02Z",
    "accepted_at":  "2026-09-13T15:21:46Z",
    "updated_at":   "2026-09-13T15:21:46Z",
    "is_gold": false,
    "participants": [
      {
        "id": 27, "role": "A", "participant_ref": "P27",
        "age": 29, "gender": "female", "ethnicity": "black", "skin_tone": null,
        "quota_group": { "group": "ethnicity", "value": "black", "label": "Black" },
        "consent": { "id": 21, "version": "1", "signed_at": "2026-09-13T15:14:35Z",
                     "signer_name": "Dorine Otieno", "url": "https://wadata.africa/api/v1/partner/consents/21" }
      },
      { "id": 28, "role": "B", "participant_ref": "P28", "age": 31, "gender": "male", "ethnicity": "brown", … }
    ],
    "files": [
      {
        "id": 108, "slot_index": 1, "slot_key": "high_five_handshake",
        "filename": "set_16_slot_01.mp4", "content_type": "video/mp4", "size_bytes": 150997,
        "sha256": "33e49805c3da38c0ea9b719c7fd714a54c61286b156a93b3641230a4d0d74945",
        "media": { "width": 1280, "height": 720, "fps": 30.0, "duration": 10.0, "codec": "h264" },
        "check_status": "passed",
        "url": "https://wadata.africa/api/v1/partner/files/108"
      },
      … 14 more slots …
    ]
  }],
  "page": 1, "per_page": 100, "total": 1, "has_more": false
}
Set field Type Description
participants[].role string A is the set lead; the same people appear in every file of the set.
participants[].participant_ref string Stable pseudonymous identifier — use it in your dataset instead of names.
participants[].quota_group object The demographic group the participant counted towards.
files[].slot_index / slot_key int / string Which scripted scenario the file is; matches spec.slots.
files[].sha256 hex string SHA-256 of the file body. Verify every download against it.
files[].media object Probe results at upload (dimensions, fps, duration, codec).
files[].url URL Download endpoint for the file (see GET /files/:id).
is_gold boolean Set is a gold-standard reference used to calibrate reviewers.

One set

GET /collections/:id/sets/:set_id

The same shape as one entry of the feed, wrapped in { "set": … }. Returns 404 if the set is not part of the collection.

Download a media file

GET /files/:id

Streams the file body. Only files of accepted sets in your own collections are served; anything else is 404.

Response header Type Description
Content-Type e.g. video/mp4 As uploaded.
Content-Disposition attachment Stable name: set_<set id>_slot_<nn>.<ext>, e.g. set_16_slot_01.mp4.
ETag "<sha256>" The file's SHA-256, same as files[].sha256.
Accept-Ranges bytes Range requests are supported for resuming.
Content-Range bytes a-b/total On 206 responses.
Resume an interrupted download 
# curl resumes from the current size of the local file
curl -C - -O -J -H "Authorization: Bearer $WADATA_KEY" https://wadata.africa/api/v1/partner/files/108

# or ask for a byte range explicitly
curl -H "Authorization: Bearer $WADATA_KEY" -H "Range: bytes=100000-" https://wadata.africa/api/v1/partner/files/108 -o tail.bin
# → 206 Partial Content, Content-Range: bytes 100000-150996/150997

An unsatisfiable range returns 416 with Content-Range: bytes */total. Downloads are not rate-limited, but please keep to a handful of parallel streams.

Incremental sync

Sets keep arriving for the length of the collection window. The recommended loop:

  1. Remember the greatest updated_at you have processed (start with nothing).
  2. Request /collections/:id/sets?status=accepted&updated_since=<that value>, walking pages while has_more is true.
  3. For each set, store metadata.json, download files whose local SHA-256 does not match, then fetch consents.
  4. Repeat every 15–60 minutes. A set can be re-delivered if it is edited after acceptance (rare); the checksum tells you whether a file actually changed.

Integrity & privacy

Verify everything

Every file carries a SHA-256 computed on the server the moment it was uploaded. Compare it after each download (and again before you ship a dataset) — a mismatch means a truncated or altered transfer; simply download again.

Pseudonymous by default

Set and participant payloads contain only the demographic fields your job asked for and a stable participant_ref. Real names appear only inside consent records — store those separately from the dataset.

Accepted means reviewed

A file is downloadable only after the whole set passed automatic checks, both consents were signed and reviewers agreed. Rejected or reworked material never reaches the API.

Audited access

Key creation and revocation are logged, and every key shows its last use and request count on your Data API page, so unexpected activity is easy to spot.

Errors

Errors are JSON objects with a single error message.

Status Type Description
401 Unauthorized Missing, unknown, expired or revoked key.
404 Not found The collection, set, file or consent does not exist, is not yours, or (for files/consents) its set is not accepted.
400 Bad request Malformed parameter, e.g. updated_since is not ISO 8601.
416 Range not satisfiable Requested byte range is outside the file.
5xx Server error Retry with exponential back-off; nothing is lost.

Ready-made sync script

Python 3 with requests. Mirrors every accepted set into ./wadata_<collection>/set_<id>/ with its metadata, resumes partial downloads and verifies checksums. Safe to run repeatedly.

sync_wadata.py 
#!/usr/bin/env python3
"""Mirror accepted sets of a Wadata collection. Usage: WADATA_KEY=… ./sync_wadata.py 25"""
import hashlib, json, os, sys, requests

BASE = "https://wadata.africa/api/v1/partner"
KEY = os.environ["WADATA_KEY"]
JOB = int(sys.argv[1])
OUT = f"wadata_{JOB}"
H = {"Authorization": f"Bearer {KEY}"}
STATE = os.path.join(OUT, ".last_updated_at")


def sha256(path):
    h = hashlib.sha256()
    with open(path, "rb") as fh:
        for chunk in iter(lambda: fh.read(1 << 20), b""):
            h.update(chunk)
    return h.hexdigest()


def download(f, dest):
    if os.path.exists(dest) and sha256(dest) == f["sha256"]:
        return  # already have it
    start = os.path.getsize(dest) if os.path.exists(dest) else 0
    headers = dict(H)
    if start:
        headers["Range"] = f"bytes={start}-"
    with requests.get(f["url"], headers=headers, stream=True, timeout=120) as r:
        r.raise_for_status()
        mode = "ab" if r.status_code == 206 else "wb"
        with open(dest, mode) as fh:
            for chunk in r.iter_content(1 << 20):
                fh.write(chunk)
    if sha256(dest) != f["sha256"]:
        os.remove(dest)
        raise RuntimeError(f"checksum mismatch for {dest}; will retry next run")


os.makedirs(OUT, exist_ok=True)
since = open(STATE).read().strip() if os.path.exists(STATE) else None
newest = since
page = 1
while True:
    params = {"status": "accepted", "page": page, "per_page": 100}
    if since:
        params["updated_since"] = since
    r = requests.get(f"{BASE}/collections/{JOB}/sets", headers=H, params=params, timeout=60)
    r.raise_for_status()
    body = r.json()
    for s in body["sets"]:
        d = os.path.join(OUT, f"set_{s['id']}")
        os.makedirs(d, exist_ok=True)
        with open(os.path.join(d, "metadata.json"), "w") as fh:
            json.dump(s, fh, indent=2)
        for f in s["files"]:
            download(f, os.path.join(d, f["filename"]))
        for p in s["participants"]:
            if p["consent"]:
                c = requests.get(p["consent"]["url"], headers=H, timeout=60).json()["consent"]
                with open(os.path.join(d, f"consent_{p['role']}.json"), "w") as fh:
                    json.dump(c, fh, indent=2)
        newest = max(newest or "", s["updated_at"])
        print("synced set", s["id"])
    if not body["has_more"]:
        break
    page += 1

if newest:
    with open(STATE, "w") as fh:
        fh.write(newest)

Need a hand integrating?

We are happy to review your ingest pipeline or add fields you need in the payload.

support@wadata.africa