Skip to main content

Export annotations

Once your team has labelled data, you'll want to get the annotations out of AfriAnnotate into a downstream training pipeline, BI tool, or archival storage.

Four ways to export:

MethodBest forTrigger
UI snapshotOne-off downloads, ad-hoc reviewsProject → Export → Download
API exportProgrammatic / scheduled pullsPOST /api/projects/<id>/exports/
Target cloud storageContinuous push to a bucketAuto on annotation submit
WebhooksReal-time per-annotation pushSee Webhooks

UI snapshot — the quick path​

  1. Open the project
  2. Click Export (top right of Data Manager, or from the project dashboard menu)
  3. Pick a format (see below)
  4. Click Download

The download covers every task that has at least one annotation. There is no scope picker in the modal — to export a subset, filter in Data Manager and export each slice, pass ids[]= / download_all_tasks=true to the sync endpoint, or use the snapshot API's task_filter_options (below). Annotator anonymisation is a project-level setting (Settings → Security), not a per-download toggle — see Anonymise annotators.

The file lands as a .zip or .json / .csv etc. depending on the format.

Available formats​

FormatWhat's in itUse for
JSONFull task JSON — data + annotations + predictions, shape documented at Task formatRe-importing, custom downstream parsing
JSON_MINOne record per annotation: the task's data fields plus id, each control's result (prettified), annotator, annotation_id, created_at, updated_at, lead_time — no nested task / annotation metadataFlat downloads with labels next to the data
CSVFlat one-row-per-task with annotation columnsSpreadsheet review, BI tools
TSVSame as CSV with tab separatorsWhen commas in your data fight CSV
CoNLL2003NER spans in BIO schemeNamed-entity-recognition models that expect this format
COCOObject-detection JSON with bounding boxesYOLO / Faster R-CNN / DETR training
YOLO.txt per image with normalised bbox linesYOLO training pipelines
Pascal VOC.xml per imageOlder object-detection pipelines
Brush masks → PNGPixel masks as PNGSemantic segmentation training
ASR ManifestJSONL with {audio_filepath, text, duration}Whisper / NeMo / Coqui ASR training

Some formats only apply to specific tag types. The Export modal greys out formats incompatible with your project's labeling config.

Which format holds everything​

JSON is the complete export. Use it as the copy of record. The other formats are views of it for a particular use, and each leaves something out:

FormatHoldsLeaves out
JSONEvery result of every annotation, with every value; and for each annotation who made it and when, the time taken, whether it was skipped, whether it is ground truth, and its review (state, reviewer, comment, date)Nothing an annotator entered
JSON-MIN, CSV, TSVEvery value of every result of every kind of annotation, one column per control; the annotator, the dates and the time taken; and columns of their own for ground_truth, review_state, review_comment and, when there are any, relations (each end described by what it marks, since a flat row has no region ids)Skipped annotations: they have no row at all (a task whose every annotation was skipped is absent). Use JSON to see skips
COCO, YOLO, VOC, CoNLL, brush masks, ASR manifestThe one kind of annotation the format was made forEvery other control. The export dialog says which results would be left out before you download

This is checked, not assumed: an automated test runs a realistic annotation of every kind the editor can save (choices, taxonomy, free text, ratings, numbers, dates, text spans, relations, per-region text, HTML and dialogue spans, boxes, polygons, ellipses, keypoints, brush masks, audio segments with transcripts, time series, video timelines and tracked boxes, pairwise choices, rankings, voice, video and photo recordings, video tiers, knowledge-base references and typed features) through each general format and looks for every value in the output.

Calibration and gold answers. A gold answer is an annotation with ground_truth: true; an annotator's calibration attempts are their own annotations on the same task. Both are in the JSON export, so calibration can be recomputed from it. In the default JSON export a task says it is a gold task through that annotation; the task's own gold flag (is_honeypot), an annotation's dispute history, drafts and model predictions are in the export when you choose all fields in the preview.

Not in any export file: agreement scores (they are computed, see the project's Quality pages), speaker details (only as totals in the dataset card), consent records, and payment records. Each has its own page or download.

Preview before you download​

Preview (from the Export dialog, or the Shape & preview link) opens a full page that shows a sample of the tasks exactly as they will appear in the file and lets you shape that file. The page has its own Export button that downloads exactly what it shows; the Export dialog itself downloads the default shape (all annotations, complete fields).

  • Scope — the sample follows the same rules as the export (annotated tasks, or all tasks when you choose that option). Opening Export from a Data Manager tab scopes both preview and download to that tab's filters and selection.
  • Annotations per task — All (default), First, Latest, or Accepted only (accepted / fixed / ground-truth annotations; tasks without one are dropped and the count says how many).
  • Fields — by default the file carries the dataset shape: the task's id, data, metadata and timestamps, and for each annotation its results plus who annotated, when, the review outcome and timing. Platform bookkeeping (skip flags, comment counters, dispute timestamps, internal ids, prediction counts) is left out. Choose fields… opens a picker grouped into task, data-column, annotation, result-entry and platform bookkeeping fields (searchable, per-group select-all) — include the bookkeeping group, or switch off anything you don't want. Result entries always keep type, value, from_name and to_name. Field selection applies to JSON, JSON-MIN, CSV and TSV; converter formats such as COCO, YOLO and CoNLL define their own fields.
  • Linked view — each sampled task is rendered read-only on the left with its shaped record on the right; click a region to highlight its result block, or a block to select the region. The sample is stable for a project and can be reshuffled.

The same controls are available to the API: annotation_choice= all|first|latest|accepted, fields=<comma-separated dotted paths> (for example id,data.text,annotations.result,annotations.result.id; omit it for the dataset default, or fields=all for everything the serializer produces) and view=<data-manager-view-id> on GET /api/projects/<id>/export; GET /api/projects/<id>/export/preview returns the sample with the same parameters plus limit, offset and seed.

See the sample through the chosen format​

Pick any format other than JSON in the preview and the record pane switches to a format lens: the task on screen is run through the real converter, so you see

  • what survives — "Through COCO: 3 of 3 results survive", with a warning pill for every result type the format cannot carry ("1 relation result won't appear in JSON_MIN"); those results are dimmed in the record and tagged not in <format>,
  • what the file looks like — the converter's actual output for that task (the CSV row, the CoNLL lines, the COCO or YOLO fragments, the ASR manifest entry), one block per output file, so there are no surprises after the download.

Formats the converter cannot produce for the project's labeling configuration are not offered in the format list at all. The lens runs only on the task currently on screen, so it is instant and costs nothing for large projects.

Large exports and timeouts​

The in-UI Export → Download button runs synchronously — the whole export is built and streamed back inside a single HTTP request (GET /api/projects/<id>/export?exportType=...). That's fine for small and medium projects, but on very large ones (roughly 1000+ tasks, more so with media resources or a license-bundled .zip) the request can outlive the reverse proxy's request budget and come back as a 408 / 502 / 504.

The fix in AfriAnnotate is the async snapshot export via the API or SDK, documented in Export with the API or SDK below: you trigger a snapshot (POST .../exports), poll it until it reports completed, then download the finished file. Because the snapshot is built by a background worker rather than the request thread, there is no synchronous timeout to hit — the download step just fetches an already-materialised file.

So: reach for the Download button for quick, small pulls; switch to the snapshot API/SDK the moment a project is large enough that the synchronous download stalls.

Export with the API or SDK​

Programmatic export — useful for scheduled pulls, CI, or integration with external pipelines:

# Trigger an export (async — returns a snapshot ID)
SNAPSHOT_ID=$(curl -X POST "{{platformURL}}/api/projects/<id>/exports/" \
  -H "Authorization: Bearer YOUR_PAT" \
  -H "Content-Type: application/json" \
  -d '{"title": "nightly", "task_filter_options": {"annotated": "only"}}' | jq -r .id)

# Poll until the snapshot is ready
while true; do
  STATUS=$(curl "{{platformURL}}/api/projects/<id>/exports/$SNAPSHOT_ID" \
    -H "Authorization: Bearer YOUR_PAT" | jq -r .status)
  if [ "$STATUS" = "completed" ]; then break; fi
  echo "Status: $STATUS"
  sleep 5
done

# Download the snapshot (JSON by default)
curl "{{platformURL}}/api/projects/<id>/exports/$SNAPSHOT_ID/download" \
  -H "Authorization: Bearer YOUR_PAT" \
  -o annotations.json

Snapshots are always JSON. For another format, POST .../exports/<snapshot>/convert with {"export_type": "CSV"}, then download with ?exportType=CSV. task_filter_options accepts view (a Data Manager tab ID), skipped, finished, annotated (only / exclude); annotation_filter_options and serialization_options are also available.

Snapshots are kept until you delete them (DELETE /api/projects/<id>/exports/<snapshot>). An org can set a retention policy (retention.export_files_max_age_days, default 0 = keep forever), which is applied only when the operator schedules the cleanup_exports management command. The list of snapshots for a project is at GET /api/projects/<id>/exports/.

Snapshots vs live export​

Two flavours:

  • Snapshots (/exports) — point-in-time copies. Tasks + annotations frozen at the moment of snapshot creation.
  • Live export (/api/projects/<id>/tasks/?fields=annotations) — current state of the database. Useful when you want the latest data but don't need a versioned artifact.

For ML training pipelines, snapshots are usually right — your training run wants to know exactly what data was in the dataset at the time of training, not "whatever's in the database now."

For BI dashboards, live export is right — you want the latest counts.

Recorded media (AudioRecord / VideoRecord)​

Clips captured in-browser by <AudioRecord> and <VideoRecord> can be exported two ways.

When a project's labeling config contains a recorder tag, the Export dialog shows a Download ZIP with media card. One click produces a self-contained ZIP — no scripts, no URL fetching:

  • annotations.json — the export, with recording links rewritten to the bundled media/… files
  • media/ — every recorded clip and pose-skeleton sidecar
  • README.txt — a plain-language guide to the contents

The same bundle is available at GET /api/projects/<id>/export/media-bundle for automation.

Standard export (URLs only)​

The plain JSON export is not embedded — it carries authenticated URLs pointing at the platform's media storage:

"value": {
  "videorecord": {
    "url": "/data/upload/12/102d6bbc-recording.webm",
    "poseUrl": "/data/upload/12/9f3a11d2-pose.json",
    "duration": 7.05, "mimeType": "video/webm", "poseFps": 14.8
  }
}
  • url — the clip, at /data/upload/<project_id>/<file> (files live in MEDIA_ROOT/upload/<project_id>/, or your S3/GCS/Azure bucket when STORAGE_TYPE is configured).
  • poseUrl — VideoRecord only, when skeleton="pose" was on: the per-frame keypoints sidecar JSON.

Opening a clip URL directly in a logged-in browser plays it in the browser's native player (streamed, with seeking). To assemble a local dataset, download each URL with an authenticated request (session cookie or API token), e.g. curl -H "Authorization: Token <token>" <host><url> -o clip.webm.

Recordings saved before August 2026 stored an unresolvable id-shaped URL; repair them once with python manage.py fix_record_upload_urls --apply.

Target cloud storage​

A project can be configured to continuously push new annotations to an external bucket. Useful when the same data flows into many downstream consumers — instead of each consumer pulling from AfriAnnotate's API, the bucket is the single source of truth.

Configure at Project → Settings → Cloud Storage → Target:

  1. Pick Storage type: S3 / GCS / Azure / Redis (or Local files on self-hosted deployments)
  2. Provide credentials (access key / service account / connection string)
  3. Provide Bucket name + Prefix (path inside the bucket)
  4. Pick Format — JSON snapshot of each annotation as it's submitted

When an annotator submits, AfriAnnotate writes the annotation JSON to <bucket>/<prefix>/<annotation-id> (no extension). Updates overwrite; deletes remove the object only if the connection's delete option is on. Set FUTURE_SAVE_TASK_TO_STORAGE=true to key by task instead (<prefix>/<task-id>.json, one object per task).

See Cloud storage for the full credential setup.

Anonymise annotators in exports​

When the project-level Anonymise annotators in exports toggle is on (Project → Settings → Security, see Data security), every annotation in every export and snapshot from that project replaces completed_by and updated_by with a pseudonym — a positive 31-bit integer derived from SHA-256 over the server SECRET_KEY, the user ID and the project ID. Expanded user objects come back with empty email / name fields.

Useful when:

  • Downstream consumers shouldn't see real annotator identities (privacy / contractor-NDA / academic-IRB)
  • You need to ship the dataset publicly but keep the labour records internal

Properties:

  • Deterministic per project — the same annotator gets the same pseudonym in every export from that project, so rows stay grouped. The same person gets a different pseudonym in a different project.
  • Irreversible — no mapping is stored; the real identity cannot be looked up from the export, by you or by AfriAnnotate. Rotating SECRET_KEY changes every pseudonym.
  • Project-wide — there is no per-export override.

Export the dataset license​

When a project has a license set (see Licensing), every export is delivered as a .zip archive containing the data file plus a human-readable LICENSE.txt: SPDX identifier, name, summary, the constraint flags (commercial use, derivatives, share-alike, …), the canonical URL, and the source project / organisation and when the license was chosen. Projects without a license keep the plain single-file download. The exported JSON itself is unchanged — there is no embedded manifest block.

The license file travels with the data, so downstream consumers always know what license applies.

Dataset cards​

Whenever an export produces a .zip archive (any multi-file format, or a single-file format once a license makes it an archive), a DATA_CARD.md is generated automatically and included alongside the data. It summarizes:

  • the purpose: the goal, target, languages and planned date set at setup (the plan), first, as a datasheet opens;
  • the project's license and its constraint flags;
  • which consent template(s) govern the data, distinct-signer counts and, per purpose, how many signers accepted it, with what leaving a standard purpose unaccepted means (left out of model training; left out of anything handed on outside the organisation). The media bundle's README.txt carries the same statement in one paragraph;
  • aggregate speaker demographics for contributors who opted in to sharing (aggregates only — never per-person data);
  • the audio-QC checks and thresholds that ran on every recording;
  • the collection method (read-aloud vs spontaneous prompts);
  • clip counts, total hours, and the collection date range.

Sections with no underlying data are omitted, never fabricated. Single-file exports (a bare .json / .csv with no license set) stay single-file and carry no card. Pushes to Hugging Face render the same card as the dataset's README.md — see Hugging Face Hub.

Performance​

Snapshots run in background workers — they don't block on the user's browser. Typical durations:

  • ≤ 10k tasks: < 30 seconds
  • 10k–100k tasks: 1–5 minutes
  • 100k–1M tasks: 5–30 minutes (dominated by media-URL signing for cloud-storage-connected projects)

For very large datasets (> 1M tasks), prefer the target cloud storage continuous-push path over snapshots — the snapshot path roundtrips through Django.

What's next​