Skip to main content

ML backend

A machine-learning backend is an external service that AfriAnnotate talks to over HTTPS to:

  • Pre-annotate new tasks (the model labels first; the annotator reviews + corrects)
  • Provide interactive predictions in the labeler (annotator draws a partial region; the model fills in the rest)
  • Train on submitted annotations and improve over time (active-learning loop)

The protocol is the same as upstream Label Studio's ML backend interface — point AfriAnnotate at any service that implements the MLBackend HTTP contract.

When to use one​

ScenarioUse ML backend?
You have a pretrained model you trust enough to bootstrap labelsYes — saves enormous amounts of annotator time
You're collecting labels from scratch with no prior modelNo (yet) — train after the first 1k labels and add the backend later
You want a foundation model (e.g. Whisper) to handle one piece of every taskYes — wire it up just for that piece
You want server-side audio QCYes — see Audio QC

Setup​

1. Run an ML backend service​

The service must speak HTTP and respond to these endpoints:

EndpointPurpose
POST /predictGiven task data, return a prediction in the same JSON format as a result entry
POST /train (optional)Receive a batch of annotations; train; return job status
GET /healthLiveness probe
POST /setupCalled on connect / validate with the project's label config and a callback token; return model metadata + version

Two ways to get a service:

  • Use upstream Label Studio's ML backend examples — clone heartexlabs/label-studio-ml-backend, pick a starter (Whisper, BERT, YOLO, Segment Anything, etc.), customise + deploy on your infrastructure
  • Write your own — implement the HTTP contract in any framework. The contract is documented in the upstream ML backend repo: HumanSignal/label-studio-ml-backend

2. Connect to your project​

Project → Settings → ML Backend.

  1. Click + Add ML backend
  2. Fill in:
    • Title — internal name
    • URL — the service's base URL (https://... or http://localhost:9090 during local dev)
    • Authentication — Basic auth, Bearer token, or none
    • Description (optional)
  3. Click Validate — AfriAnnotate pings /health
  4. (Optional) Tick Use for interactive preannotations
  5. (Optional) Tick Start model training on annotation submit
  6. Save

Once connected, the backend shows up in Settings → ML Backend with status (Connected / Down / Error) + last-checked timestamp.

Pre-annotation flow​

When a project has an ML backend with "Use for interactive preannotations" off (default) but the backend is configured:

  • A bulk-predict job runs after import — for each new task, the backend's /predict is called and the response is stored as a prediction on the task
  • Annotators opening the task see the prediction as draft regions they can accept / edit / discard
  • On submit, the annotation is the human's edited version; the prediction stays as-is for comparison

The bulk-predict job is async; watch progress at Settings → ML Backend → Predict status.

To re-run predictions (e.g. after deploying a new model version), click Retrieve predictions on the ML Backend settings tab.

Interactive predictions​

When "Use for interactive preannotations" is on, the backend is called live as the annotator works:

  • For image segmentation: the annotator clicks a point or scribbles a region; the backend (e.g. Segment Anything) fills in the precise mask
  • For text NER: the annotator highlights a phrase; the backend classifies the entity type
  • For audio transcription: the annotator opens the task; the backend (e.g. Whisper) auto-fills the transcript

Latency matters here — the labeler waits for the response. Aim for < 500 ms p95. If your backend is slower, prefer bulk pre- annotation over interactive.

Active learning​

Two levels:

  • "Start training on annotation submit" (legacy toggle) — every submitted annotation triggers a POST /train to the backend. Nothing else happens: predictions are not refreshed, so task ordering doesn't follow the retrained model.
  • The active-learning loop (Settings → Model → Active learning) — AfriAnnotate retrains every N annotations and then refreshes predictions for unlabeled tasks, so uncertainty sampling keeps ranking on the current model's scores. This is the one to use with Task sampling → Active learning; the full mechanics, requirements, and tuning live on the Active learning page.

Either way the backend can run an incremental step, queue a batched retrain, or just log — AfriAnnotate triggers training, it does not host it.

Multiple backends per project​

A project can have multiple ML backends. Useful when:

  • Different tags need different models (e.g. text classification + NER on the same task)
  • You're A/B-testing two model versions and want to capture both predictions

Each backend's predictions land as a separate entry in the task's predictions[] array, distinguished by model_version.

Outage alerts​

If a backend that was connected stops responding — or a training run fails — the project creator is emailed (at most once per day per backend). A backend that never connected doesn't alert: that's setup-in-progress, not an outage.

Network requirements​

The cloud Django needs outbound HTTPS access to the ML backend URL. If your backend is on a private network:

  • Use a VPN connector (Cloud Run's serverless VPC connector for the reference cloud)
  • Or expose the backend via a reverse proxy with allowlist
  • Or run the backend inside the same VPC as AfriAnnotate

For local development, run the backend on localhost:9090 and configure the project's backend URL as http://host.docker.internal:9090 (if cloud Django runs in Docker) or http://localhost:9090 otherwise.

Authentication​

Three modes:

  • None — backend's URL is the only secret. Fine for local dev.
  • Basic auth — username + password sent on every request.
  • Bearer token — Authorization: Bearer <token> header. Most common for production. Token rotates manually.

The credentials live encrypted in AfriAnnotate's database (with SECRET_KEY as the encryption key). Rotating SECRET_KEY invalidates all stored credentials.

Reliability​

The backend can fail without crashing the labeler:

  • Pre-annotation — failed predictions are logged; the task opens without a draft annotation, the annotator labels from scratch
  • Interactive — the labeler shows a tooltip "Predictions unavailable" and lets the annotator continue manually
  • Training-on-submit — annotation is still saved to AfriAnnotate's database; training failures are logged but don't roll back the annotation

Monitor backend health at Settings → ML Backend → Health, and optionally pipe the /health endpoint into your platform's monitoring (Cloud Monitoring / Datadog / etc.).

What's next​