Offline sync

The desktop app can pull a project's metadata + task data to a local SQLite mirror, letting annotators label offline. The afri_sync module handles the bidirectional sync: incremental pulls from the cloud, immediate push of annotation work when online, and a deliberate per-project review flow for anything that could collide. The mobile app has its own, JS-side equivalent (an IndexedDB mirror

• outbox in the SPA — see Mobile architecture); this page is about the desktop engine.

This is AfriAnnotate-specific (not in upstream Label Studio) — useful for field annotators in low-connectivity environments where "always online" isn't an option.

Architecture at a glance

Desktop and mobile can both label offline against their local mirror and sync back on reconnect — desktop via afri_sync (SQLite), mobile via the SPA's IndexedDB mirror (paged offline-pack download with incremental refresh + pruning, media prewarm under a 500 MB default LRU cap, and an ordered write outbox). The web SPA needs the cloud reachable to write.

Key modules

Module / file	Role
label_studio/afri_sync/sync.py	The sync engine — run_pull, run_push, preview_sync / apply_sync, eviction
label_studio/afri_sync/models.py	CloudConfig, SyncMeta, OfflineProjectPref, SyncRun
label_studio/afri_sync/signals.py	post_save / post_delete handlers that mark rows dirty + spawn the autopush
label_studio/afri_sync/apps.py	Spawns the autopull daemon on desktop installs
label_studio/afri_sync/cloud_proxy_middleware.py	Cloud-first writes + metadata-only read proxying + online-only write gate
label_studio/afri_sync/views.py + urls.py	The /api/afri/sync/... and /api/afri/auth/... endpoints the SPA calls
label_studio/afri_sync/management/commands/	afri_sync_pull / afri_sync_push for shell-driven syncs

SyncMeta — the shadow table

Sync state lives in shadow rows, not in extra columns on Label Studio's own models — that keeps the schema unforked and upstream merges clean. SyncMeta hangs off any model (Project, Task, Annotation, AnnotationDraft, …) via a generic foreign key on (content_type, object_id) and carries:

• cloud_id — the row's primary key on the cloud; NULL means the row was created locally and has never been pushed
• synced_at — last time the local row's state matched the cloud
• dirty — local changes awaiting push (set by save signals, cleared by the push worker)
• deleted_locally — tombstone; the next push sends a DELETE
• last_action / last_error — diagnostics (pull, push, push_match, push_conflict, …)
• last_remote_updated_at + last_remote_hash — the three-way-merge baseline: what the cloud row looked like at the last sync. Used to detect "cloud moved since we last saw it" (see conflict holding below).

The important consequence: any row without a SyncMeta cloud mapping is local-only — the sync loop and the cloud proxy leave it alone. That is the mechanism behind solo mode.

CloudConfig

Singleton row (one per device, id=1) telling the desktop where to sync to:

• cloud_url — base URL of the cloud Django (e.g. https://afriannotate.waraka.org)
• auth_token — the cloud refresh JWT, stored encrypted (Fernet, per-device key — see afri_sync/secret.py); plaintext never touches the DB column
• last_pull_at / last_push_at — the incremental-pull watermark: the next pull only fetches rows modified after last_pull_at
• force_reauth — set when the cloud rejects the stored JWT (password changed cloud-side, token revoked); forces the user back through online sign-in

The desktop shell (Tauri) sets cloud_url at first launch via /api/afri/sync/bootstrap/. There is no in-app "switch cloud" UI — per the model's design note, switching clouds means wiping the local DB and re-syncing.

OfflineProjectPref — the selective-sync record

Per-device, keyed by cloud_project_id (not local PK, so the preference survives a local DB rebuild). Two flags:

• enabled — the Download for offline toggle. When true, pulls mirror this project's tasks + annotations locally and pushes upload its local edits.
• manual_sync — the Manual mode toggle. When true, the project is excluded from autopush and the autopull task refresh; its changes only move when the user presses the per-project Push / Pull buttons in the Sync modal.

Selective sync — task data is opt-in

Mirroring is deliberately two-tier. A pull always mirrors the project rows the user can see (title, label config, policies), but task + annotation data only lands locally for projects the user has explicitly downloaded. The tier is computed per project (ProjectSerializer.get_local_sync_tier):

Tier	Meaning
full	Downloaded — labeling reads/writes go to the local mirror; works offline
metadata_only	Project row mirrored, no local task data — labeling endpoints are proxied to the cloud, so the project is usable online-only

The user-facing control is the cloud icon on each project card (ProjectTierBadge, desktop + mobile only):

Badge	State	Click
Green check-on-cloud	Downloaded for offline	Removes the local copy
Blue cloud-down arrow	Download for offline available	Starts the download
Muted cloud-slash	Online only (allow_offline_for_annotators off)	—
Muted lock	Cloud only (allow_metadata_mirror off)	—

The same toggles appear as a checklist in Profile → Cloud sync → Projects (backed by /api/afri/sync/available_projects/, which walks every cloud org the user belongs to).

Mechanics (set_offline_pref_view):

• Enabling starts a background run_project_download immediately — the toggle is the download button. Progress is a SyncRun row the SPA polls via /api/afri/sync/progress/.
• Disabling runs a self-healing eviction (evict_project_locally): the project's local tasks, annotations, drafts and their SyncMeta shadows are deleted in one sweep. The cloud copies stay — this is purely "stop keeping a local copy."
• Eviction is guarded: if the project still has unpushed (dirty) rows, the first call returns needs_confirm + a preview instead of evicting; the SPA asks the user before re-posting with force: true.
• Multi-device policies are enforced at download time: when allow_multi_device is off and the user already holds the project on another device (or allow_multi_user_per_device is off and another user has signed in on this device), the toggle returns a 409 needs_confirm and only proceeds with an explicit evict-elsewhere confirmation.

The sync loop

Pull is incremental ("delta pull"): task listings are filtered with ?updated_since=<last_pull_at> so steady-state pulls move only what changed. Pulls run:

• From the autopull daemon (apps.py) — ticks every 30 s, throttled to one pull per 5 min by default (tunable via the AFRI_AUTOPULL_SECS env var). Only spawned on desktop installs (a CloudConfig with a cloud_url).
• After sign-in / account-link (background "post-link" pull).
• On demand — the Sync now button and the Sync modal's per-project Pull.

Push is event-driven: post_save / post_delete signals on annotations + drafts mark the SyncMeta dirty and spawn a background run_push as soon as the transaction commits. Online, work lands on the cloud within seconds of submit; offline, rows stay dirty and the next push (auto or manual) drains them.

Projects in Manual mode are skipped by both automatic paths — their queues only move via the Sync modal.

Note the push scope: run_push moves annotations, drafts, and rate-change decisions (the accept/exit answers from the rate-change gate — see the next section). Projects, tasks and org structure flow cloud→local via pull (that's why task imports into cloud-mirrored projects are online-only — see the middleware section).

Deliberate sync + conflict holding

When the desktop is about to PATCH an annotation the user edited offline, it does not blindly overwrite the cloud ("deliberate sync", 2026-07-18). The push worker first re-reads the cloud row and compares it against the stored three-way baseline (last_remote_hash / last_remote_updated_at):

• Cloud unchanged since last sync → push proceeds.
• Cloud already has identical content (e.g. pushed from another device) → no-op, baseline refreshed (push_match).
• Cloud moved since last sync → the row is held as a conflict (last_action = 'push_conflict'), not pushed. Same for the sharpest case: local edit vs cloud-side delete.

Held rows are excluded from autopush and surface in the Sync modal with a ⚠ conflict count. The user resolves each one deliberately with a side-by-side Local/Cloud diff and a per-row choice — Local (ours), Cloud (theirs), Both, or Skip — then applies (/api/afri/sync/preview/ + /api/afri/sync/apply/).

New annotations never conflict — a first push is a POST that lands alongside whatever the cloud already has; Manager / Reviewer roles decide what to accept.

Money, policies & governance offline

The pull also mirrors the money/governance rows the desktop needs to behave exactly like the cloud — scoped to projects downloaded for offline, best-effort (an older cloud without the newer endpoints is skipped silently):

What works offline now

• Org policies are enforced. Every pull mirrors each org's OrgPolicy row (via /api/afri/cloudsync/org_policies/ — readable by any active member, unlike the admin-gated management endpoint), and the desktop's enforcement code reads local rows — so an enforced policy (private-only visibility, future-only rate changes, manager exclusions, …) does not lapse when the network does.
• The rate-change gate. Pending rate changes for the signed-in user are mirrored per downloaded project, so the blocking accept/exit modal appears offline too. The decision is recorded locally and pushed to the cloud's ack endpoint on the next push — where the exit side effects (membership disabled, owner notified) run. Replays are idempotent; a change the user decided on another surface is pruned from the local queue on the next pull.
• Earnings, read-only. The user's own ledger entries (units, stamped amounts, currency) are mirrored, with minimal stubs of the payout runs that already paid them — so the profile statement, including its paid/pending split and CSV download, renders honestly offline.
• Speech-clip durations (Annotation.duration_seconds) round-trip in both pull and push, so hour counts match everywhere.

What stays cloud-only (the desktop shows a "manage online" card that links out instead of an empty page)

• Payout management — creating runs, marking paid, receipt templates.
• Receipt downloads — the frozen breakdown and verification code exist only on the cloud; local stubs deliberately carry neither.
• Policy editing — mirrored and enforced locally, edited on the cloud.
• Reputation profile + sharing preferences (cross-org, cloud-computed).
• Notifications — written by cloud-side actions into the cloud DB; the desktop hides the bell rather than showing a permanently empty list.

Solo mode — local-only projects and accounts

"Download the app and use it yourself" is a supported mode, not a degraded one. Anything created on the desktop without a cloud peer (no SyncMeta cloud mapping) is the user's own solo workspace:

• Project creation is never blocked, online or offline.
• Local-only projects stay fully usable offline — import data, edit or delete tasks (Data Manager actions), change settings, label. The middleware's online-only gates apply only to cloud-mirrored projects (mirrored_only entries in _ONLINE_ONLY_WRITES + the Data Manager destructive-action check).
• Local-only projects are never uploaded — the push worker only pushes annotations/drafts on mirrored projects, and the account-link consent explicitly promises device-only projects stay on the device.
• Organizations that exist only on the device are badged Local in the sidebar org switcher, with a tooltip explaining the data isn't backed up.

Solo accounts (created by a device-only signup while offline) follow the same rule at the user level — see Cloud sync (Profile) for the user-facing signup + consent-first linking story. The /api/afri/auth/state/ endpoint reports local_only: true for a signed-in user with no cloud identity; the SPA hangs the "Link this account" card off that flag.

What genuinely needs the cloud is collaboration: org membership + invites, workspaces, ML backends, webhooks, cloud storages, and the Hugging Face hub are online-only writes — attempted offline they return 503 {code: "offline"}, which the SPA surfaces as the global offline modal instead of half-happening (e.g. a member row whose invite email could never be sent).

CloudProxyMiddleware

Desktop-only middleware in front of the embedded Django (localhost:18080). Three jobs:

1. Cloud-first writes. When online, annotation/draft mutations (and project settings PATCH / project DELETE on mirrored projects) are forwarded to the cloud first; the cloud's canonical response is mirrored into the local DB and handed back with IDs translated to the local namespace. Cloud stays the source of truth for anything shared. When the cloud is unreachable, annotation/draft writes fall through to the local view + the autopush outbox; project settings/deletes refuse with a clear online-only error.
2. Metadata-only reads. For projects not downloaded for offline (metadata_only tier), task reads, the label stream and the Data Manager grid are proxied to the cloud — the project is fully usable online without a local task mirror. Offline, these return a structured 503 the SPA recognises.
3. Online-only write gate. The collaboration endpoints listed above are refused while offline (never for local-only projects).

Cloud reachability is probed against the cloud's /version/ and cached for 30 s; manual "Sync now" invalidates the cache eagerly.

Debugging sync

When a user reports "my offline work disappeared" or "my project isn't syncing":

1. Profile → Cloud sync → Activity shows the recent SyncRun rows: direction (pull/push), rows processed/failed, a per-model breakdown (by_type), a note (auto, download, post-link, sync-modal-apply, …) and any error.
2. Profile → Cloud sync → Pending lists every unpushed local change with its state; a row held with "cloud copy changed since last sync — resolve in Sync review" is a conflict waiting in the Sync modal, not a failure.
3. Common error codes surfaced by the sync endpoints:
   • cloud_unreachable — network; the desktop keeps retrying, the work stays local.
   • cloud_auth_failed — the stored cloud token was rejected; CloudConfig.force_reauth is set and the user must sign in to the cloud again from the Cloud sync tab.
   • push_conflict (in SyncMeta.last_action) — held for deliberate resolution, see above.
4. For operator-level inspection there's a raw state dump at /api/afri/sync/debug/ on the device, and the afri_sync_pull / afri_sync_push management commands for shell-driven syncs.

Operator policies that affect sync

These are configured per-project (see Data security) and mirrored to the desktop on every pull so it can enforce them without a roundtrip:

• allow_offline_for_annotators — master switch for Download for offline; when off, the project is online-only (proxied) on the desktop, and managers can revoke it even after an annotator has the project locally
• allow_offline_for_reviewers — same gate for reviewers
• allow_metadata_mirror — strictest gate; when off the pull skips the project entirely, so it only appears on the device while the cloud is reachable
• allow_multi_device — if false, downloading on a second device requires confirming an evict of the first
• allow_multi_user_per_device — if false, only one annotator can hold this project on a given device

These are the levers operators have to control offline residency without changing per-user behaviour.

What's next

• Data security → — the per-project toggles that govern what gets synced where
• Cloud sync (Profile) → — the user-facing surface: device-only accounts, consent-first linking, the Sync modal
• Security model → — how the desktop's local cache is encrypted at rest
• Mobile architecture → — the mobile model (IndexedDB read mirror + write outbox in the SPA, no SQLite / afri_sync)