How the mobile app works

The mobile app is not a separate codebase that re-implements AfriAnnotate's features in Swift / Kotlin. It's a thin native wrapper around the same React SPA that powers the web at afriannotate.waraka.org.

This page explains the architecture — useful when you're trying to understand why a feature that ships on web "also" ships on mobile without any per-platform work, or why a mobile-specific bug almost always turns out to be a SPA or backend bug.

The big picture

• One backend — the same Django + PostgreSQL + cloud storage cluster handles every client. There's no per-platform API; mobile and web both call /api/projects/..., /api/annotations/..., and so on.
• One SPA bundle — the same React app webpacked from web/apps/labelstudio/. Mobile builds copy this exact bundle into the Capacitor wrapper.
• Different shells — iOS gets a WKWebView wrapped in a Swift app; Android gets a Chromium-derived WebView wrapped in a Kotlin app; web gets a regular browser. All three render the same SPA.

Why thin-client?

Three reasons:

1. Feature parity is free. A new screen in the SPA appears on web, iOS, and Android the next time you upload a build to each. No "we need to port this to iOS" work.
2. Mobile is a tiny fraction of users for an annotation platform. Most labelling happens on a laptop with a keyboard + a real mouse. The mobile app is for field annotators, on-call reviewers, etc. — a small audience that doesn't justify a parallel codebase.
3. Cloud-first writes match how teams already operate. Every annotation goes to the cloud, every export comes from the cloud. A native app with its own database would have to solve sync — instead the WebView talks directly to the cloud, and offline support is layered on inside the SPA (an IndexedDB read mirror + write outbox), not per-platform native code.

The result: mobile is cloud-first but genuinely offline-capable for annotation work. Reads come from an on-device mirror populated by "Download for offline" (plus a write-through cache of anything browsed online); writes queue in an IndexedDB outbox with idempotent, ordered replay, temporary-id rewriting for offline creates, and a dead-letter review UI for writes the server rejects — rejected work is preserved for the user to retry or discard, never silently dropped. Native connectivity events (@capacitor/network) make the offline switch instant. See Mobile install → What works offline for the user-facing guidance.

What's actually inside the .apk / .ipa

Building the Android .apk or iOS .ipa produces a roughly 10 MB file with these pieces:

Piece	What it is	Size
Native shell	Capacitor's iOS / Android wrapper (Swift / Kotlin)	~1 MB
Embedded WebView	The OS-provided WebView itself isn't bundled — it's shared with Safari (iOS) or Chrome (Android)	0 MB
Bundled SPA index.html	Boot HTML that checks the stored JWTs, routes the local SPA to its sign-in screen when signed out, and live-fetches feature flags	~5 KB
React SPA JS + CSS	The full labelstudio SPA bundle (webpacked, minified)	~8 MB
Icons + splash	App icon, launch screen images	~500 KB

The bundled SPA does NOT include a copy of Django or any backend logic. There is no embedded SQLite. Every write goes to the cloud over HTTPS (queued in IndexedDB while offline); reads come from the cloud when online and from the on-device mirror when not.

What the mobile app does on first launch

1. Native shell starts. iOS launches App.app; Android launches MainActivity. Both immediately create a WKWebView / WebView and tell it to load index.html from the bundle — no network needed for the app shell.
2. index.html runs the boot script. Reads the SPA's localStorage for an access JWT + refresh JWT and decodes each token's exp claim. If either is still live (or a session cookie exists), the user is treated as authenticated. Otherwise expired tokens are cleared and the script history.replaceStates the URL to the local /auth/login route — the WebView stays on the bundled page; nothing navigates to the cloud.
3. The bundled React SPA mounts. Signed out → React Router renders the local sign-in screen, which posts credentials to the cloud API (/api/token/obtain/) and stores fresh JWTs. Signed in → the SPA renders straight away; if the cloud is unreachable it serves reads from the IndexedDB mirror.
4. From this point onwards, every screen is rendered from the bundled JS/CSS; only API calls and media go to afriannotate.waraka.org (the API base URL is baked in at build time via CAP_API_URL). Feature flags are live-fetched from /api/feature-flags/public/ without blocking boot.

This is why a "fix a bug in the mobile app" task almost always means "fix a bug in the SPA" — the same bundle runs on web and in the apps; the apps just need a rebuild to pick it up.

The offline model — and its limits

Offline lives entirely in the SPA (web/apps/labelstudio/src/utils/offline-{fetch,outbox,sync,storage}.ts); there is no native database.

• Read mirror. "Download for offline" pulls a project's tasks, annotations and drafts into IndexedDB through the paged offline-pack endpoint (see below); anything browsed online is also cached write-through. Refresh is incremental and prunes tasks deleted on the server.
• Media. Task media (images, audio, video) is prewarmed into the on-device cache four files at a time and kept under a user-configurable cap (500 MB default, choices from 100 MB to unlimited) with least-recently-used eviction. Task / annotation / draft rows and the outbox are never evicted — only re-fetchable media is.
• Write outbox. Annotation / draft writes queue in IndexedDB and replay in order with idempotency UUIDs, temporary-id rewriting for offline creates, retry with backoff, and a reviewable dead-letter store for server-rejected writes (amber pill in the menubar) — nothing is silently dropped.
• Known limit: conflicts are last-writer-wins. The outbox does not merge concurrent edits: if the same annotation changed on another device while this one was offline, the later replay wins.
• Known limit: server-side features need the server. ML-backend predictions and other cloud-only operations wait until the device is back online.

Offline round-trips survive app restarts (the queue and mirror live in IndexedDB); the Offline storage page shows what's downloaded, how big it is, and what's queued, with per-project removal.

What's different across platforms

Three places the platforms diverge meaningfully:

1. Status bar handling. iOS uses viewport-fit=cover semantics + safe-area-inset-*; Android uses an opaque status-bar plugin. Capacitor abstracts most of this but the SPA still ships some per-platform CSS branches.
2. Hardware back button. Android has a hardware back button (or gesture); iOS doesn't. The SPA registers a back-button handler that pops history when running inside Capacitor on Android, and exits the app at the root. iOS uses swipe-from-left-edge for history back.
3. Distribution channel. APK side-load is Android-only; TestFlight is iOS-only. See the Distribution overview for the per-platform channels.

Where to look in the codebase

• Capacitor config — native/mobile/capacitor.config.ts — WebView allowlist, status-bar plugin config, scheme selection.
• Bundled boot HTML — native/mobile/spa-template/index.html — the boot script that checks stored JWTs, routes the local SPA to /auth/login when signed out, and live-fetches feature flags.
• iOS Xcode project — native/mobile/ios/App/ — generated by cap add ios; you edit Info.plist + signing settings here.
• Android Gradle project — native/mobile/android/ — generated by cap add android; you edit AndroidManifest.xml + build.gradle here.
• Build scripts — native/mobile/scripts/ — build-release.sh (Android), build-release-ios.sh (iOS), build-bundle.sh (the SPA-into-Capacitor step shared by both).

What's next

• Install the mobile app → — practical install instructions for end users
• Distribution overview → — channels for shipping new builds to testers / users
• Security model → — TLS, auth, data storage end-to-end

Offline downloads: the offline-pack endpoint (2026-08)

"Download for offline" pulls everything through GET /api/projects/<id>/offline-pack/ — pages of 100 tasks with their annotations and drafts embedded. A 166-task text project is two requests (measured: 5.6s end-to-end, down from ~65s when the app fetched annotations and drafts per task).

Refresh is incremental: the client stamps the server time of the last successful pull and sends it back as updated_after, so only tasks that changed come over the wire (measured: 2.4s when nothing changed). Page 1 carries all_ids, which the client uses to prune tasks deleted on the server. Media files are prewarmed into the service-worker cache four at a time and are never re-downloaded unless they changed.

Older servers without the endpoint fall back to the legacy per-task path automatically.

Passkeys in the native apps (2026-08)

WebAuthn cannot run inside the WebView (its origin is localhost, credentials are bound to the cloud domain), so the apps run the ceremony natively:

• iOS — ASAuthorizationPlatformPublicKeyCredentialProvider, trusted via the webcredentials: associated domain and /.well-known/apple-app-site-association.
• Android — Credential Manager, trusted via /.well-known/assetlinks.json; assertions arrive with an android:apk-key-hash: origin the server explicitly accepts (AFRI_ANDROID_APK_KEY_HASHES).

Sign-in needs no email: passkeys are discoverable, the account is resolved from the credential ID. The OS chooses the biometric (Face ID on iPhone; fingerprint/face per device settings on Android). If the ceremony fails, the login falls back to email + password.

Permissions and security posture

See native/mobile/PERMISSIONS.md for the full table. Summary: nothing but Internet at install; microphone and camera are requested by the OS the first time a speech or video annotation task uses them; release builds refuse cleartext HTTP; allowBackup is off so tokens never enter cloud backups; the SPA is bundled (no remote code).

Review: Fix & Accept provenance (2026-08)

A reviewer saving changes to someone else's annotation completes the fix server-side: the annotator's original result is snapshotted once (prefix_result), fixed_at is stamped, and the verdict is recorded automatically — no separate button press, no forgotten verdicts.

Everywhere the annotation appears, the fix is visible:

• Review stream — verdict chip plus an amber note ("Fixed by <reviewer>: 2 regions changed, 1 added") with before → after chips that include locations (character offsets, or m:ss for media). Verdicted cards offer Change verdict and Amend fix instead of the raw buttons.
• Editor — a toast explains the lock ("Accepted work is locked; changes go through the review stream") and reviewer-changed regions are highlighted on the canvas.
• Identical annotations cluster into one card; one verdict applies to every member (each annotator keeps their own review record and ledger credit).

Provenance and verdict fields are server-stamped only — they cannot be set or cleared through the annotation API.

Submission requirements

Project Settings → Annotation → Submission requirements controls whether an annotator can submit without labelling anything. "Require at least one region or answer" disables Submit until something is labelled; Skip stays available for genuinely empty tasks.

Connectivity states (2026-08)

The apps distinguish four states instead of one "Offline":

Pill	Meaning
red, wifi-off	No internet — the radio is off or has no route
red, cloud-off	Server unreachable — you have internet but the cloud isn't answering
amber gauge	Slow connection — everything works, expect delays
(none)	Online

The same states are shown before sign-in. While offline, the indicator also carries the count of annotations queued for sync. One shared probe drives every indicator (a single 12-second heartbeat for the whole app, battery-friendly).

Downloading for offline reports one continuous progress figure across the whole run and confirms completion with a summary of what was downloaded.

Exports everywhere (CSV/PDF) fetch with your credentials and open the device share sheet in the apps (save to Files/Drive, send via WhatsApp…), and export buttons are disabled until there is something to export.