Skip to main content

Management commands

Operator-facing CLI commands for housekeeping, backfills, and manually triggering scheduled jobs. Run via manage.py (or docker exec ... manage.py if your platform runs in a container).

Most operators interact with these through their orchestrator (Cloud Run jobs, cron, Kubernetes CronJob) rather than running them by hand — but it's worth knowing what's available for debugging + one-off operations.

Quality + agreement​

run_auto_suspend_rules​

Manually trigger an evaluation pass of every active auto-suspend rule across all orgs. Normally runs as a 24-hour scheduled job; run it directly if you want to validate rule behaviour or kick a delayed run.

python manage.py run_auto_suspend_rules

Flags:

  • --org <N> — limit to a specific org
  • --dry-run — report which members would be warned / suspended without writing or notifying. Useful when rolling out a new rule
  • --no-reschedule — skip the "enqueue next run" step (testing)

backfill_agreement​

Recompute Task.precomputed_agreement for tasks that don't have a score yet (or whose score is stale after a metric change).

python manage.py backfill_agreement [--project <N>] [--dry-run]

Omit --project to scan every project.

Useful after:

  • Changing a project's agreement metric
  • Importing historical annotations that didn't trigger live computation
  • A bug-fix to an agreement-metrics function (rescore everything)

detect_label_drift​

Scan projects for per-annotator label-distribution drift — an annotator whose recent label mix has diverged from their own long-run baseline — useful for catching cases where someone's interpretation of "POSITIVE" drifted over months without anyone noticing.

python manage.py detect_label_drift [--project <N>] [--dry-run]

Omit --project to scan every project. Flagged annotators become review items in the project (no file output); the command prints a per-project summary and a final projects_scanned=… annotators_flagged=… line to stdout. --dry-run scans and reports without creating review items.

Housekeeping​

expire_pending_invites​

Hard-delete invite records that are older than the org's invite- expiry policy + grace window. Usually runs as a 24-hour job; run by hand if your invite table is bloated.

python manage.py expire_pending_invites [--days 14] [--dry-run]

send_inactivity_reminders​

Send email reminders to org members who haven't logged in for N days. Usually runs as a 24-hour job; run by hand for testing.

python manage.py send_inactivity_reminders [--days 14] [--limit 500] [--dry-run]

Dry-run shows who would receive a reminder without sending.

send_notification_digests​

Send periodic notification digest emails to users who opted in from Profile → Preferences (off / daily / weekly, default off). Each digest summarises what happened since the user's last one — review outcomes on their annotations, contribution-goal progress, newly required consents, and new public projects on Discover — and is only sent when there's something to report.

python manage.py send_notification_digests [--frequency daily|weekly] [--dry-run]

Schedule it once a day (cron, Cloud Run job, Kubernetes CronJob); each user's own frequency preference decides whether they are due on a given run. Idempotent — re-running the same day sends nothing, and content never repeats across digests.

The command also flushes stale review verdicts before building digests: any annotation reviewed more than an hour ago whose annotator was never told (the live rollup only fires when a reviewer's queue drains) gets the standard "your work has been reviewed" notification, so a rejection can never sit unannounced in a half-finished queue (last_digest_sent_at is stamped per user after each send). The "digest" notification kind respects the platform-wide email kill-switch in Notification Settings.

Flags:

  • --frequency daily|weekly — restrict a run to one cadence (useful if you want separate schedules); default processes both
  • --dry-run — report who would get a digest and which sections, without sending or stamping

cleanup_exports​

Delete export snapshots older than each org's Delete export snapshots after (days) policy (retention.export_files_max_age_days; 0 / unset = keep forever). Removes the Export row, its snapshot file, and any converted-format files. The policy applies whether or not it is marked Enforced.

python manage.py cleanup_exports [--dry-run]

Schedule daily alongside the other maintenance commands. --dry-run lists what would be deleted without deleting.

measure_waiting_files​

Measures the media files of projects where the Data Manager's Measure files action could not finish in one go (it notes the project and leaves the rest to this command). A project is done, and dropped from the list, once nothing in it is left to measure. Part of maintenance.

python manage.py measure_waiting_files [--minutes 20]

purge_expired_project_data​

Retention sweep for projects whose effective retention window is > 0 days (per-project data_retention_days, else the org's retention.project_data_max_age_days). For each such project it computes the deletion date from the oldest task's created_at (project created_at if it has no tasks), sends escalating owner warnings (in-app + email) at 30 / 7 / 1 days before — each once — and, once the deadline passes, deletes tasks, annotations, drafts, predictions, uploaded media files, and export snapshots while keeping the Project row as an empty shell. Bookkeeping lives in Project.retention_state (warnings_sent, purged_at); an already-purged project with no newer data is skipped.

python manage.py purge_expired_project_data [--dry-run]

Schedule daily. --dry-run reports what would be warned / purged without changing anything. See Org policies.

cleanup_orphan_hf_tasks​

Delete tasks that were imported from Hugging Face Hub (Task.meta.hf_dataset_id set) but whose project's HF source binding is gone or now points at a different dataset — leftovers from a cleared or swapped binding that can no longer push annotations back. Annotations, drafts, and predictions cascade with the tasks.

python manage.py cleanup_orphan_hf_tasks [--project <N>] [--dry-run]

One-off; run after noticing stranded HF tasks in the Data Manager. Omit --project to scan every project. See Hugging Face Hub.

prune_legacy_tokens​

Blacklist accumulated legacy refresh tokens (minted before desktop / mobile tokens carried a device claim) so a user's Personal Access Token list shows at most one. Per user it keeps every token with a device claim plus the single newest unclaimed token; every other unclaimed token is blacklisted.

python manage.py prune_legacy_tokens                 # dry run, all users
python manage.py prune_legacy_tokens --apply
python manage.py prune_legacy_tokens --user someone@example.com --apply

Dry-run by default; --apply writes. One-time cleanup after upgrading past the device-claim change.

annotations_fill_updated_by​

One-time migration helper that backfills Annotation.updated_by for historical annotations imported before that field existed. Idempotent. Useful when upgrading from a very old version.

python manage.py annotations_fill_updated_by

calculate_stats / calculate_stats_all_orgs​

Recompute task / annotation aggregate stats stored on Project + Organization records. Normally maintained live by signals; the command is for fixing drift after a manual database edit or migration.

python manage.py calculate_stats <organization_id>
python manage.py calculate_stats_all_orgs [--from-scratch] [--redis]

Backfills + repairs​

backfill_clip_durations​

Fill Annotation.duration_seconds for speech clips recorded before the audio-QC dispatcher started storing durations at ingest (the field feeds contribution hours + goals). Only touches annotations whose result carries an audio URL and that have no duration yet, so it is safe to re-run and resumable. Durations are read header-only via soundfile; without it, WAV files fall back to a RIFF header parse and other formats are left null for a later run.

python manage.py backfill_clip_durations [--project <N>] [--batch-size 200] \
  [--limit <N>] [--start-id <id>] [--dry-run]

Flags: --project (one project id), --batch-size (annotations per batch, default 200), --limit (stop after N candidates), --start-id (resume after this annotation id), --dry-run (report, write nothing). One-off; run once after upgrading, or again if a run was interrupted. See Goals and progress.

fix_record_upload_urls​

Repair AudioRecord / VideoRecord results (and drafts) saved with the old id-shaped URL /data/upload/<id>, which the path-based media route cannot serve — so replay and export failed. Rewrites each to the real /data/upload/<project>/<uuid>-….webm path by joining the id against FileUpload.

python manage.py fix_record_upload_urls                  # dry-run report
python manage.py fix_record_upload_urls --apply          # write changes
python manage.py fix_record_upload_urls --project 12 --apply

Dry-run by default; --apply writes; --project scopes to one project. Run once on deployments that hold recordings made before the fix. See AudioRecord.

Deployment + infrastructure​

locked_migrate​

Wrapper around migrate that takes an advisory database lock before running. Use on multi-instance deployments (Cloud Run with min-instances > 0, Kubernetes Deployments with replicas > 1) where two pods could otherwise race to apply migrations.

python manage.py locked_migrate

Behaves identically to migrate if no other instance is running; otherwise waits for the lock holder to finish. Mandatory on Cloud Run with --max-instances > 1.

show_async_migrations​

List in-flight async migrations + their progress. Some migrations run as background workers (tracked in the AsyncMigrationStatus table); this command surfaces them.

python manage.py show_async_migrations

Useful when migrate finishes instantly but the platform's still catching up on a big backfill behind the scenes.

send_test_email​

Send a test email to verify SMTP wiring without going through the signup / forgot-password flow.

python manage.py send_test_email you@example.com

Reads the standard EMAIL_HOST / EMAIL_HOST_USER / EMAIL_HOST_PASSWORD env vars (see Email delivery) and sends a fixed-template message. If you don't receive it, your SMTP isn't wired correctly — the first diagnostic when "the platform doesn't send mail."

Org lifecycle​

destroy_organization​

Hard-purge an org + everything under it (projects, tasks, annotations, member rows). Bypasses the 30-day retention window.

python manage.py destroy_organization <organization_id>

Interactive: prints the org id + title and asks Are you sure? yes/no: — only a literal yes proceeds (so it can't be run unattended by accident). Use for testing deployments where seed orgs need cleaning up, or for compliance-mandated immediate-purge requests. It does not write an audit event of its own.

seed_sample_projects​

Bootstrap a fresh platform with ~34 demo projects spanning text / NLP, image, document, video, audio, and time-series interfaces.

python manage.py seed_sample_projects [--org "<title>"] [--owner <email>] \
  [--tasks-per-project 12] [--reset]

--org is the organisation title (default AfriAnnotate Testing, created if missing); --owner must already exist (run seed_testers first). --reset deletes same-titled demo projects before recreating. Uses synthetic data from a small bundled corpus; nothing leaves the deployment. Useful for demo platforms + giving operators a starting point to explore the UI.

seed_testers​

Pre-create a fixed set of role-spread tester accounts (owner, admin, managers, reviewers, annotators) in a target org.

# Password mode (default): shared known password, printed at the end
python manage.py seed_testers [--org "<title>"] [--password <pw>] [--reset]

# Invite mode: inactive accounts, each emailed a set-password link
python manage.py seed_testers --send-invites --emails a@x.org,b@x.org \
  [--base-url https://label.example.org]

Flags: --org (title, default AfriAnnotate Testing), --password, --reset (re-stamp passwords / resend invites), --send-invites, --emails (real addresses replace the default @afri.test ones in role order), --base-url, --quiet. Useful for demo deployments, integration tests, or practising the onboarding flow.

seed_agreement_demo​

Seed one demo project with simulated multi-annotator annotations at a known agreement level, for exercising the Quality settings + Metrics tab. Creates Demo · Agreement Sandbox (<method>) in the target org; idempotent on that title.

python manage.py seed_agreement_demo [--org "<title>"] [--owner <email>] \
  [--tasks 40] [--annotators 4] [--agreement 0.75] [--method cohens_kappa] \
  [--metric <key>] [--seed 42] [--reset]

--org (title, default AfriAnnotate Testing) and --owner (default owner@afri.test) must already exist — run seed_testers first. --agreement is the probability (0..1) each simulated annotator copies the gold answer; --method sets the project's agreement_method; --metric optionally pins an agreement_metric key (blank = auto per control); --seed makes the run reproducible; --reset deletes the existing demo project first.

seed_translation_demo​

Seed a small machine-translation demo project (Demo · Machine Translation (EN→YO): Text source + TextArea target), a handful of tasks that deliberately overlap the seeded Translation Memory (so fuzzy suggestions fire), some TM segments, and a small glossary. Idempotent on the title.

python manage.py seed_translation_demo [--org "<title>"] [--owner <email>] [--reset]

Org title and owner email must already exist (the defaults are a developer test org / account — pass your own). --reset deletes and rebuilds.

seed_tm_test_corpus​

Seed larger pre-translated corpora for smoke-testing TM accuracy (fuzzy matching, glossary lookup, near-duplicate resolution). One project per pair — TM Test · English → Hausa / Yoruba / Swahili — each task pre-filled with an initial annotation (post-edit workflow), the same pairs loaded into the TM, plus a glossary and intentional near-duplicates.

python manage.py seed_tm_test_corpus [--org "<title>"] [--owner <email>] \
  [--pair en-ha|en-yo|en-sw] [--count <N>] [--reset]

--pair empty = all three; --count caps tasks per project (0 = full curated corpus); --reset rebuilds. Org / owner defaults are a developer test org / account — pass your own.

Sync (desktop)​

afri_sync_pull​

Manually trigger a pull from the cloud into the local SQLite mirror on a desktop install. Only useful on the desktop runtime; no-op on the cloud Django.

python manage.py afri_sync_pull [--cloud-url <url>] [--token <jwt>] [--full] [--note "<text>"]

--full ignores the delta anchor and re-pulls every task row.

Useful when a sync is stuck and you want to force-restart the pull loop. The same primitive runs as part of the desktop's scheduled sync.

afri_sync_push​

Symmetric to afri_sync_pull: push dirty local annotations + drafts from a desktop install up to the cloud. Same run_push the desktop's /api/afri/sync/push/ view calls in-process; the CLI form is for testing, cron, or draining an outbox when the desktop web bundle is broken.

python manage.py afri_sync_push [--cloud-url <url>] [--token <token>] \
  [--note "<text>"] [--project-pk <N>]

--cloud-url / --token override — and persist to — the stored CloudConfig; --note is written to SyncRun.note; --project-pk narrows the push to one local project pk (not the cloud id), default every dirty row.

Background jobs (RQ)​

Beyond explicit management commands, several jobs run through django-rq when Redis is reachable:

  • auto_suspend_rules_daily — the production auto-suspend runner; self-reschedules every 24 h from Django boot
  • Agreement precomputation — fires on annotation submit; computes per-task agreement and updates Task.precomputed_agreement
  • Webhook delivery — fires on subscribed events
  • Export snapshot — generates the requested format
  • Email send — notification emails are queued on the low queue

No Redis? Nothing is lost. The notification dispatcher pings Redis (1 s timeout) before enqueueing; if it's unreachable, or the enqueue fails, the email is sent inline, synchronously in the request. Exports and webhooks use the same start-async-or-sync helper. A Cloud Run / desktop deployment without Redis therefore works — at the cost of SMTP latency on the request path.

Monitor via RQ Dashboard (/django-rq/, staff-only).

How operators should use this page​

Three patterns:

  1. Wire into orchestration — set up Cloud Run jobs / Kubernetes CronJobs that run run_auto_suspend_rules, expire_pending_invites, send_inactivity_reminders, send_notification_digests, cleanup_exports, purge_expired_project_data, measure_waiting_files and clearsessions on a daily schedule (the maintenance command runs them all). The reference cloud uses Cloud Run jobs with Cloud Scheduler triggers; deploy/gcp/setup-maintenance.sh is the template.
  2. One-off backfills + repairs — when you import historical data or change a metric, run backfill_agreement; after upgrading, run backfill_clip_durations, fix_record_upload_urls --apply, and prune_legacy_tokens --apply once each as needed.
  3. Debugging — when "the auto-suspend rules don't seem to be firing," run run_auto_suspend_rules --dry-run to see what the system would have done.

What's NOT a management command​

  • Project / annotation creation — these are user actions; use the API
  • Bulk delete / archive — use the Data Manager UI or API; the CLI doesn't reach into user workspaces
  • Auth + session ops — use the user-facing flows; CLI doesn't have password-reset etc.

The CLI is for operator-level housekeeping, not application logic.

What's next​