Skip to main content

Annotation — Multi-label classification (Choices, multiple)

Ticks are recorded live

Work these steps on the in-app checklist — account menu → My testing checklist — where every Pass / Fail / Skip is recorded the moment you press it, and Fail files the report for you. This page (and its PDF) is the same list for reading, printing, or working offline; if you used paper, transfer your ticks into the app afterwards.

Project fixture: fixtures/projects/18-dialog-intent-labeling.json (20 tasks). Steps are the same shape for every data type so a tester can compare behaviour across types.

Each step: Do this, then check You should see.

Setup (manager does this once)​

  • AN-MULTI-01 Create a project from fixtures/projects/18-dialog-intent-labeling.json (scripts/seed.py --only 18 or Create project → paste the label config → Import the JSON). You should see: 20 tasks in the Data Manager; the labeling interface preview renders without red warnings.

  • AN-MULTI-02 Publish the project and add tester3 (annotator) and tester4 (reviewer). Set annotators per task = 2. You should see: the project appears for tester3 on Home → Continue working and on the Projects page.

Label​

  • AN-MULTI-03 As tester3 open the project and label the first task: tick two or more labels. You should see: the region/answer appears; Submit enables; Ctrl/Cmd+Enter submits and loads the next task.

  • AN-MULTI-04 Label 2 more tasks, using hotkeys shown in the tooltips where possible. You should see: each submit moves on; the counter in the bottom bar advances.

  • AN-MULTI-05 Press Undo (Ctrl/Cmd+Z) after an action, then Redo. You should see: the exact last action reverts and re-applies.

  • AN-MULTI-06 Skip a task. You should see: the next task loads; the skipped one is marked skipped in the Data Manager.

  • AN-MULTI-07 Open a task you already submitted and change something, then Update. You should see: the change is saved; the history panel shows the update.

Review round-trip​

  • AN-MULTI-08 As tester4 open the review stream for this project. You should see: tester3's annotations appear one at a time with Accept / Fix & accept / Reject.

  • AN-MULTI-09 Click Fix & accept on one. You should see: the editor opens THAT task with THAT annotation (same text/media, regions visible); saving returns you to the stream.

  • AN-MULTI-10 Reject one with a comment, then as tester3 open My work → rejected. You should see: the rejection and comment are visible; 'request re-review' is offered.

Export​

  • AN-MULTI-11 As manager: Export → Preview. Switch the format between JSON, CSV. You should see: the record pane shows the task exactly as the file will; non-JSON formats show the file output and mark any results that will not survive.

  • AN-MULTI-12 Choose fields… and untick all bookkeeping fields, Apply, then Export. You should see: the downloaded file has only the chosen fields; the count in the button matches the Data Manager view.

  • AN-MULTI-13 Untick a label and submit. You should see: the record shows exactly the ticked set.

Mobile / small screen​

  • AN-MULTI-14 Repeat one labeling task at phone width (or in the phone app if the project allows mobile). You should see: the interface is usable without horizontal scrolling; controls are at least finger-sized.