Skip to main content

Rounds, criteria and sign-off

Before Round 1 starts (entry criteria)​

  • Every environment on the Environments tab is downloadable by the people who need it, and its build ID is recorded.
  • Every tester has their account, has signed in once on each surface they own, and is marked Onboarded on the Testers tab.
  • Every tester has read Terms, Running a step and Reporting a bug.
  • The fixtures load (tester2 has run one setup step successfully).
  • Assigned To has no blank cells and the Dashboard's unassigned counter reads zero.

Finishing Round 1 (exit criteria)​

The round closes when all three are true:

  1. No Blocker defect is open.
  2. At least 95% of steps are executed (Skipped and N/A do not count as executed).
  3. At least 90% of executed steps passed.

If the deadline arrives first, you may close the round anyway, but the Round Log must say which criteria were missed and why that was acceptable. That sentence is the most important thing the log will ever hold.

Round 2 and retests​

Never overwrite a Round 1 result. To retest a step:

  1. Copy its row to the bottom of Test Runs.
  2. Set Round to R2 and Status to Not Run.
  3. Put the original Run ID in Retest Of.
  4. Add an R2 row to the Round Log.

The Dashboard's round selector then shows either round alone, or both with ALL. Round 2 typically covers:

  • every failed or blocked step;
  • every step that touches a fixed area;
  • the full Smoke tier, as a regression check.

Known at the start of Round 1​

EnvironmentBuildWhat to know
ENV-WEBafri-cloud-00204-sg7Released 1 Sep with audio and video settings and the auth fixes. The most current build; check here before filing anything found on a phone.
ENV-AND88270403Predates the 2 Sep auth fixes. Sign-in failures on Android may already be fixed; record them, but note the build.
ENV-IOS88270486Same as Android: predates the auth fixes. TestFlight invitation needed.
ENV-MACnotarized 1 SepApple Silicon only. Intel Macs cannot run it; mark those steps Blocked with a note, not Fail.
ENV-WIN2 Sep buildUnsigned, so SmartScreen warns once. Fixes the local server failing to start. Click More info, then Run anyway.

The example defect row on the Defects tab is set to Not a Bug so it does not count as an open blocker. Delete it when the first real defect goes in.