Skip to main content

Recording sign and video clips

A collection task is a prompt plus a <VideoRecord> tag. The signer reads the prompt, records to camera, checks the playback, and submits. This page covers the behaviour; the attribute table lives on the <VideoRecord> tag page.

Prompts​

The prompt is whatever object the recorder points at with toName:

  • Written text — <Text name="prompt" value="$text"/>: a word, phrase, sentence, or question. Used by Sign this prompt and Prompted question.
  • A picture — <Image name="prompt" value="$image"/>: for signers of any spoken-language literacy level. Used by Sign a picture.
  • Anything else that renders — a <Header> instruction, a short reference video, a <Text> question for a spontaneous answer.

Import one task per prompt; the CSV/JSON column must match the template's $variable (text or image in the shipped templates).

The capture flow​

  1. Record. The browser asks for camera and microphone (audio is captured by default — mouthing and other non-manual signal lives in it). A denied permission or a missing camera shows a plain-language message instead of a raw error.
  2. Warm-up. The live preview mounts first and recording starts only once the camera is actually delivering frames (up to a four-second wait), so clips never begin with black "getting ready" seconds. The live preview is mirrored like a selfie; the recorded clip is not.
  3. Timer. Elapsed time counts against the cap (maxDuration, default 300 s; the shipped sign templates set 30 s). Recording stops automatically at the cap; Stop ends it earlier.
  4. Playback + metrics. The take plays back inline with native controls, and a small panel shows duration, the negotiated resolution and frame rate (read from the camera track — the browser may give less than the requested 1280×720@30), the container format, and — with tracking on — how many skeleton frames were captured at what rate.
  5. Re-record or submit. Re-record discards the take (the annotation result is removed, not kept alongside), and returns to step 1. A tag holds one clip; there is no multi-take history.

hotkey starts and stops recording from the keyboard.

Constraints you can set​

WhatAttributeBehaviour
Clip lengthmaxDurationHard cap; auto-stop. Keep elicited items short (one sign or phrase per clip).
CamerafacingModeuser (front — the default, right for signing) or environment (rear).
Capture size / ratewidth, height, framerateRequested values; the device negotiates down.
EncodingmimeType, bitsPerSecondDefault WebM at 2.5 Mbps; codec picked per browser (VP9/Opus → VP8/Opus → WebM → MP4 on Safari).
Live trackingskeleton, skeletonFpspose, hands, face, combinations, or all. Draws a live skeleton over the preview (framing feedback) and saves per-frame keypoints as a sidecar. Lower skeletonFps for weak phones.

There is no minimum-duration check, no silence gate, and no live signal QC — those are audio measurements that <AudioRecord> has and this tag does not. Framing guidance (head to waist, even front lighting, plain background, hands in frame) is given in the template descriptions and, with tracking on, by the live skeleton itself.

Tracking is strictly additive: a device that cannot run the model (no WebGL, assets blocked) records a plain clip and shows a quiet note.

What gets stored​

On Stop the clip uploads through the standard import API into project media storage at upload/<project_id>/<uuid>-recording.webm (served authenticated at /data/upload/<project_id>/<file>). The annotation result is one videorecord entry whose from_name and to_name are both the tag:

"value": { "videorecord": {
  "url": "/data/upload/12/102d6bbc-recording.webm",
  "duration": 7.05, "mimeType": "video/webm",
  "width": 1280, "height": 720, "frameRate": 30,
  "poseUrl": "/data/upload/12/9f3a11d2-pose.json", "poseFps": 14.8
} }

poseUrl / poseFps appear only when tracking produced frames; the sidecar format is on the tag page. Reopening a saved annotation replays the clip with its skeleton overlaid.

Exports carry the URLs; the Download ZIP with media export bundles the files themselves (Export).

Offline and flaky connections​

The recording is durable the moment it exists:

  • If the device is offline, or the upload fails mid-flight, the clip (and its pose sidecar) is written to on-device storage and the signer is told it is saved on this device and will upload automatically. The annotation can still be submitted; the result is patched with the real URL when the upload lands.
  • A background flusher retries on reconnect, on a timer, and on page load, with exponential back-off; large queued clips go up in chunks so one dropped packet does not restart the whole file.
  • A persistent note shows how many clips are still waiting across the project.

Mobile​

The mobile app runs the same recorder inside its web view — there is no separate native camera flow. What differs:

  • Permissions are requested by the OS the first time a <VideoRecord> task opens (camera and microphone; nothing is asked at install).
  • Front camera is selected by facingMode="user".
  • The project must have Allow on mobile enabled (Security settings); the Listen page honours the same gate.
  • Offline capture uses the same on-device queue; the app's outbox replays the submission itself (Mobile architecture).

Labeling Setup preview​

In the config preview the button reads Test camera / Test tracking: it opens the camera and draws the live skeleton so you can tune skeleton and framing, but nothing is recorded, uploaded, or saved.