How sign-language annotation works

This page explains the ideas. It assumes you sign, or work with people who do, and that nobody has yet explained why corpus annotation is laid out the way it is. Once these make sense, Annotation covers the buttons and Templates covers the configs.

What you are looking at

Open a sign project and you get a row of tier tabs above a stack of tracks:

[ utterance ] [ rh ↳utterance ] [ lh ↳utterance ] [ mouthing ↳rh ] …  [ + row ]
─────────────────────────────────────────────────────────────────────────────
utterance    ▐──────────────────────────▌
rh              ▐─HOUSE─▌  ▐─BIG─▌
lh                        ▐─BIG─▌
mouthing        ▐─house─▌

• Each tab selects the tier you are currently working on. + row adds a segment to that tier at the playhead.
• The ↳ utterance under a tab is its parent. Segments on a child tier are held inside their parent's start and end — a mouthing cannot run past the sign it belongs to. Boundaries that break the rule are painted red and counted.
• The tracks below show every tier at once, so you can see how the channels line up in time.
• Empty tracks are normal on a fresh clip. Nothing is wrong; you have not segmented anything yet.

Because children live inside parents, segment the parent tier first. Marking rh before any utterance exists leaves those signs with nothing to attach to.

The one idea to get right: an ID-gloss is not a translation

A gloss is a written label for a sign. An ID-gloss is a label chosen so that every occurrence of the same sign gets the same label, everywhere in the corpus, forever.

That sounds obvious and is constantly got wrong, because the natural instinct is to write what the sign means here. Consider one sign used in three sentences:

Sentence	Natural translation	ID-gloss
"I'm going home"	home	HOUSE
"a big house"	house	HOUSE
"the family building"	building	HOUSE

If you write HOME, HOUSE, BUILDING, you have written three translations and destroyed the only thing the corpus was for: being able to ask "show me every time this sign is used". The ID-gloss stays HOUSE in all three. The meaning in context goes on the translation tier, where it belongs.

An ID-gloss is a name for the sign, not a description of it and not a translation of it. English words are used because they are convenient handles, not because the sign means the English word.

The test

Before you type a gloss, ask: if I saw this exact sign in a completely different sentence tomorrow, would I write the same thing? If not, you are writing a translation.

Why there are two hand tiers

Signing is two-handed, and the hands do not always do the same thing:

• One-handed signs — only the dominant hand moves.
• Two-handed symmetric — both hands do the same thing.
• Two-handed asymmetric — the dominant hand moves against a stationary non-dominant hand.
• Buoys — the non-dominant hand holds something (a list, a reference) while the dominant hand carries on signing something else entirely. The two hands are saying different things at the same time.

A single gloss track cannot express any of the last three honestly. That is why the template has rh and lh as separate tiers.

"rh" and "lh" mean dominant and non-dominant, not literally right and left. A left-handed signer's dominant hand is their left. Pick one convention for your corpus, write it down, and make sure every annotator follows it — mixing the two silently corrupts every two-handed sign in the corpus.

What each tier is for

Tier	What goes on it	Scope
utterance	The unit of meaning — roughly a sentence. Everything else lives inside one.	Segment these first
rh	ID-glosses for the dominant hand.	One segment per sign
lh	ID-glosses for the non-dominant hand, when it is doing something of its own.	Often sparse — that is correct
mouthing	Mouth movements borrowed from the spoken language.	Inside the sign it accompanies
brow	Eyebrow position. Raised brows mark questions and conditionals.	Spans a clause, not a sign
head	Head movements. A headshake marks negation.	Spans a clause, not a sign
translation	A natural sentence in the spoken language.	One per utterance

brow and head hang off utterance rather than off a sign on purpose. They are grammar, and their scope is the clause: a brow raise running across a whole question is the marker that makes it a question. Attaching it to one sign would misrepresent what it does.

Mouthing is not the same as mouth gesture

A mouthing silently articulates a spoken-language word. A mouth gesture is native to the sign language and has no spoken equivalent. They are different things; decide which your corpus records, and note the decision in your project instructions.

Do not try to fill every tier at once

No corpus project glosses on the first pass, and asking new annotators to do so is the most reliable way to produce a corpus nobody trusts. Work in rounds:

1. Utterances and translations. Where does each utterance start and end, and what does it mean? This is immediately useful on its own and it creates the parent segments everything else needs. The Utterances and translation (first pass) template is exactly this and nothing else.
2. Dominant-hand glosses. Add rh once a lexicon exists to gloss against.
3. Everything else — lh, mouthing, non-manuals — with trained annotators.

Each round builds on boundaries the last one established, so nothing is wasted and no one is asked to hold seven decisions in their head at once.

The lexicon, and what to do about a sign that is not in it

Consistency across annotators cannot survive free typing. A project can point at a lexicon — a knowledge base of ID-glosses — and then the gloss field autocompletes against it and stores each entry's stable id alongside the label. Two annotators picking the same entry produce the same identifier, which is what makes the corpus searchable by lemma. The id travels into .eaf exports as EXT_REF, so it survives into desktop tools.

A new corpus has no lexicon, and that is the normal starting state for a language that has never been documented this way. You build the lexicon and the corpus together: when you meet a sign that is not listed, type it and choose Add to the lexicon from the suggestions. The entry is created, marked provisional, and your segment is anchored to it immediately. A curator reviews provisional entries later.

If a sign is already in the lexicon, pick it rather than typing your own version. That single habit is most of what corpus consistency is.

Faces cannot be blurred

Brows, eye gaze, head movement and mouthing are part of the grammar. A signer's face carries meaning, so sign video cannot be anonymised the way audio can. Signers are identifiable in their data, permanently. Say this plainly when you take consent — see Consent.

Conventions worth agreeing before you start

Write these into your project instructions. Disagreements found halfway through a corpus are expensive.

• Glosses in UPPERCASE, hyphenated when multi-word: LOOK-AT.
• Pointing signs as INDEX-1 (to self), INDEX-2 (to addressee), INDEX-3 (to a third referent).
• Fingerspelling prefixed: FS:ABUJA.
• Gestures that are not lexical signs prefixed G:.
• Depicting or classifier constructions prefixed DS:.
• Boundaries — decide whether a sign begins when the hand starts moving toward its target or when it arrives, and apply it uniformly.
• Frame rate — you do not have to work this out. The player measures the clip as it plays and shows the rate beside the frame-step buttons; boundaries snap to that. If it disagrees with the project's setting the badge is marked — your work is still accurate, but tell whoever set the project up. When you are recording footage, 25 fps is a floor and 50 fps is much better for fingerspelling.

Common mistakes

Mistake	Why it hurts	Instead
Writing translations in the gloss field	Kills lemma search, the corpus's main purpose	Same sign, same gloss; meaning goes on translation
Inventing a new gloss for a listed sign	Splits one sign into several entries	Pick the existing lexicon entry
Glossing both hands on one tier	Cannot represent buoys or asymmetric signs	Use rh and lh
Attaching a brow raise to one sign	Misstates its grammatical scope	Put it on brow, spanning the clause
Mixing "dominant" and "right" between annotators	Corrupts every two-handed sign	Agree one convention, write it down
Glossing before utterances are segmented	Signs have no parent to attach to	Segment utterance first