Reputation profiles
When someone requests to join your project from Discover, they may be a complete stranger — no shared org, no history with you. A reputation profile is their portable track record across every organization on the platform: how much of their work has been validated, how often reviewers accepted it, and how they scored on ground-truth evaluations. It exists so a join request doesn't have to be a leap of faith — in either direction.
Consent first
Sharing is off by default and entirely the contributor's choice, in line with the platform's other sharing switches (speaker-profile export consent, leaderboard opt-in):
- Opt-in — Profile → Analytics → Reputation → Share reputation. Until it's on, organization managers see nothing — not even that a record exists.
- Org-name masking — a second, independent switch. With it off (the default), viewers see "3 organizations", never which ones — who you work with can itself be sensitive.
- Revocable instantly — nothing is cached or stored; the profile is computed from the underlying records at the moment it's requested, so turning sharing off hides it immediately.
- Preview what others see — your own Analytics → Reputation tab shows the profile exactly as a manager would see it, masked names and all.
Only organization managers (owner, admin, or manager role in any organization) can view a shared profile — the people who review join requests. Fellow annotators cannot browse each other's records.
What's included — and why you can trust it
Everything in the profile is computed live from validation events that already exist on the platform. Nothing is self-reported.
| Signal | Source |
|---|---|
| Validated units per kind | The append-only ledger — credit lands at validation only, never at submission |
| Acceptance rate | Reviewer verdicts on their annotations. Fix & Accept counts as accepted — the reviewer kept the work |
| Evaluation pass rate | Ground-truth evaluation snapshots, each judged against the threshold in force at the time |
| Projects, organizations, languages | The breadth of projects holding their validated work; languages come from each project's declared annotator-facing language |
| Active since / last active | First and most recent validated event — not first login |
Volume floors keep rates honest. A rate is only quoted once there is enough evidence behind it (20 reviews for acceptance, 10 scored evaluations for pass rate). Below the floor the profile says insufficient data and shows the raw counts — three accepted reviews is not "100% acceptance".
What's deliberately excluded
- Rate-change exits. Leaving a project because its pay rate changed is a neutral fact about the project's rates, not evidence about the person. Exits never appear in the profile, positively or negatively.
- Disputes and removals. Same reasoning — one-sided events with context the platform can't adjudicate don't belong in a portable score.
- Unvalidated submissions. Pending work proves nothing yet, and counting it would reward volume over quality.
- Anything self-written. No bio, no skill checkboxes, no claimed languages — only what validated activity demonstrates.
Where it appears
- Your own profile — Analytics → Reputation, always visible to you, with a status pill in the title row saying whether it's currently shared.
- Join-request review — on a project's Members → Contributors tab, pending requests from opted-in contributors carry a compact line like "1,240 validated units · 96% acceptance · 4 orgs", linking to the full profile.