feat(people): confidence-aware suggestion scores (LCB) #1070

Merged
chris merged 2 commits from feat/merge-suggestion-confidence-score into main 2026-10-07 22:41:59 +02:00
Owner

Problem

Winner's Curse at small-N centroids. A merge suggestion (and a picker ranking) compares two estimated unit centroids: the mean of a person's face embeddings, L2-normalized. A centroid over N embeddings still carries measurement scatter ~σ/√N, so a pair of two small clusters can look far more similar than it is — the classic winner's curse of picking the minimum of many noisy estimates. Confirmed empirically: 4/4 pairs admitted by the raw 0.35 gate during 06.–07.10. were owner-rejected, all with min N ≤ 6. σ̂ = 0.139 comes from a leave-one-out residual analysis of centroid means inside large face sets (N ≥ 8, n = 2689, 2026-10), rounded up to 0.14 and used as a conservative upper bound (the residuals include genuine intra-person variance, not only measurement noise).

Formula

d_pess = d + Z · σ · sqrt(1/n1 + 1/n2)
score  = clamp(1 - d_pess, 0, 1)
  • d — cosine distance between the two unit centroids (shared embedding space only).
  • Z = 1.645 — one-sided 95% normal quantile.
  • σ = 0.14 — leave-one-out centroid residual scatter (see Problem).
  • n1/n2 — the vector counts actually averaged into each centroid (bucket vector counts, including pending-fallback seeds) — never the SQL face count.

Constants and helpers are shared by every consumer: ASSIGNMENT_DISTANCE_SIGMA, ASSIGNMENT_DISTANCE_Z, confidenceMargin(n1, n2), lcbScore(d_pess) in ClassInstanceDiscoveryService.

Scope

  1. mergeSuggestion (gate + ranking) — pessimistic distance only:
    • gate: skip pairs with d_pess > 0.35 (pairs at exactly 0.35 stay in);
    • ranking: d_pess ASC → combinedFaceCount DESC → firstId;
    • MergeSuggestionCandidate.rawDistance / .pessimisticDistance, MergeSuggestionView.score.
  2. personRanking — PersonRankingEntry.score, sorted score DESC (nulls last); n_source = candidate face embeddings in the dominant model pair, n_target = the person's shared-space embeddings.
  3. mergeTargetRankings — MergeTargetRankingEntry.score, sorted score DESC (nulls last); source side = the source instance bucket vectors.
  4. suggestSimilarInstances — InstanceSuggestionResult.score, ranked by d_pess with n_source = 1 (a single source face; pending fallback rows too); per-instance row selection still picks the raw-distance minimum (minimal change); name-fallback rows keep score = null.
  5. No hard gates in the three picker endpoints — explicit product decision: the lists stay complete, so small-person growth stays possible; they only rank honestly against larger ones.

User-visible behavior changes

  • The merge card can be empty — the gate now skips borderline pairs that the raw distance admitted. Intended (those were exactly the 4/4 false positives).
  • Picker chips now read {NN}% Score instead of a bare {NN}% — merge pickers (merge-picker-similarity, merge-target-picker-similarity), the move / tentative-assign pickers (*-similarity), and the instance-suggestion labels in the media panel / edit dialog. The raw similarity moved into a title tooltip; testids unchanged.
  • Picker row order is now the score order, which differs from raw-similarity order whenever the two sides have different bucket sizes (the API tests pin exactly such a flip).
  • Non-comparable rows (embedding-less / cross-space / name fallback) still carry similarity = null and score = null, still listed, still sorted last.

Tests

  • API rtk ./gradlew :apps:api:test — BUILD SUCCESSFUL, 1622 tests, 0 failures, 0 errors, 9 skipped across 142 test classes; ClassInstanceDiscoveryServiceTest 265 tests, 0 failures, DiscoveryControllerTest 73 tests, 0 failures.
    • New per endpoint: small-vs-big flip (better raw similarity ranks below a better-known centroid), score < similarity for finite N, null-comparable stays null and last, and n_source = 1 correctness for suggestSimilarInstances; plus score parity between personRanking and mergeTargetRankings, and controller/record arity.
    • Existing merge-gate tests reworked in commit 1 (pessimistic gate/ranking, exact score formula, 4 new cases).
  • Web — svelte-check: 0 errors, 22 pre-existing warnings; vitest: 66/66 passed; Playwright: discovery-review 132/132 (desktop + mobile), full suite 832 passed, 0 failed (9 flaky, all passed on retry; 23 skipped).

Commits

  • 90a3b0d7 feat(people): confidence-aware merge suggestion score (LCB)
  • dd7eb9f4 feat(people): confidence-aware scoring for person ranking and assignment pickers
## Problem **Winner's Curse at small-N centroids.** A merge suggestion (and a picker ranking) compares two *estimated* unit centroids: the mean of a person's face embeddings, L2-normalized. A centroid over N embeddings still carries measurement scatter ~σ/√N, so a pair of two small clusters can look far more similar than it is — the classic winner's curse of picking the minimum of many noisy estimates. Confirmed empirically: **4/4 pairs admitted by the raw 0.35 gate during 06.–07.10. were owner-rejected**, all with min N ≤ 6. σ̂ = **0.139** comes from a **leave-one-out** residual analysis of centroid means inside large face sets (N ≥ 8, n = 2689, 2026-10), rounded up to **0.14** and used as a conservative upper bound (the residuals include genuine intra-person variance, not only measurement noise). ## Formula ``` d_pess = d + Z · σ · sqrt(1/n1 + 1/n2) score = clamp(1 - d_pess, 0, 1) ``` - `d` — cosine distance between the two unit centroids (shared embedding space only). - `Z = 1.645` — one-sided 95% normal quantile. - `σ = 0.14` — leave-one-out centroid residual scatter (see Problem). - `n1/n2` — the vector counts **actually averaged into each centroid** (bucket vector counts, including pending-fallback seeds) — *never* the SQL face count. Constants and helpers are shared by every consumer: `ASSIGNMENT_DISTANCE_SIGMA`, `ASSIGNMENT_DISTANCE_Z`, `confidenceMargin(n1, n2)`, `lcbScore(d_pess)` in `ClassInstanceDiscoveryService`. ## Scope 1. **`mergeSuggestion` (gate + ranking)** — pessimistic distance only: - gate: skip pairs with `d_pess > 0.35` (pairs at exactly 0.35 stay in); - ranking: `d_pess` ASC → `combinedFaceCount` DESC → `firstId`; - `MergeSuggestionCandidate.rawDistance` / `.pessimisticDistance`, `MergeSuggestionView.score`. 2. **`personRanking`** — `PersonRankingEntry.score`, sorted score DESC (nulls last); `n_source` = candidate face embeddings in the dominant model pair, `n_target` = the person's shared-space embeddings. 3. **`mergeTargetRankings`** — `MergeTargetRankingEntry.score`, sorted score DESC (nulls last); source side = the source instance bucket vectors. 4. **`suggestSimilarInstances`** — `InstanceSuggestionResult.score`, ranked by `d_pess` with **`n_source = 1`** (a single source face; pending fallback rows too); per-instance row selection still picks the raw-distance minimum (minimal change); name-fallback rows keep `score = null`. 5. **No hard gates in the three picker endpoints** — explicit product decision: the lists stay complete, so **small-person growth stays possible**; they only rank honestly against larger ones. ## User-visible behavior changes - The merge card **can be empty** — the gate now skips borderline pairs that the raw distance admitted. **Intended** (those were exactly the 4/4 false positives). - Picker chips now read **`{NN}% Score`** instead of a bare `{NN}%` — merge pickers (`merge-picker-similarity`, `merge-target-picker-similarity`), the move / tentative-assign pickers (`*-similarity`), and the instance-suggestion labels in the media panel / edit dialog. The **raw similarity moved into a `title` tooltip**; testids unchanged. - Picker row order is now the **score** order, which differs from raw-similarity order whenever the two sides have different bucket sizes (the API tests pin exactly such a flip). - Non-comparable rows (embedding-less / cross-space / name fallback) still carry `similarity = null` **and** `score = null`, still listed, still sorted last. ## Tests - **API** `rtk ./gradlew :apps:api:test` — **BUILD SUCCESSFUL**, `1622 tests, 0 failures, 0 errors, 9 skipped` across 142 test classes; `ClassInstanceDiscoveryServiceTest` **265 tests, 0 failures**, `DiscoveryControllerTest` **73 tests, 0 failures**. - New per endpoint: small-vs-big **flip** (better raw similarity ranks *below* a better-known centroid), `score < similarity` for finite N, null-comparable stays null **and** last, and `n_source = 1` correctness for `suggestSimilarInstances`; plus score parity between `personRanking` and `mergeTargetRankings`, and controller/record arity. - Existing merge-gate tests reworked in commit 1 (pessimistic gate/ranking, exact score formula, 4 new cases). - **Web** — `svelte-check`: **0 errors**, 22 pre-existing warnings; `vitest`: **66/66 passed**; Playwright: `discovery-review` **132/132 (desktop + mobile)**, full suite **832 passed, 0 failed** (9 flaky, all passed on retry; 23 skipped). ## Commits - `90a3b0d7` `feat(people): confidence-aware merge suggestion score (LCB)` - `dd7eb9f4` `feat(people): confidence-aware scoring for person ranking and assignment pickers`
feat(people): confidence-aware scoring for person ranking and assignment pickers
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 14m46s
dd7eb9f41c
Extend the merge-suggestion lower-confidence-bound (LCB) score to the three
picker endpoints, reusing the shared ASSIGNMENT_DISTANCE_SIGMA/Z constants via
new helpers confidenceMargin(n1, n2) and lcbScore(d_pess) — one formula, no
duplication.

- personRanking: PersonRankingEntry.score, ordered score DESC (nulls last,
  instance-id tiebreak); n_source = the candidate's face embeddings of the
  dominant model pair, n_target = the person's shared-space embeddings.
- mergeTargetRankings: MergeTargetRankingEntry.score, ordered score DESC
  (nulls last); source side = the source instance bucket vector counts.
- suggestSimilarInstances: InstanceSuggestionResult.score, ranked by the
  pessimistic distance with n_source = 1 (the single source face, pending
  fallback rows included); per-instance row selection stays on the raw
  distance (minimal change), name-fallback rows keep score null.

No gates: the three lists stay complete — ranking and display only, so small
persons stay selectable as merge targets and can grow.

Web: pickers render {NN}% Score (raw similarity moved into a title tooltip),
client row re-sort moves from similarity to score, and the instance-suggestion
labels in the media panel / edit dialog show the score too. testids unchanged.

Tests: per endpoint a small-vs-big flip (better raw similarity ranks below a
better-known centroid), score < similarity for finite N, null scores stay
last, n_source = 1 correctness for suggestSimilarInstances, plus score parity
between personRanking and mergeTargetRankings; Playwright mocks/assertions
updated to the Score chips.
chris merged commit bd2e54f59e into main 2026-10-07 22:41:59 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1070
No description provided.