feat(people-worker): confidence-aware target scoring for cluster_person_match #1071
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/worker-confidence-target-uncertainty"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
cluster_person_matchcards picked their suggested target by raw centroid distanced. That is a winner's-curse estimator: the target is the minimum over candidate identities, so noise pushing one competitor down is selected for — the winning raw distance systematically underestimates the true assignment distance, worst for small persons (both centroids are means over n faces; standard errorσ·√(1/n_source + 1/n_target)). Leave-one-out on the owner library (Okt 2026) measured per-face scatter σ = 0.139; all 4 rejected assignments in that LOO run had min n ≤ 6 and were raw-distance wins whose pessimistic distance would have lost.confidence = clamp(1 - distance)inherits the overstatement.Formel
Constants at the quality-score constants site:
TARGET_DISTANCE_SIGMA = 0.14(LOO σ = 0.139, Okt 2026, rounded up; same scale as APIASSIGNMENT_DISTANCE_SIGMA, PR #1070),TARGET_DISTANCE_Z = 1.645(one-sided 95 %, same level as the APIlcbScore).Changed emission sites (exactly 3):
clustering.pyconstrained identities loop (n_source = len(movable_vectors))_emit_group_candidatesB outcome, semisupervised v1 + v2 (n_source = len(unlabeled_offsets))main.pyweak single-face pass (n_source = 1,n_target= person's feed size)Parity harness
experiments/linkage_comparison/grouping.pymirrored in lockstep (harness == productiontests stay green).Felder
distanceconfidenceclamp(1 - distance)(Core contract)scoreclamp(1 - round(d_pess, 6)), always ≤ confidence, equal only as n → ∞targetUncertaintyUnknown person sizes default to
n = 1(conservative; ordering-neutral when shared by all competitors). Envelope: optional fields, no contract version bump (ADR 0103qualityScoreprecedent).Nicht-Änderungen
d_pessonly ranks which target a card suggests, never whether it is emitted.qualityScore/ isolation / ambiguity unchanged: the scored shortlist stays RAW-sorted; these features describe the shortlist, not the target race (product decision, documented in ADR 0106).score+targetUncertaintyaccepted / absent)._cluster_centroid_distancebranch, legacy DBSCAN fallback.Testbefund
cd workers/people && PYTHONPATH=src python3 -m pytest -q tests→ 263 passed in 10.70s (249 baseline + 14 new intests/test_target_uncertainty_scoring.py).rtk ./gradlew :apps:api:test --tests '*WorkerResultSemisupervisedEnvelopeTest*'→ BUILD SUCCESSFUL, 44 tests, failures=0 errors=0.test_weak_single_face_suggestions.pyfield-set pin extended withscore/targetUncertainty+ value asserts.test_linkage_comparison.pyoracle fixture_ORACLE_ANGLES[1]1.9696069505 → 1.85— original Q won only raw; with N=1 the penalty (0.0554) exceeds the raw gap (0.036) and would erase the fixture's anchor-target-disagreement purpose. Same graph adjacency, same grouping; comment in fixture documents it.ADR:
docs/adr/0106-confidence-aware-target-scoring.json(0105 taken).