ADR 0106: Confidence-aware target scoring: pessimistic distance for cluster_person_match target choice
Status: accepted Date: 2026-10-07
Context
cluster_person_match cards pick their suggested target by raw centroid distance d. That is a winner's-curse estimator: the target is the MINIMUM over the candidate identities, so noise that pushes one competitor down is selected for and the winning raw distance systematically underestimates the true assignment distance. The effect grows with the number of competitors and shrinks with sample size — both centroids are means over n faces, so the standard error of d is sigma*sqrt(1/n_source + 1/n_target). Leave-one-out measurement on the owner library (Okt 2026) put the residual per-face scatter at sigma = 0.139 (rounded up to 0.14); in the same LOO run every one of the 4 rejected assignments (all with min n <= 6) was a raw-distance win whose pessimistic distance would have lost. Raw d therefore overstates small-sample matches, and the card's confidence = clamp(1 - distance) inherits the overstatement directly.
Decision
Choose the target identity by pessimistic distance d_pess = d + Z * sigma * sqrt(1/n_source + 1/n_target) with documented worker constants TARGET_DISTANCE_SIGMA = 0.14 (LOO sigma = 0.139, Okt 2026, rounded up) and TARGET_DISTANCE_Z = 1.645 (one-sided 95%), at exactly three emission sites: the constrained identities loop, the semisupervised v1/v2 group-candidate B outcome, and the weak single-face pass (n_source = 1). distance stays the RAW centroid distance and confidence keeps clamp(1 - distance) — the envelope contract and existing validators are untouched. Two additive optional fields carry the correction: score = clamp(1 - d_pess, 0, 1) (always <= confidence, equal only in the n -> infinity limit) and targetUncertainty = Z * sigma * sqrt(1/n_source + 1/n_target). No gate changes: no acceptance threshold, no auto-accept band change, no grouping change — d_pess only ranks which single target a card suggests, it never decides whether the card is emitted. Unknown person sizes default to n = 1 (conservative, and ordering-neutral because all competitors share the default). The scored shortlist stays RAW-sorted, so isolation and ambiguity features (ADR 0103/0105) are explicitly NOT uncertainty-corrected: they describe the emitted candidate's neighborhood, not the target-selection race, and correcting them would silently change the persisted ranking score this ADR deliberately leaves alone.
Consequences
Small-sample merge suggestions no longer win on lucky noise: a target whose edge over a competitor is smaller than the uncertainty margin yields to the better-supported identity, and the visible score is honest about how much the card is guessing (score < confidence exactly by targetUncertainty). Cards are still emitted at the same points with the same distance/confidence, so downstream gates, the auto-accept band and persisted quality_score behaviour are byte-identical apart from the two new optional fields; pre-0106 envelopes remain valid and the clustering contract version does not bump (ADR 0103 optional-field precedent). Anchored-hybrid target picks, the constrained 0-anchor centroid branch and the legacy DBSCAN fallback keep raw-distance selection — out of scope here, revisit if LOO data shows the same winner's-curse pattern on those paths. A support-minimum (reject targets below a floor n) remains a possible follow-up if the 4/4 LOO rejections recur after deploy.
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference