ADR 0100: Constrained Density Clustering For Forward Face Recognition
Status: proposed Date: 2026-09-29
Context
ADR 0017 established constrained DBSCAN-style density retargeting for unnamed candidate assignments in Core (fixed anchors, cannot-link between identities, movable owner_tentative faces), motivated by the Morfar case: faces density-connected to manually moved anchor faces stayed in an unnamed candidate because pairwise threshold retargeting could not see cluster structure. Retrospective retargeting implementing those semantics exists in ClassInstanceService (constrainedDensityComponents / violatesCannotLink).
The forward recognition path, however, never applies these semantics. The people worker's owner_face_recognition job clusters only unassigned faces with unconstrained DBSCAN and uses confirmed anchors exclusively through threshold comparisons: nearest single anchor face (strong band), cluster-centroid-to-person-centroid band, and weak single-face kNN consensus. Consequence: a face density-connected through an anchor cloud but farther than maxDistance from every individual anchor is invisible in forward runs; it only becomes attached via manual retrospective retargeting or by luck through the centroid band. Owners experience this asymmetry as inconsistent recognition quality between manual runs and automatic ingestion.
Three additional gaps compound this: (a) thresholds are global, not per-person; (b) strong-band matching has extreme-value exposure to a single outlier anchor (the same failure mode the weak band already fixed with consensus gating); (c) density components that touch two or more confirmed identities are silently matched to the single best centroid person or proposed as one new person, with no review surface for the ambiguity.
Ingestion recon (2026-09-29): Core has no candidateType registry (stored generically, no CHECK constraint, OpenAPI plain string); the candidate carousel already offers merge-into-a-different-person via a picker; band auto-assignment fires on any cluster_person_match carrying existingInstanceId, including the stash-shadow branch, and the only review-only guard is weakMatch on single-face candidates. Any review-only candidate shape must therefore be guarded Core-side, not just flagged worker-side.
Owner decisions (2026-09-29): ambiguous multi-identity components become review-only suggestions in the suggestion carousel (actions unchanged: merge into suggested person, merge into other person, new person, stash, reject); the auxiliary suggestion bands remain; the new behavior ships default-on with a revert flag.
Decision
Extend ADR 0017's constrained density semantics to the forward recognition path. The forward pass and the retrospective retargeting pass share one clustering model.
-
Constrained forward clustering (people worker). The owner_face_recognition job builds one density graph per (modelName, modelVersion) pair over confirmed anchor faces (from payload assignments) plus unassigned faces. Anchor-anchor pairs with different person identities are cannot-link (no edge). Connected components are admitted when they contain at least minimumRecognizedFaces total nodes and at least one core node (degree >= minimumRecognizedFaces). Per admitted component: zero anchor identities -> legacy new_person_cluster emission (payload parity); exactly one identity -> cluster_person_match toward that person, with weakMatch:true when the movable-centroid distance exceeds clusterMatchMaxDistance so Core treats it review-only; two or more identities -> one review-only cluster_person_match toward the closest person centroid with weakMatch:true, ambiguous:true, and conflictingPersonIds listing the other identities. Tentative faces are never anchors and never step-1 labels; under ADR 0102 (semisupervised-v1 only) they are movable unlabeled pool members and subject to the outcome-A release.
-
Implementation location: the worker. The forward pass already receives the full owner embedding feed plus assignments (ADR 0097/0098 anchor split), so no new data flow and no ADR 0010 privacy-boundary change. Core's ADR 0017 retargeting remains authoritative for retrospective moves after manual edits.
-
Rollout flag: constrainedClusteringEnabled, default true, stored on discovery_domains, threaded into the job payload settings, read by the worker; false restores the legacy path byte-identically (pinned by existing golden tests).
-
Core-side defense in depth: ambiguous candidates (cluster_person_match + ambiguous true) are never auto-assigned anywhere - the named-target auto branches and the stash-shadow branch all guard on ambiguity, not relying solely on the worker setting weakMatch. weakMatch-only stash behavior is unchanged (ADR 0097 semantics preserved).
-
Auxiliary bands retained: the centroid-band cluster suggestions and weak single-face consensus suggestions continue unchanged; constrained clustering is a superset for anchor-connected faces and replaces only the unconstrained clustering step for admitted components. Non-admitted components' faces fall through to the weak-consensus scan as today.
-
Review UX: ambiguous cards get a distinct review presentation (lower certainty than strong matches, higher visibility than muted weak matches), keeping all existing actions including merge-into-other-person.
Consequences
Positive: Morfar-style density-connected faces attach to the right person in forward runs without manual retrospective retargeting; multi-identity ambiguity becomes visible and resolvable in review instead of being silently resolved toward the closest centroid; per-owner revert is possible without redeploy; legacy behavior remains pinned by golden tests.
Costs/risks: the worker graph now includes anchor nodes, making clustering O((anchors+unassigned)^2) per model pair - acceptable at household scale; revisit if owner libraries grow (mirrors ADR 0017's pgvector revisit trigger). Default-on behavior change: a bad interaction could alter automatic assignment patterns; the flag is the mitigation and the first validation run on a real library is required before wide release. Carousel ranking: ambiguous review cards must not outrank confident matches; ordering is presentation-level and to be validated in review. Ambiguity frequency depends on library density; if ambiguous suggestions prove too numerous, the follow-up is tuning the ambiguity threshold (e.g. requiring conflicting identities within a tighter radius), not removing the review surface.
Tests: worker golden tests pin the flag-off path; new tests pin Morfar-style attachment, cannot-link separation, ambiguous emission shape, admission gating fall-through, and flag-off parity. Core tests pin: settings threading, migration default, ambiguity guard on named-target and stash branches, and unchanged weakMatch stash behavior.
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference