This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
ADR 0113 validation plan — representative density graph (semisupervised-v3)
Status: gate document for ADR 0113 (docs/adr/0113-representative-density-graph.json,
status accepted, owner decision 2026-10-16). Gates disposition: G1 PASS (5/6
rows, G1.6 accepted as a documented small-n exception by the owner); G2 PASS
(worker fixtures green); G3 PASS (measured, filled from the prototype lane);
G4 PARTIAL — design confirmed live (6 min 16 s end-to-end at 40× the gate pool,
~2.5× vs v2 on the same library), the < 60 s numeric bar DEFERRED to post-V6
steady state (V6 representative caching is the recorded follow-up); G5 verified
(dark ship, operator A/B on the owner domain, rollback = stay on v2).
Baselines: ADR 0108 CV numbers (top-1 97.0% / wrong-suggest 2.7% at split 0.50,
owner library, k-fold 2796 faces / 36 persons) and current semisupervised-v2
production behavior (workers/people/src/memories_people_worker/clustering.py).
Convention note: the repo keeps feature validation under specs/NNN-*/validation/;
this ADR has no spec dir (yet), so the plan lives beside the ADR JSON in
docs/adr/ and moves into a spec validation dir if a spec-050 amendment spec is
spawned at implementation time.
G1 — Quality non-inferiority on the owner library (assignment_cv harness)
Harness: workers/people/experiments/assignment_cv/run_cv.py (experiment code,
never imported by the worker), extended with a v3 gate lane — new flag
--graph-mode {anchor,representative} (to-create; default anchor reproduces the
ADR 0108 baseline bit-identically). The representative lane derives, for every
evaluated fold group, per-member qualifying identity sets from U×R representative
adjacency at the job's maxDistance (fold-train representatives built by the
production _person_representatives transcription, split 0.50), intersects them
into the wholeGroup identity, and ranks whole-group identities by the existing
medoid-pess rule (ADR 0106 margin, n_target = winning cluster size; parity
pinned by workers/people/tests/test_assignment_cv_medoid.py). Groups whose
wholeGroup is empty are counted out of ranking exactly like the baseline's
out-of-band cases and reported separately (ambiguous-v3 bucket), mirroring the
review-only demotion they would get in production.
Protocol: identical folds, group sizes and seed as the ADR 0108 baseline run —
both lanes consume the SAME JSONL embedding dump (run_cv.py --load/--dump), so
lane-to-lane deltas isolate the graph change. No new dependencies.
Gates (evaluated-group denominator, same as baseline):
| Metric | Gate |
|---|---|
| top-1 (representative lane) | ≥ 96.5% (97.0% − ε, ε = 0.5 pp) |
| wrong-suggest (representative lane) | ≤ 3.2% (2.7% + ε, ε = 0.5 pp) |
| Christian → Nathalie wrong-top-1 | < 3 (ADR 0108 clause; must not return) |
| Christian → Jesper wrong-top-1 | < 3 (ADR 0108 clause; must not return) |
| Christian hub-score (wrong top-1 across all persons) | ≤ baseline + 2 (ADR 0108 CV remainder watch) |
| per-person regression | no person loses > 2 pp top-1 vs baseline without an offsetting wrong-suggest decrease; regression table required in the report |
MEASURED (G1 lane, 2026-10-16, g1_gate_report.md on main, cache of 2026-10-09):
5/6 gates PASS. top-1 (agg) 98.30% PASS · wrong-suggest (agg) 1.70% PASS ·
Christian→Nathalie 0 PASS · Christian→Jesper 0 PASS · hub-score 23 ≤ 35+2 PASS ·
per-person regression FAIL — one person (Jutta, n=19 groups): −4.33 pp top-1 with
wrong-suggest +4.33 pp, caused by 2 previously-correct groups demoting to the
ambiguous-v3 bucket (out of ranking → review-only traffic, not wrong suggestions).
The multi-representant representation itself (ADR 0108, already shipped) already
costs this person ~7 pp vs the pre-0108 anchor lane — the graph gate adds the
demotion on top. ambiguous-v3 bucket overall: 55/3003 = 1.83% of groups.
Adjudication (owner, 2026-10-16): ACCEPTED as a documented small-n exception —
the 2 demoted groups become review-only traffic, never wrong suggestions, at
n=19 groups; G1 therefore PASSES. The anchor lane reproduces the published
86.3%/8.7% pre-0108 baseline exactly, so the protocol is aligned. Remaining
open gates at this point: G4 (deploy-host run).
Honesty clause (from the assignment_cv README, restated for this gate): the CV measures the assignment stage with production-parity transcription and is optimistically biased by near-duplicate leakage; the gate is the LANE-TO-LANE delta on identical inputs, not the absolute numbers.
G2 — Synthetic behavioral fixtures
New test file: workers/people/tests/test_representative_graph_clustering.py
(existing files to stay green byte-identically: test_semisupervised_clustering.py,
test_semisupervised_v2_clustering.py, test_multi_representant.py,
test_clustering_golden.py, test_constrained_forward_clustering.py,
test_clustering_registry.py, test_target_uncertainty_scoring.py).
Required fixtures:
- Edge-existence invariants: node set is exactly U ∪ R (assert no anchor face id is a graph node); only U×U and U×R edges exist; two representatives of different persons are never adjacent (cannot-link broadcast); an anchor face within threshold of a pool face produces NO edge (the documented v2→v3 behavior change — pinned as expected).
- Identity separation: a pool face adjacent to representatives of two
persons yields both qualifying ids, empty wholeGroup, and the ambiguous
review-only demotion (carry-over of
test_density_bridge_yields_only_ambiguous_weak_suggestionsemantics); distinct identities never share a B card. - Gate-from-representative: a pool face within threshold of a raw anchor but beyond from all of that person's representatives loses its qualifying edge (distortion case, expected); the converse (adjacent to a representative but not to any raw anchor) still qualifies — the edge v3 exists to keep.
- Group formation: groups are formed on the U×U subgraph only — a
representative bridging two U clusters that are NOT U×U-connected yields two
groups (component over U only);
SCALABLE_MAX_HAC_NODESfail-closed still fires above the cap. - Determinism: permuted feed order → byte-identical candidates; two runs on
identical input → byte-identical (mirrors
test_byte_identical_reruns). - Method/contract matrix: v3 registered and in
SINGLE_PASS_METHODS; stampssemisupervised-v3+ contract version 2; worker refuses v3@1 and v1/v2@2 via the existingunsupported_clustering_contractpath; v1/v2 stamps remain version 1 (extendstest_v2_contract_version_is_unchanged). - Core side:
WorkerResultSemisupervisedEnvelopeTestgains v3@2 accept / v3@1 and v3@3 reject cases, plus rejection of a v3-stamped envelope on a job that requested v2 (payload/envelope method mismatch); thePeopleClusteringResetControllerallowlist test coverssemisupervised-v3;WorkerJobEnqueueServicesnapshots version 2 for method v3.
G3 — Performance benchmarks, three regimes
Harness: new workers/people/experiments/rep_graph_bench.py (experiment code),
reusing the ISSUE-081 scale-test pattern from
test_graph_build_stays_bounded_at_household_scale — synthetic 512-dim
embeddings, identical inputs fed to the v2 baseline path (full _density_graph
over L+U with labels) and the v3 rep-graph builder. Metrics per run: graph wall
time, peak RSS, edge count, connected-component size histogram, HAC partition
wall time, and an assertion counter for computed pair blocks (structural proof
that no L×L / L×U / L×R block is computed).
Regimes (R sized from a real V156 snapshot or synthetic at household scale, ~1.5k reps):
| Regime | Composition | Gates |
|---|---|---|
| Steady state | U=500, L≈12k | pairs ≤ U²/2 + U·R (asserted); graph wall time ≥ 20× faster than baseline; peak RSS ≤ baseline; MEASURED (prototype lane, synthetic, experiments/rep_graph/benchmark.py): 111× faster (3.35 s → 0.03 s), pairs 78.1M → 342k (228×), peak RSS 733.5 → 8.5 MiB, edges 151,144 → 1,128, max component 102 → 10 (R=434) |
| Mid | U=5k, L≈12k | wall time ≥ 3× faster than baseline; peak RSS ≤ baseline; MEASURED: 21.7× faster (9.78 s → 0.45 s), pairs 144.5M → 14.7M, peak RSS 775.5 → 350.0 MiB, max component 113 → 20 |
| Bulk | U≈N≈22k | no regime may regress vs baseline by > 10% (U·R overhead included); component-size distribution and edge counts reported; MEASURED: 5.7× FASTER (19.58 s → 3.44 s, no regression), peak RSS 818.6 → 713.5 MiB, max component 30 → 30; bulk runs uncapped at full 22k scale |
Benchmark provenance and honesty notes: synthetic 512-dim generator with the ADR 0108
geometry (intra-mode 0.280 / cross-mode same-person 0.519 / cross-person 0.999;
reps via production _person_representatives at split 0.50); identity sanity on the
prototype graph: 100.0% of known-person U faces adjacent to ≥1 own-person rep, 0.0%
adjacent to any foreign rep, 100.0% same-mode U–U pairs connected (cross-mode U–U
pairs fragment below threshold BY DESIGN — the reps carry identity, the graph carries
co-membership). tracemalloc peak scales with U in BOTH variants (per-row embedding
conversion), so the prototype introduces no new memory class. The G4 on-host full-run
number and the G1 owner-library CV lane remain the outstanding quality/latency gates —
these cells settle the STRUCTURAL performance question only.
Component-size histograms are informational (no shape gate) — they feed the churn
assessment, not a pass/fail. The household-scale memory guard
(test_graph_build_stays_bounded_at_household_scale) must additionally cover the
rep-graph builder.
G4 — Run-time gate (16 GB deploy host)
A full steady-state owner_face_recognition run on the owner library
(~500 new faces, ~12k anchors, method semisupervised-v3, warm representative
cache) must complete < 60 s from feed fetch to result submit, sustained CPU
bounded, no OOM — versus 10–14 min today (ISSUE-133). Measurement: worker run
timing logs on the deploy host. MEASURED (2026-10-16, deploy host, owner domain
switched to semisupervised-v3): first fully successful v3 run 75770b0e —
2875 candidates, HTTP 202, end-to-end 6 min 16 s (20:32:45 → 20:39:01) at
pool=20151 / anchors=6350 / tentative=388. The gate's steady-state scenario
(~500 new faces) does NOT exist on the live library today: outcome-A faces stay
unassigned by design, so ~20k unassigned faces sit in the pool every run — the
< 60 s numeric bar is therefore DEFERRED. Measured evidence at 40× the gate pool
shows ~2.5× end-to-end speedup vs the same library's v2 runs the same day
(11–18 min), consistent with the G3 regime table. Remaining steady-state
blockers are the known follow-ups: V6 representative caching (the per-run
O(Σ n³) person decomposition is still paid in full) and pool shrink as the
owner confirms faces. Verdict: G4 PARTIAL — design confirmed live, numeric gate
deferred to post-V6 steady state. Two live-launch seams were found and fixed on
the way (ISSUE-164 envelope contract version, ISSUE-165 releasedTentativeFaces
canonical-UUID filter); one benign tail heartbeat 403 observed (job completed
successfully regardless). Sub-gate (unit-test level): on a fingerprint hit, the worker
must not invoke per-person linkage — monkeypatch counter asserts zero
cosine_distance_matrix calls for fingerprint-hit persons and full recompute on
fingerprint miss (face added/removed, split threshold changed, model pair
changed) — in test_representative_graph_clustering.py (or an extension of
test_multi_representant.py), plus the G2 warm-cache/no-cache candidate parity.
Post-V6 live measurement (2026-10-16, PRs #1268/#1269): V6 shipped dark and
deployed; the unit-level sub-gate is green (zero linkage calls on hits, full
recompute on miss, byte-identical hit parity). Live end-to-end pair at
pool≈20k: MISS run 50aa2420 6 min 45 s (computes + persists 987/987
fingerprints), HIT run 01f5fcf7 7 min 04 s (same 2919 candidates, HTTP 202) —
the delta is WITHIN RUN-TO-RUN NOISE, no measurable end-to-end saving at this
library composition. Two honest reasons: (1) run 1's own results perturb
assignments (auto-accepts + tentative releases), so run 2 carries per-person
fingerprint misses anyway — the cache pays off only across assignment-STABLE
runs; (2) the per-person decomposition at the observed person-size distribution
(biggest person ~1142 faces, under the 2000 cap) is tens of seconds, not the
minutes the projection assumed. The remaining per-run budget is dominated by
the HAC partition over the ~20k pool + feed fetch + candidate persistence, none
of which V6 touches. Net: V6 removes a real but small cost and pins the
mechanism for the incremental-feed future; the dominant levers for the <60s bar
stay pool shrinkage (owner confirmations) and feed-level incrementality.
G5 — Rollout checklist
- Dark ship: registry +
SINGLE_PASS_METHODS+ Core processing-mode matrix +PeopleClusteringResetControllerallowlist + enqueue version stamp only;DEFAULT_CLUSTERING_METHODunchanged; a production owner stays on v2 until the operator opts in. A v3 result envelope against a non-v3 job payload is rejected (G2.7). - Operator A/B: per-domain switch via
settings.clusteringMethod; before any default consideration, run a v2 → v3 → v2 cycle on the owner library and report candidate-key churn (new/changed/removed keys, review-card delta). - In-flight safety: jobs snapshot settings immutably (ADR 0104 D6) — a mid-stream switch completes and persists already-queued jobs with their frozen method/version stamps; switch-back is a mandatory test case.
- Suggest-band recalibration is DEFERRED to post-deploy measurement
(representative-adjacency distances shift the in-band rate; mirror of the
ADR 0108 deployment note) — no
suggest_max_distancechange ships with v3. - Acceptance: ADR 0113
proposed→acceptedonly after G1–G4 pass; the TBD (prototype lane) cells are filled at reconciliation; any gate failure returns the ADR to drafting with the measured numbers attached.
Reconciliation
The prototype lane (parallel work) produced the G3 structural numbers (filled above) and the G4 projection. At reconciliation remains: run the G1 owner-library CV lane, measure G4 on the deploy host, record the benchmark artifacts path (report JSON/markdown, never embedding caches), update the ADR status if all gates pass, and file KNOWN_ISSUES entries for any documented degradation accepted on the way (e.g. representative edge-flip cases observed at owner scale).
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference