feat(people-worker): multi-representant person representation (CV-validated) #1081

Merged
chris merged 1 commit from feat/multi-representant-person-centroids into main 2026-10-09 23:21:12 +02:00
Owner

Messgrundlage (PR #1080, Owner-Bibliothek, 2796 Faces / 36 Personen)

Regel top-1 wrong-suggest
Baseline single-centroid (Produktion heute, Band 0.50) 86.3% 8.7%
Multi-Repräsentant @ 0.50 (dieser PR) 97.0% 2.7%

Christian-Verwechslungen →Nathalie (51×) / →Jesper (40×) fallen auf <3. Neu unter Multi-Repräsentanten: Christian wird selbst Verwechslungs-Hub (max. 8× als falsches top-1 Ziel) — Band-Neukalibrierung beobachten.

Band-Empfehlung (Deployment-Entscheidung, separat): nach Deploy suggest_max_distance 0.50→0.45 setzen — gleiche Vorschlagsquote wie heute bei WS 1.6% (bei 0.50: 2.7%). Multi-Repräsentant-Distanzen schrumpfen systematisch (korrektes top-1 Median 0.325 vs 0.409).

Design (ADR 0108, docs/adr/0108-multi-representant-person-representation.json)

  • Nur semisupervised-v2: je Person/Modell-Pair werden die zugewiesenen Faces per Average-Linkage-Clustering (Cosine) bei representative_split_distance zerlegt; je Cluster ein Unit-Mean-Centroid (gemessener Gewinner vs. Medoid), Singletons = eigener Repräsentant, facelose Person = keiner.
  • Zuordnung (B-Outcome): pro Repräsentant d_pess = d + Z·σ·√(1/n_source + 1/size_rep) (ADR-0106-Margin mit Clustergröße als n_target); persons Pessimistic-Key = min über Repräsentanten; distance = RAW-Distanz des pessimistisch gewählten Repräsentanten; Shortlist (Isolation/Ambiguität) bleibt RAW (Formeln unverändert, Geometrie = Raw-Min).
  • Contract unverändert: confidence = clamp(1 - round(distance,6)), kein Envelope-Feld neu, keine Contract-Version-Bump. Gates, qualityScore, review-only, Emissions-Pfade, v1-Pfad und Legacy-Methoden byte-identisch.
  • Setting: representative_split_distance (V155, NUMERIC(4,3), finite [0,2], People-Default 0.50 = gemessenes Knee; 0.40 hätte besserens top-1, aber ~7× Repräsentanten). 2.0 = alles in einem Cluster = exakt die alte Single-Centroid-Darstellung (Operator-Aus-Schalter). Snapshot als settings.representativeSplitDistance (nur v2-Payloads) in den unveränderlichen Job-Payload; GET/PATCH /api/admin/people/representative-split-distance (super-admin, null=Reset).
  • Single Source: die deterministische Linkage-Core-Funktionen (unit_rows, cosine_distance_matrix, average_linkage_merges, cut_dendrogram) zogen verbatim ins Produktions-clustering.py; das CV-Experiment importiert sie (Kopie gelöscht) — Produktion ist die gemessene Implementierung by construction, Paritätstest pinnt beide Seiten.
  • Grenzen: Personen > REPRESENTATIVE_SPLIT_MAX_FACES (2000) nutzbare Faces behalten den Single-Centroid (dokumentierte Degradierung, kein Job-Failure; Live-Max ~1200). Deterministisch (aufsteigende Face-IDs), Neuberechnung pro Lauf, keine Persistenz.

Nicht geändert

v1-Pfad, qualityScore/Isolation/Ambiguität-Formeln, Emissions-Gates, review-only-Semantik, Web-Code, Deployment-Settings (Band bleibt 0.50 im Deployment bis der Owner umstellt).

Tests

  • Worker: 314 passed (297 bestehende + 17 neue: Parität Produktion↔Experiment, Determinismus, Singleton/All-in-one/Over-cap-Edges, Margin-Rep-Flip mit n_target=Clustergröße, Payload-Semantik (pess_raw), v1-Golden-Guard, Handler-Threading, Envelope-Contract).
  • Core: 1651 Tests, 0 Failures (13 neue Postgres-Contract-Tests: V155-Migration/Default 0.50, GET/PATCH-Validierung [0,2], null-Reset, v2-only Payload-Snapshot, v1/Legacy-Payload unverändert, Queued-Jobs unverändert). WorkerResultSemisupervisedEnvelopeTest 44/44 grün.
  • Paritätsnachweis: CV-Experiment (jetzt auf Produktions-Core) re-run auf dem identischen Live-Cache reproduziert die PR-#1080-Zahlen exakt (16/16 Gruppengröße×Band-Zellen identisch: medoid 2692/91, medoid-pess 2706/74 @ Band 0.50).

🚫 Nicht mergen ohne Owner-Freigabe — Deployment-Band-Entscheidung (0.45) ist bewusst NICHT Teil dieses PRs.

## Messgrundlage (PR #1080, Owner-Bibliothek, 2796 Faces / 36 Personen) | Regel | top-1 | wrong-suggest | |---|---|---| | Baseline single-centroid (Produktion heute, Band 0.50) | 86.3% | 8.7% | | **Multi-Repräsentant @ 0.50 (dieser PR)** | **97.0%** | **2.7%** | Christian-Verwechslungen →Nathalie (51×) / →Jesper (40×) fallen auf **<3**. Neu unter Multi-Repräsentanten: Christian wird selbst Verwechslungs-Hub (max. 8× als falsches top-1 Ziel) — Band-Neukalibrierung beobachten. **Band-Empfehlung (Deployment-Entscheidung, separat):** nach Deploy `suggest_max_distance` 0.50→0.45 setzen — gleiche Vorschlagsquote wie heute bei WS 1.6% (bei 0.50: 2.7%). Multi-Repräsentant-Distanzen schrumpfen systematisch (korrektes top-1 Median 0.325 vs 0.409). ## Design (ADR 0108, `docs/adr/0108-multi-representant-person-representation.json`) - **Nur `semisupervised-v2`**: je Person/Modell-Pair werden die zugewiesenen Faces per Average-Linkage-Clustering (Cosine) bei `representative_split_distance` zerlegt; je Cluster ein **Unit-Mean-Centroid** (gemessener Gewinner vs. Medoid), Singletons = eigener Repräsentant, facelose Person = keiner. - **Zuordnung (B-Outcome):** pro Repräsentant `d_pess = d + Z·σ·√(1/n_source + 1/size_rep)` (ADR-0106-Margin mit **Clustergröße** als n_target); persons Pessimistic-Key = min über Repräsentanten; `distance` = **RAW-Distanz des pessimistisch gewählten Repräsentanten**; Shortlist (Isolation/Ambiguität) bleibt RAW (Formeln unverändert, Geometrie = Raw-Min). - **Contract unverändert:** `confidence = clamp(1 - round(distance,6))`, kein Envelope-Feld neu, keine Contract-Version-Bump. Gates, qualityScore, review-only, Emissions-Pfade, **v1-Pfad und Legacy-Methoden byte-identisch**. - **Setting:** `representative_split_distance` (V155, NUMERIC(4,3), finite [0,2], People-Default **0.50** = gemessenes Knee; 0.40 hätte besserens top-1, aber ~7× Repräsentanten). **2.0 = alles in einem Cluster = exakt die alte Single-Centroid-Darstellung** (Operator-Aus-Schalter). Snapshot als `settings.representativeSplitDistance` (nur v2-Payloads) in den unveränderlichen Job-Payload; GET/PATCH `/api/admin/people/representative-split-distance` (super-admin, null=Reset). - **Single Source:** die deterministische Linkage-Core-Funktionen (`unit_rows`, `cosine_distance_matrix`, `average_linkage_merges`, `cut_dendrogram`) zogen verbatim ins Produktions-`clustering.py`; das CV-Experiment importiert sie (Kopie gelöscht) — Produktion ist die gemessene Implementierung **by construction**, Paritätstest pinnt beide Seiten. - **Grenzen:** Personen > `REPRESENTATIVE_SPLIT_MAX_FACES` (2000) nutzbare Faces behalten den Single-Centroid (dokumentierte Degradierung, kein Job-Failure; Live-Max ~1200). Deterministisch (aufsteigende Face-IDs), Neuberechnung pro Lauf, keine Persistenz. ## Nicht geändert v1-Pfad, qualityScore/Isolation/Ambiguität-Formeln, Emissions-Gates, review-only-Semantik, Web-Code, Deployment-Settings (Band bleibt 0.50 im Deployment bis der Owner umstellt). ## Tests - **Worker:** 314 passed (297 bestehende + 17 neue: Parität Produktion↔Experiment, Determinismus, Singleton/All-in-one/Over-cap-Edges, Margin-Rep-Flip mit n_target=Clustergröße, Payload-Semantik (pess_raw), v1-Golden-Guard, Handler-Threading, Envelope-Contract). - **Core:** 1651 Tests, 0 Failures (13 neue Postgres-Contract-Tests: V155-Migration/Default 0.50, GET/PATCH-Validierung [0,2], null-Reset, v2-only Payload-Snapshot, v1/Legacy-Payload unverändert, Queued-Jobs unverändert). `WorkerResultSemisupervisedEnvelopeTest` 44/44 grün. - **Paritätsnachweis:** CV-Experiment (jetzt auf Produktions-Core) re-run auf dem identischen Live-Cache reproduziert die PR-#1080-Zahlen **exakt** (16/16 Gruppengröße×Band-Zellen identisch: medoid 2692/91, medoid-pess 2706/74 @ Band 0.50). 🚫 **Nicht mergen ohne Owner-Freigabe** — Deployment-Band-Entscheidung (0.45) ist bewusst NICHT Teil dieses PRs.
feat(people-worker): multi-representant person representation (CV-validated)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m36s
e2a3f125e2
ADR 0108. semisupervised-v2 only: each person is represented by MULTIPLE
representatives — the person's assigned usable faces are decomposed by
average-linkage clustering (cosine) cut at the new owner setting
representative_split_distance (V155, default 0.50, the CV-measured knee;
2.0 = single-centroid off-switch) and each cluster is represented by its
unit mean centroid. The B outcome scores per representative: d_pess =
d + Z*sigma*sqrt(1/n_source + 1/size_rep) with the representative's own
cluster size as n_target; payload distance = raw distance of the
pessimistically chosen representative; the shortlist (isolation/
ambiguity) keeps its raw formulas on the raw-min geometry. distance/
confidence contract, gates, qualityScore, v1 and legacy methods are
untouched.

CV measurement (PR #1080, owner library, 2796 faces / 36 persons):
top-1 86.3% -> 97.0%, wrong-suggest 8.7% -> 2.7% at band 0.50;
Christian -> Nathalie/Jesper confusions 51/40 -> <3. Same suggestion
volume reachable by suggest band 0.50 -> ~0.45 (operator decision after
deploy, WS 1.6% at equal volume).

The deterministic linkage core (unit_rows, cosine_distance_matrix,
average_linkage_merges, cut_dendrogram) moved verbatim into production
clustering.py; the CV experiment imports it (copy deleted) so production
is the measured implementation by construction. Persons above
REPRESENTATIVE_SPLIT_MAX_FACES (2000) keep the single centroid
(documented degradation); decomposition is deterministic (ascending
face id) and recomputed per run.

Core: settings.representativeSplitDistance snapshot (v2-only payload
key), V155 migration (people backfill 0.50), GET/PATCH
/api/admin/people/representative-split-distance (finite [0,2],
null = reset), test schemas mirror the column.

Tests: worker 314 passed (17 new: parity production<->experiment,
determinism, singleton/all-in-one/over-cap edges, margin rep-flip,
payload-semantics, v1 golden guard, handler threading); API 1651
tests 0 failures (13 new Postgres contract tests); CV experiment
re-run on the live cache reproduces the PR #1080 numbers exactly
(16/16 group-size x band cells identical).
chris merged commit 06f5b667ae into main 2026-10-09 23:21:12 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1081
No description provided.