feat(api): multi-representant assignment metric for person ranking (ADR 0109) #1083
Loading…
Reference in a new issue
No description provided.
Delete branch "feat/api-multi-representant-ranking"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Problem
ADR 0108 switched the people worker (semisupervised-v2) to multi-representant person representation — per-person average-linkage split at representative_split_distance=0.50, unit-mean centroid per subcluster, per-representative pessimistic target choice. The three API read sites (
personRanking,mergeTargetRankings,mergeSuggestioninClassInstanceDiscoveryService) kept computing the pre-0108 single live centroid. On multi-modal persons that centroid decays away from every real visual mode.Proven production bug (2026-10): a candidate whose faces sit 0.228 cosine from target "Rikke"'s winning mode ranked "Jutta" first at 0.362, with Rikke outside the top 6 — Rikke holds 435 accepted faces across 47 subclusters and her collapsed centroid resembles no real appearance. The UI title ranking lied against the very cards the worker mints from the same faces.
Solution: persist, don't recompute
No Java port of the linkage (permanent parity risk, O(n²) per request). The worker already computes the representatives per run — it now ships them on the result envelope and the API persists them:
main.py): additive top-level envelope fieldpersonRepresentatives([{modelName, modelVersion, personId, reps: [{repIndex, clusterSize, embedding}]}], deterministic order, full precision). Per-candidate envelope contract untouched:distancestays raw,confidence == clamp(1 - round(distance, 6)), contract version unchanged. v1/legacy methods never attach it.WorkerResultService): fail-closed before any write — canonical person UUIDs, finite equal-length embeddings,clusterSize >= 1, unique(pair, person, repIndex); absent-legal for older workers.person_instance_representatives: JSONBembedding(source of truth) + V136-style trigger-maintainedembedding_vecpgvector mirror; unique(owner, domain, class, model pair, person, rep_index); FK fail-closed toclass_instances. Deliberately no speculative HNSW index (the unused V101 GIN placeholder, dropped in V136, is the precedent).persistSemisupervisedCandidates): FULL-REPLACE per(owner_id, domain_key, class_key)inside the same transaction as candidate ingestion. Empty array = real replacement; absent field (older worker/v1) leaves prior rows untouched. Applied even for empty candidate lists (an empty run still declares the current representation). State/409-stale guards untouched.d_pess = d + Z·σ·√(1/n_source + 1/cluster_size)with the representative's OWN cluster size asn_target(ADR 0106), person key = first-minimal over its representatives in rep_index order (worker convention), reported raw distance = the pessimistically chosen representative's raw distance. No "min over all reps" shortcut, no blending with the single-centroid value. Merge surfaces (unordered pairs) take the pessimistic min over the rep-ified directions;personRankingis exact worker parity (its candidate side is a pending card, never an instance).candidatesPaged,countCandidates, ADR 0107 named-target filters: untouched.Design decisions
suggestSimilarInstances(operator dialog) keeps the live single-centroid metric.Tests
personRankingGoldenVectorsFromWorkerReference): pins the Java ranking to vectors produced by the production Python path —clustering._person_representatives+_pessimistic_target_distanceviaworkers/people/experiments/api_ranking_golden.py(committed, regenerable, verified reproducible). Chosen representative's raw distance, pessimistic score and full ranking order asserted per candidate/person._person_representativesoutput deterministically (feed-order shuffled), singleton rep size 1, v1/legacy never attach, envelope contract invariant holds under multi-rep scoring.Results:
rtk ./gradlew :apps:api:test— all green (full suite, 4 min).pytestinworkers/people(PYTHONPATH=src, CI convention) — 316 passed. Version 1.21.74.ADR:
docs/adr/0109-api-read-path-multi-representant-ranking.json(replaces 0103's single-centroid ranking-metric statement for the three sites; references 0106/0107/0108).