feat(workers): semisupervised-v3 — representative density graph clustering (ADR 0113 V1–V5) #1259

Merged
chris merged 5 commits from feat/semisupervised-v3-worker into main 2026-10-16 11:56:45 +02:00
Owner

Summary

Worker-side of ADR 0113 (companion Core PR above): the measured representative-density-graph optimization ships as the new single-pass method semisupervised-v3, dark (registry addition only, default unchanged).

  • V2 — graph: per model pair, node set = U ∪ R (unlabeled pool + ADR 0108 representatives as labeled identity-evidence nodes); pairwise work restricted to U×U + U×R blocks — NO L-blocks (anchor faces are not graph nodes). Substrate promoted from the validated prototype (PR #1255, 111× steady-state measured speedup) into clustering.py; prototype module kept as identity re-export so the G3 benchmark + its tests run on production code.
  • V3 — group formation: constrained average-linkage HAC (partition_hac_components, unchanged) over the U-only filtered adjacency; representatives never form/extend/split groups; SCALABLE_MAX_HAC_NODES fail-closed propagates.
  • V4 — gates: _emit_group_candidates generalized, NOT forked: optional rep_adjacency view; None → original labeled-graph loop untouched (v1/v2 byte-identity preserved by their suites). qualifying/wholeGroup read rep adjacency; ADR 0106/0108 pessimistic B-scoring, ADR 0103 qualityScore, ambiguity demotion, consumption semantics unchanged.
  • V5 — contract: SEMISUPERVISED_V3_CONTRACT_VERSION = 2; handler accepts v3 only at version 2 (per-method required-version set; v1/v2 still require 1; wrong version → existing unsupported_clustering_contract fail path). _stamped derives the version from the method. person_representatives now computed for v2 AND v3.
  • New pairs with no anchors no longer skipped: rep-less graph degenerates to the plain pool graph and emits honest C cards (documented).

Test plan

  • New test_semisupervised_v3_clustering.py: 20 tests covering the G2 fixture groups (edge invariants incl. the pinned anchor-no-edge behavior change, identity separation, gate-from-representative distortion + converse, U-only grouping, fail-closed cap, determinism, contract matrix)
  • ENTIRE workers/people/tests/: 345 passed, 1 skipped (pre-existing heif), 99 subtests — v1/v2 suites byte-identical (one membership pin extended by semisupervised-v3, the set's contract IS its membership, ADR 0104 precedent)
  • Reproduce: cd .worktrees/v3-worker && PYTHONPATH=workers/people/src .venv/bin/python -m pytest workers/people/tests/ -q
  • Cross-lane contract verified against the API PR: method string, v3@2, split-distance stamping
  • Known cosmetic: ISSUE-162 (v3 fail-closed message text says v2; shared static string, behavior correct)
## Summary Worker-side of ADR 0113 (companion Core PR above): the measured representative-density-graph optimization ships as the new single-pass method `semisupervised-v3`, dark (registry addition only, default unchanged). - **V2 — graph:** per model pair, node set = U ∪ R (unlabeled pool + ADR 0108 representatives as labeled identity-evidence nodes); pairwise work restricted to U×U + U×R blocks — NO L-blocks (anchor faces are not graph nodes). Substrate promoted from the validated prototype (PR #1255, 111× steady-state measured speedup) into `clustering.py`; prototype module kept as identity re-export so the G3 benchmark + its tests run on production code. - **V3 — group formation:** constrained average-linkage HAC (`partition_hac_components`, unchanged) over the U-only filtered adjacency; representatives never form/extend/split groups; SCALABLE_MAX_HAC_NODES fail-closed propagates. - **V4 — gates:** `_emit_group_candidates` generalized, NOT forked: optional `rep_adjacency` view; `None` → original labeled-graph loop untouched (v1/v2 byte-identity preserved by their suites). qualifying/wholeGroup read rep adjacency; ADR 0106/0108 pessimistic B-scoring, ADR 0103 qualityScore, ambiguity demotion, consumption semantics unchanged. - **V5 — contract:** `SEMISUPERVISED_V3_CONTRACT_VERSION = 2`; handler accepts v3 only at version 2 (per-method required-version set; v1/v2 still require 1; wrong version → existing `unsupported_clustering_contract` fail path). `_stamped` derives the version from the method. `person_representatives` now computed for v2 AND v3. - New pairs with no anchors no longer skipped: rep-less graph degenerates to the plain pool graph and emits honest C cards (documented). ## Test plan - New `test_semisupervised_v3_clustering.py`: 20 tests covering the G2 fixture groups (edge invariants incl. the pinned anchor-no-edge behavior change, identity separation, gate-from-representative distortion + converse, U-only grouping, fail-closed cap, determinism, contract matrix) - ENTIRE `workers/people/tests/`: **345 passed, 1 skipped (pre-existing heif), 99 subtests** — v1/v2 suites byte-identical (one membership pin extended by `semisupervised-v3`, the set's contract IS its membership, ADR 0104 precedent) - Reproduce: `cd .worktrees/v3-worker && PYTHONPATH=workers/people/src .venv/bin/python -m pytest workers/people/tests/ -q` - Cross-lane contract verified against the API PR: method string, v3@2, split-distance stamping - Known cosmetic: ISSUE-162 (v3 fail-closed message text says v2; shared static string, behavior correct)
docs(known-issues): record v3 fail-closed message misnomer (ISSUE-162)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
805b0d6167
chris force-pushed feat/semisupervised-v3-worker from 805b0d6167
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
to a2e55eaace
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
2026-10-16 11:56:39 +02:00
Compare
chris merged commit 7150a5150c into main 2026-10-16 11:56:45 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1259
No description provided.