feat: semisupervised-v6 anchor-quality gate (ADR 0119, dark ship) #1501

Merged
chris merged 3 commits from feat/semisupervised-v6-anchor-gate into main 2026-11-13 21:13:39 +01:00
Owner

Implements ADR 0119 V1–V4 + V7 dark-ship parts (V5 requalify audit job + V6 gated CV lane are follow-ups).

Worker (94be636c)

  • SEMISUPERVISED_V6_METHOD registered (CLUSTERING_METHODS / SINGLE_PASS_METHODS), contract version 5
  • _semisupervised_v6_candidates: v5 flow over gated inputs — _anchor_labels(min_confidence=None) and _person_representatives_with_fingerprints(min_confidence=None) plus a gated labeled partition node set
  • Gate: feed rows with detectionConfidence < anchorMinConfidence are excluded from anchor-adjacency evidence, representative construction (filtered before fingerprinting → V168 cache self-invalidates once) and the labeled node set. Anchor floor: all-weak person retains its single highest-confidence anchor. Fail-open: absent field ⇒ never gated. v1–v5 paths byte-identical (insert-only diff).
  • main.py settings plumbing + v6@5 in the unsupported_clustering_contract matrix

Core (5ff4ad94, 9f534953)

  • V188: discovery_domains.anchor_min_confidence NUMERIC(3,2) (nullable, NULL = off)
  • SemisupervisedMethod enum gains V6@5 (snapshotsAnchorMinConfidence()) — single-source matrix
  • Enqueue snapshots version 5 + gate (NULL column ⇒ key absent; v1–v5 payload shapes byte-identical)
  • OwnerFaceEmbeddings feed: additive optional detectionConfidence (NULL ⇒ omitted; old workers unaffected)
  • Envelope matrix: v6@5 accepted, v6@≤4 and cross-stamp rejections mirror v5
  • PeopleClusteringResetController: v6 allowlist + GET/PATCH /api/admin/people/anchor-min-confidence

Tests

  • Worker: 636 passed (full suite), incl. 23 new v6 tests; v1–v5 goldens unchanged
  • Core: 1121 tests, 0 failures (worker.* + people.* + web.PeopleClustering* + ClassInstanceDiscoveryServiceTest), incl. +10 envelope matrix rows, new AnchorMinConfidencePostgresTest (Testcontainers, real Flyway), feed fail-open tests, 7 H2 fixture schemas extended (gotcha: new discovery_domains columns must be added there too)

ADR 0119 follow-ups (separate PRs): people_anchor_requalify audit job; gated v6 CV lane + shadow-aware sweep (decides anchor_min_confidence default and re-opens clustering_max_distance).

Implements ADR 0119 V1–V4 + V7 dark-ship parts (V5 requalify audit job + V6 gated CV lane are follow-ups). ## Worker (94be636c) - `SEMISUPERVISED_V6_METHOD` registered (CLUSTERING_METHODS / SINGLE_PASS_METHODS), contract version 5 - `_semisupervised_v6_candidates`: v5 flow over gated inputs — `_anchor_labels(min_confidence=None)` and `_person_representatives_with_fingerprints(min_confidence=None)` plus a gated labeled partition node set - Gate: feed rows with `detectionConfidence < anchorMinConfidence` are excluded from anchor-adjacency evidence, representative construction (filtered **before** fingerprinting → V168 cache self-invalidates once) and the labeled node set. **Anchor floor**: all-weak person retains its single highest-confidence anchor. **Fail-open**: absent field ⇒ never gated. v1–v5 paths byte-identical (insert-only diff). - main.py settings plumbing + v6@5 in the unsupported_clustering_contract matrix ## Core (5ff4ad94, 9f534953) - V188: `discovery_domains.anchor_min_confidence NUMERIC(3,2)` (nullable, NULL = off) - SemisupervisedMethod enum gains V6@5 (`snapshotsAnchorMinConfidence()`) — single-source matrix - Enqueue snapshots version 5 + gate (NULL column ⇒ key absent; v1–v5 payload shapes byte-identical) - OwnerFaceEmbeddings feed: additive optional `detectionConfidence` (NULL ⇒ omitted; old workers unaffected) - Envelope matrix: v6@5 accepted, v6@≤4 and cross-stamp rejections mirror v5 - PeopleClusteringResetController: v6 allowlist + GET/PATCH `/api/admin/people/anchor-min-confidence` ## Tests - Worker: **636 passed** (full suite), incl. 23 new v6 tests; v1–v5 goldens unchanged - Core: **1121 tests, 0 failures** (worker.* + people.* + web.PeopleClustering* + ClassInstanceDiscoveryServiceTest), incl. +10 envelope matrix rows, new AnchorMinConfidencePostgresTest (Testcontainers, real Flyway), feed fail-open tests, 7 H2 fixture schemas extended (gotcha: new discovery_domains columns must be added there too) ADR 0119 follow-ups (separate PRs): people_anchor_requalify audit job; gated v6 CV lane + shadow-aware sweep (decides anchor_min_confidence default and re-opens clustering_max_distance).
Registry method semisupervised-v6 (contract version 5, ADR 0116 shape):
reuses the v5 flow with three gated inputs, v1-v5 byte-identical.

- _anchor_labels gains optional min_confidence=None (byte-identical when
  absent): feed rows with detectionConfidence < threshold are excluded
  from anchor-adjacency evidence and from the labeled partition node
  set. Fail-open on absent confidence (old Core feed). Anchor floor: an
  all-weak person retains exactly its single highest-confidence anchor
  (documented degradation).
- _person_representatives_with_fingerprints gains the same optional
  parameter, applied to the row selection BEFORE the fingerprint — the
  V168 fingerprint hashes the selected faceAnnotationId set, so the
  cache is content-addressed and self-invalidating (first v6 run:
  one-time recompute; a threshold change that leaves the selection
  identical keeps the fingerprint). Anchor floor covers the
  single-centroid degenerate case.
- _semisupervised_v6_candidates = the v5 flow verbatim over the gated
  node set, stamped v6@5; _semisupervised_v5_candidates,
  _emit_group_candidates, _build_rep_graph and _pair_threshold_adjacency
  are never modified.
- main.py: settings.anchorMinConfidence (optional, fail-open default
  off, finite [0,1]); v6@5 joins the unsupported_clustering_contract
  matrix; the gated representative build + gate thread through the v6
  dispatch only.

Tests: 23 new cases in test_semisupervised_v6_clustering.py — registry/
stamps, handler contract matrix (v6@5 runs, [email protected] + (v1..v5)@5
refused), inert-gate v5 parity golden, gated-reference partition
golden, permuted-feed determinism, gate units (threshold boundary,
fail-open, anchor floor), fingerprint self-invalidation + re-hit, and
the end-to-end attractor-removal semantics (B -> C/A when the only
qualifying anchor is sub-threshold). Existing v1-v5 golden fixtures
unchanged and green (636 passed).

Refs ADR 0119 (docs/adr/0119-anchor-quality-gated-identity-evidence.json).
- V188: discovery_domains.anchor_min_confidence NUMERIC(3,2), nullable,
  NO table default — NULL = gate off (the ADR 0108 off-switch pattern;
  the CV sweep decides any default, never the schema).
- GET/PATCH /api/admin/people/anchor-min-confidence mirroring the
  representative-split-distance endpoints: super-admin, finite [0,1],
  null/blank resets to NULL (the off state — there is no default),
  NUMERIC(3,2)-honest rounding, normalized string round-trip; other
  domains untouched.
- PeopleClusteringResetController ALLOWED_CLUSTERING_METHODS gains
  semisupervised-v6 (registry coexistence — v1-v5 stay selectable).
  Admin UI exposure is a follow-up; the distance section already
  renders for v6 via the prefix-based visibility.

Tests: AnchorMinConfidencePostgresTest (Testcontainers PostgreSQL +
production Flyway chain — V188 column contract, nullable/no-default on
people + fresh domains, endpoint round-trips, boundaries, out-of-range
rejection, cross-domain isolation) and the mock-controller allowlist
tests.

H2 test fixture schemas gain the additive column (nullable, [0,1]
CHECK) so the enqueue-path tests keep their schema contract.

Refs ADR 0119 (docs/adr/0119-anchor-quality-gated-identity-evidence.json).
feat(api): semisupervised-v6 contract matrix + feed detectionConfidence (ADR 0119 V2/V3)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Deploy review instance (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 2m43s
9f534953f6
- SemisupervisedMethod gains SEMISUPERVISED_V6 (wire
  semisupervised-v6, contract version 5) — single source for the
  result-envelope acceptance matrix, the enqueue snapshot, the
  persistence-mode derivation and the provenance guards.
- WorkerJobEnqueueService: v6 jobs snapshot
  clusteringContractVersion=5 plus anchorMinConfidence from
  discovery_domains.anchor_min_confidence (null-safe: NULL column ⇒
  the settings key is ABSENT — the worker-side fail-open off state;
  0.0 snapshots as explicit off). v1-v5 payloads never carry the key:
  byte-identical shapes. New SEMISUPERVISED_V6_CLUSTERING_CONTRACT_
  VERSION=5 constant and snapshotsAnchorMinConfidence() capability.
- ClassInstanceDiscoveryService: SEMISUPERVISED_V6_METHOD provenance
  stamp constant (valid everywhere the v1 stamp is).
- OwnerFaceEmbeddingsController: the owner-face-embeddings feed ships
  additive optional detectionConfidence per embedding row, sourced
  from discovery_results.confidence in both feed branches (NULL ⇒
  field omitted — old feeds never gate; old workers ignore the
  unknown key). No V168 change: the fingerprint self-invalidates via
  the gated selection (ADR 0119 V3).

Tests: envelope matrix (v6@5 accepted; [email protected], v6@null, v5@5, v1@5,
cross-method stamps, legacy forged v6 markers rejected), enqueue
snapshot (v6@5 + anchorMinConfidence; NULL omits; v5/v1 never carry
the key; allowlist + unknown rejected), feed detectionConfidence
(present with stored confidence incl. pending branch, omitted on
NULL), v6 provenance stamp mirroring, and the
PeopleClusteringMethodPostgresTest v6 allowlist/snapshot rows.

Refs ADR 0119 (docs/adr/0119-anchor-quality-gated-identity-evidence.json).
chris merged commit b39e629c72 into main 2026-11-13 21:13:39 +01:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1501
No description provided.