docs(adr): ADR 0104 draft — constrained average-linkage as opt-in semisupervised-v2 #1034

Merged
chris merged 2 commits from design/linkage-average-proposal into main 2026-10-05 07:42:39 +02:00
Owner

What

ADR 0104, status proposed — a DRAFT design proposal, pending user design approval. No production change, no implementation authorization. Evidence base: PR #1033 (reviewed synthetic-only linkage comparison).

Decisions already made by the user (2026-10-05)

  • D1 — new registry method id (e.g. semisupervised-v2); semisupervised-v1 stays byte-unchanged; per-domain opt-in via settings.clusteringMethod; supported (method, contractVersion) matrix in Core, fail closed on unsupported combinations. No new envelope fields.
  • D2 — rollout keeps the stored threshold (clustering_max_distance, default 0.35). Recalibration toward ~0.45 is a separate, evidence-based follow-up (synthetic-only so far). Rollback also restores 0.35.
  • D4 — scalability is a design gate before any code: prove the scalable algorithm produces exactly the reference HAC partition (incl. stable tie-break), without a full N×N matrix (ISSUE-081). Alternatives (component decomposition, hybrid size cap) are evaluated inside the gate, not assumed. The spec-050 "ONE pass" wording must be re-read against the chosen approach.

Explicitly open (user input required)

  • Q3 — wholeGroup rule for B cards under average linkage. Recommended (and reviewed): keep the gate X qualifies iff every unlabeled member has some confirmed anchor of X within threshold adjacency — anchors are read from the graph, not from HAC group membership, so an anchor partitioned elsewhere still qualifies. Mean-based identity attachment is NOT proposed (no reviewed evidence).

Key content

  • Not a one-line linkage swap: HAC changes both the search strategy and anchor consumption (anchors are partitioned and group-gated instead of reused during greedy growth). Anchor-containing groups can emit C; confidence/qualityScore/keys change where group composition changes (that is the point — fewer wrong 2-face fragments; two-face clusters per se are acceptable).
  • Identity safety: explicit group-level confirmed-identity conflict check before each merge (removing cannot-link edges alone does not prevent a mean ≤ threshold merge across identities).
  • Rollback = future-job selection only; in-flight v2 jobs complete and persist (immutable job settings snapshot); supersede-exempt backing cards and tentative retarget/release (ADR 0102 amendment) are untouched — method-switch lifecycle handling is required planning.
  • Out of scope: weak extra pass, multi-pass restructuring, qualityScore refit, concurrent ranking changes, private-library benchmarking, legacy method changes.

Verification

Docs-only change (one new ADR JSON); CI scope detection will skip builds. Merging does not deploy anything.

## What ADR 0104, status **proposed** — a **DRAFT design proposal**, pending user design approval. **No production change, no implementation authorization.** Evidence base: PR #1033 (reviewed synthetic-only linkage comparison). ## Decisions already made by the user (2026-10-05) - **D1 — new registry method id** (e.g. `semisupervised-v2`); `semisupervised-v1` stays byte-unchanged; per-domain opt-in via `settings.clusteringMethod`; supported `(method, contractVersion)` matrix in Core, fail closed on unsupported combinations. No new envelope fields. - **D2 — rollout keeps the stored threshold** (`clustering_max_distance`, default 0.35). Recalibration toward ~0.45 is a separate, evidence-based follow-up (synthetic-only so far). Rollback also restores 0.35. - **D4 — scalability is a design gate** before any code: prove the scalable algorithm produces **exactly** the reference HAC partition (incl. stable tie-break), without a full N×N matrix (ISSUE-081). Alternatives (component decomposition, hybrid size cap) are evaluated inside the gate, not assumed. The spec-050 "ONE pass" wording must be re-read against the chosen approach. ## Explicitly open (user input required) - **Q3 — wholeGroup rule for B cards under average linkage.** Recommended (and reviewed): keep the gate *X qualifies iff every unlabeled member has some confirmed anchor of X within threshold adjacency* — anchors are read from the graph, **not** from HAC group membership, so an anchor partitioned elsewhere still qualifies. Mean-based identity attachment is NOT proposed (no reviewed evidence). ## Key content - Not a one-line linkage swap: HAC changes **both** the search strategy **and** anchor consumption (anchors are partitioned and group-gated instead of reused during greedy growth). Anchor-containing groups can emit C; confidence/qualityScore/keys change where group composition changes (that is the point — fewer wrong 2-face fragments; two-face clusters per se are acceptable). - Identity safety: explicit group-level confirmed-identity conflict check before each merge (removing cannot-link edges alone does not prevent a mean ≤ threshold merge across identities). - Rollback = future-job selection only; in-flight v2 jobs complete and persist (immutable job settings snapshot); supersede-exempt backing cards and tentative retarget/release (ADR 0102 amendment) are untouched — method-switch lifecycle handling is required planning. - Out of scope: weak extra pass, multi-pass restructuring, qualityScore refit, concurrent ranking changes, private-library benchmarking, legacy method changes. ## Verification Docs-only change (one new ADR JSON); CI scope detection will skip builds. Merging does not deploy anything.
docs(adr): ADR 0104 draft — constrained average-linkage as opt-in semisupervised-v2
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
8706555de1
DRAFT, status proposed, pending user design approval. Proposal only; no
production change and no implementation authorization.

Encodes the user's 2026-10-05 decisions: new registry method id (v1 byte-
unchanged, per-domain opt-in, supported method/contractVersion matrix),
rollout keeps stored clustering_max_distance default 0.35 (recalibration is
a separate evidence-based follow-up), scalability proven as a design gate
(exact-partition equivalence incl. tie-break, no full N x N matrix,
ISSUE-081) before any code.

Open question Q3 recorded: keep the reviewed wholeGroup anchor-adjacency
gate for B cards (recommended; anchors read from the graph, not HAC group
membership) vs a separate mean-based identity-attachment investigation.

Explicitly documents: not a one-line linkage swap (search strategy AND
anchor consumption change), HAC identity-conflict rule necessity, churn as
a measured risk, rollback = future-job selection with in-flight job handling
as required planning, out-of-scope list (no weak pass, no multi-pass, no
qualityScore refit, no private-library benchmarking).

Evidence base: PR #1033 reviewed synthetic-only linkage comparison.
docs(adr): 0104 — user confirms D3 wholeGroup gate (anchors within threshold of every member); Q3 closed, ADR stays draft
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
7ef2551a47
chris merged commit dec12b6cdd into main 2026-10-05 07:42:39 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1034
No description provided.