feat(api,people-worker): spec 045 P0-P2 — tentative stash auto-assign, no anchor growth #876

Merged
chris merged 8 commits from spec/045-stash-shadow-auto-assign-with-tentative-basis-and-no-anchor-growth into main 2026-09-25 22:33:50 +02:00
Owner

Problem

"Stash away" on people clusters (spec 039) silently auto-accepts future matched faces as state=accepted occurrences — including weak single-face candidates that bypass the spec 040 suggestion-only guard for stashed targets. Every accepted face is exported to the people worker as a matching anchor, so one wrong absorbed face compounds: centroid drift, distinct persons merging into one stash, and the UI hardcoding every stash face as "Confirmed" even though the owner never confirmed it.

Decision (ADR 0097)

Stash auto-assignments become tentative: a new confirmation-axis column class_instance_occurrences.assignment_basis (user_confirmed | auto_assigned). Auto-assigned faces absorb quietly (spec 039 promise preserved) but are never Confirmed and never worker anchors — the stash anchor set is its user_confirmed core. The axis is a column, not a new state enum value, so all lifecycle paths that supersede over state='accepted' (disband, dissolve, merge, split) keep covering auto-assigned faces. Stash decisions become insert-only (no face stealing). ADR: docs/adr/0097-stash-auto-assignments-are-tentative-and-never-anchors.json.

What P0–P2 do

P0 — migration V139__occurrence_assignment_basis.sql (additive, zero behavior change on apply):

  • assignment_basis TEXT NOT NULL DEFAULT 'user_confirmed' CHECK (... IN ('user_confirmed','auto_assigned')).
  • Partial index idx_class_instance_occurrences_auto_assigned_instance on (class_instance_id) WHERE assignment_basis='auto_assigned' (serves the P1 split and the P2 guard).
  • Provenance-driven, idempotent backfill: accepted occurrences whose review decision carries details.action='auto_accepted_stashed_target' on a currently stashed instance become auto_assigned; original stash-time faces (action='stashed') and all non-stash acceptances stay user_confirmed.
  • Count report: a DO block RAISE NOTICEs backfilled rows + total auto_assigned population for operator verification.
  • No legacy rows predate the action string: auto_accepted_stashed_target was introduced together with stashes (commit 8c324330, migration V128 / spec 039). No earlier action ever meant "stash auto-accept", so the backfill is complete and deterministic — any auto_accepted_existing_instance_candidate on a stashed instance (possible when an instance was stashed later) deliberately stays user_confirmed.

P1 — break the anchor loop

  • API enqueueOwnerFaceRecognitionJob now splits the export: stashed instances contribute only user_confirmed faces to assignments (anchors); their auto_assigned faces go to a new optional payload field tentativeAssignments. Unstashed instances keep the pre-045 shape.
  • Worker _handle_owner_face_recognition reads tentativeAssignments (missing field → empty): those faces join the assigned-faces set (excluded from clustering, weak suggestions and re-proposal — no churn loop) but never assignments (no centroids, no best-match math).
  • Deploy-skew guard in persistWorkerCandidates: an old worker that ignores the new field re-proposes faces already auto-assigned to the same stashed shadow. Those candidates are now superseded silently (no review decision) instead of being rejected as all_faces_already_assigned / lost-insert-race once per recognition run.

P2 — tentative auto-assign + insert-only stash

  • occurrenceBasis threaded through CandidateDecisionContext → acceptPendingCandidatesIntoExistingInstance → acceptCandidateOccurrences → insertOccurrenceNoSteal (H2 MERGE and PG ON CONFLICT arms) and upsertCandidateOccurrence. Every existing caller defaults to user_confirmed (zero behavior change); the stash shadow auto-accept passes auto_assigned.
  • decideCandidate stash branch: allowMove=false (insert-only). A stash never steals / re-homes a face already accepted on another instance; an all-conflict stale candidate fails loudly as 409 (IllegalStateException → ApiExceptionHandler), leaving the face where it was.
  • Churn-loop guard: the tentative auto-assign still marks the owner dirty (unchanged), but the absorbed face is accepted, so the reconcile scheduler reaches its steady-state branch and enqueues nothing — pinned by an explicit test (tentativeAutoAssignDoesNotArmRecognitionChurnLoop).

Skew matrix

API Worker Behavior
new new Anchor split honored; tentative faces never anchors, never re-proposed.
new old Old worker ignores tentativeAssignments and re-proposes auto-assigned faces → API supersedes the re-proposal silently (skew guard). No candidate/decision spam; assign remains auto_assigned.
old new Old API exports everything as assignments → new worker treats them as normal anchors (missing field → empty tentative set). No crash; behavior identical to status quo until the API rolls.
old old Unchanged.

Test evidence

  • API: rtk ./gradlew :apps:api:test --no-daemon → 1194 tests, 0 failures, 9 skipped (post-rebase on origin/main@dda31c88).
    • New AssignmentBasisMigrationPostgresTest (Testcontainers + real Flyway chain): backfill matches only the shadow action on stashed instances; stash-time/other-action/no-decision rows stay user_confirmed; re-run idempotent; default + CHECK + partial index + flyway_schema_history assertions.
    • New Spec045StashTentativePostgresTest: insert-only stash does not steal a face from another instance (409 + face stays), stash decision writes user_confirmed, shadow auto-accept writes auto_assigned, old-worker re-proposal superseded with no decision spam.
    • ClassInstanceDiscoveryServiceTest: shadow/weak auto-accept write auto_assigned; stash decision writes user_confirmed; skew re-proposal superseded silently; dissolve supersedes auto_assigned occurrences and releases faces (lifecycle safety); churn-loop guard.
    • PeopleDiscoveryPostCompletionListenerTest: export split incl. empty tentativeAssignments for unstashed owners.
  • People worker: PYTHONPATH=src python3 -m pytest tests -q → 103 passed (new tests/test_tentative_assignments.py: backward-compat missing field, no re-proposal, no anchor/centroid/weak-consensus contribution, no clustering-pool contribution) + python3 -m compileall clean.

Backfill count report

Verified via the migration test (seeded pre-V139 state): exactly the provenance-matched rows flip (auto_assigned total = 1 in the fixture scenario; unrelated actions/instances unchanged). On a live database V139 emits RAISE NOTICE 'V139 spec 045 assignment_basis backfill: N occurrence(s) backfilled ...; M total auto_assigned occurrence(s) after backfill' — run alembic-style via the normal Flyway deploy and read the migration log for N/M.

Follow-ups (explicitly out of scope for this PR)

  • P3 / T004: expose origin/basis in occurrence views (replace hardcoded "accepted"/Confirmed in ClassInstanceService), autoAssignedCount, web labels, filter stash-targeted cards from the carousel.
  • P4 / T005: dirty-mark on unstash/naming, block merge-into-stash, carousel requireUnstashedTarget guard.
  • Workers: once P3 lands, note that a new_person_cluster re-proposal (no existingInstanceId) during the old-worker skew window is not covered by the P1 guard — it stays pending. It disappears as soon as the worker is updated (tentative faces leave the unassigned pool); P3's carousel filtering should cover the residual display.
  • Pre-stash DBSCAN density chaining remains a separate worker-clustering spec (spec S4 out of scope). ISSUE-045 stale pending candidate sweep remains open.

Notes

  • No manual version bump: repo policy (docs/dev/versioning.md) is pipeline-computed versions — "Agents never manually set version".
  • No KNOWN_ISSUES.md entry (this is a tracked fix, spec 045).
  • Spec artifacts + ADR committed first: docs(spec): add spec 045 stash tentative basis + ADR 0097.
## Problem "Stash away" on people clusters (spec 039) silently auto-accepts future matched faces as `state=accepted` occurrences — including weak single-face candidates that bypass the spec 040 suggestion-only guard for stashed targets. Every accepted face is exported to the people worker as a matching anchor, so one wrong absorbed face compounds: centroid drift, distinct persons merging into one stash, and the UI hardcoding every stash face as "Confirmed" even though the owner never confirmed it. ## Decision (ADR 0097) Stash auto-assignments become **tentative**: a new confirmation-axis column `class_instance_occurrences.assignment_basis` (`user_confirmed` | `auto_assigned`). Auto-assigned faces absorb quietly (spec 039 promise preserved) but are **never Confirmed and never worker anchors** — the stash anchor set is its `user_confirmed` core. The axis is a column, not a new state enum value, so all lifecycle paths that supersede over `state='accepted'` (disband, dissolve, merge, split) keep covering auto-assigned faces. Stash decisions become insert-only (no face stealing). ADR: `docs/adr/0097-stash-auto-assignments-are-tentative-and-never-anchors.json`. ## What P0–P2 do **P0 — migration `V139__occurrence_assignment_basis.sql`** (additive, zero behavior change on apply): - `assignment_basis TEXT NOT NULL DEFAULT 'user_confirmed' CHECK (... IN ('user_confirmed','auto_assigned'))`. - Partial index `idx_class_instance_occurrences_auto_assigned_instance` on `(class_instance_id) WHERE assignment_basis='auto_assigned'` (serves the P1 split and the P2 guard). - Provenance-driven, idempotent backfill: accepted occurrences whose review decision carries `details.action='auto_accepted_stashed_target'` on a currently stashed instance become `auto_assigned`; original stash-time faces (`action='stashed'`) and all non-stash acceptances stay `user_confirmed`. - Count report: a `DO` block `RAISE NOTICE`s backfilled rows + total `auto_assigned` population for operator verification. - **No legacy rows predate the action string**: `auto_accepted_stashed_target` was introduced together with stashes (commit 8c324330, migration V128 / spec 039). No earlier action ever meant "stash auto-accept", so the backfill is complete and deterministic — any `auto_accepted_existing_instance_candidate` on a stashed instance (possible when an instance was stashed later) deliberately stays `user_confirmed`. **P1 — break the anchor loop** - API `enqueueOwnerFaceRecognitionJob` now splits the export: stashed instances contribute only `user_confirmed` faces to `assignments` (anchors); their `auto_assigned` faces go to a new optional payload field `tentativeAssignments`. Unstashed instances keep the pre-045 shape. - Worker `_handle_owner_face_recognition` reads `tentativeAssignments` (missing field → empty): those faces join the assigned-faces set (excluded from clustering, weak suggestions and re-proposal — no churn loop) but never `assignments` (no centroids, no best-match math). - Deploy-skew guard in `persistWorkerCandidates`: an old worker that ignores the new field re-proposes faces already auto-assigned to the same stashed shadow. Those candidates are now **superseded silently** (no review decision) instead of being rejected as `all_faces_already_assigned` / lost-insert-race once per recognition run. **P2 — tentative auto-assign + insert-only stash** - `occurrenceBasis` threaded through `CandidateDecisionContext → acceptPendingCandidatesIntoExistingInstance → acceptCandidateOccurrences → insertOccurrenceNoSteal` (H2 `MERGE` and PG `ON CONFLICT` arms) and `upsertCandidateOccurrence`. Every existing caller defaults to `user_confirmed` (zero behavior change); the stash shadow auto-accept passes `auto_assigned`. - `decideCandidate` stash branch: `allowMove=false` (insert-only). A stash never steals / re-homes a face already accepted on another instance; an all-conflict stale candidate fails loudly as 409 (`IllegalStateException` → `ApiExceptionHandler`), leaving the face where it was. - Churn-loop guard: the tentative auto-assign still marks the owner dirty (unchanged), but the absorbed face is `accepted`, so the reconcile scheduler reaches its steady-state branch and enqueues nothing — pinned by an explicit test (`tentativeAutoAssignDoesNotArmRecognitionChurnLoop`). ## Skew matrix | API | Worker | Behavior | |---|---|---| | new | new | Anchor split honored; tentative faces never anchors, never re-proposed. | | new | old | Old worker ignores `tentativeAssignments` and re-proposes auto-assigned faces → API supersedes the re-proposal silently (skew guard). No candidate/decision spam; assign remains `auto_assigned`. | | old | new | Old API exports everything as `assignments` → new worker treats them as normal anchors (missing field → empty tentative set). No crash; behavior identical to status quo until the API rolls. | | old | old | Unchanged. | ## Test evidence - **API**: `rtk ./gradlew :apps:api:test --no-daemon` → **1194 tests, 0 failures, 9 skipped** (post-rebase on `origin/main@dda31c88`). - New `AssignmentBasisMigrationPostgresTest` (Testcontainers + real Flyway chain): backfill matches only the shadow action on stashed instances; stash-time/other-action/no-decision rows stay `user_confirmed`; re-run idempotent; default + CHECK + partial index + `flyway_schema_history` assertions. - New `Spec045StashTentativePostgresTest`: insert-only stash does not steal a face from another instance (409 + face stays), stash decision writes `user_confirmed`, shadow auto-accept writes `auto_assigned`, old-worker re-proposal superseded with no decision spam. - `ClassInstanceDiscoveryServiceTest`: shadow/weak auto-accept write `auto_assigned`; stash decision writes `user_confirmed`; skew re-proposal superseded silently; dissolve supersedes `auto_assigned` occurrences and releases faces (lifecycle safety); churn-loop guard. - `PeopleDiscoveryPostCompletionListenerTest`: export split incl. empty `tentativeAssignments` for unstashed owners. - **People worker**: `PYTHONPATH=src python3 -m pytest tests -q` → **103 passed** (new `tests/test_tentative_assignments.py`: backward-compat missing field, no re-proposal, no anchor/centroid/weak-consensus contribution, no clustering-pool contribution) + `python3 -m compileall` clean. ## Backfill count report Verified via the migration test (seeded pre-V139 state): exactly the provenance-matched rows flip (`auto_assigned` total = 1 in the fixture scenario; unrelated actions/instances unchanged). On a live database V139 emits `RAISE NOTICE 'V139 spec 045 assignment_basis backfill: N occurrence(s) backfilled ...; M total auto_assigned occurrence(s) after backfill'` — run `alembic`-style via the normal Flyway deploy and read the migration log for N/M. ## Follow-ups (explicitly out of scope for this PR) - **P3 / T004**: expose `origin`/basis in occurrence views (replace hardcoded "accepted"/Confirmed in `ClassInstanceService`), `autoAssignedCount`, web labels, filter stash-targeted cards from the carousel. - **P4 / T005**: dirty-mark on unstash/naming, block merge-into-stash, carousel `requireUnstashedTarget` guard. - Workers: once P3 lands, note that a `new_person_cluster` re-proposal (no `existingInstanceId`) during the old-worker skew window is not covered by the P1 guard — it stays pending. It disappears as soon as the worker is updated (tentative faces leave the unassigned pool); P3's carousel filtering should cover the residual display. - Pre-stash DBSCAN density chaining remains a separate worker-clustering spec (spec S4 out of scope). ISSUE-045 stale pending candidate sweep remains open. ## Notes - No manual version bump: repo policy (`docs/dev/versioning.md`) is pipeline-computed versions — "Agents never manually set version". - No `KNOWN_ISSUES.md` entry (this is a tracked fix, spec 045). - Spec artifacts + ADR committed first: `docs(spec): add spec 045 stash tentative basis + ADR 0097`.
Spec 045 P0 / ADR 0097. New confirmation-axis column
class_instance_occurrences.assignment_basis (user_confirmed|auto_assigned),
partial index for the auto-assigned read pattern, and a provenance-driven
idempotent backfill: accepted occurrences whose review decision carries the
spec 039 shadow action auto_accepted_stashed_target on a currently stashed
instance become auto_assigned; stash-time original faces stay user_confirmed.
A DO block reports backfilled and total auto_assigned row counts via NOTICE.
H2 test schemas mirror the column.
Spec 045 P1 / ADR 0097. The owner_face_recognition payload now separates the
worker anchor set (assignments: user_confirmed faces; all accepted occurrences
of unstashed instances) from tentatively auto-assigned faces of stashed
shadows (tentativeAssignments). The worker treats tentative faces as assigned
(excluded from clustering, weak suggestions and re-proposals) but never as
anchors (no centroid or best-match contribution). Missing field degrades to
empty, so old API/worker deploy orders stay safe.
Spec 045 P2 / ADR 0097. occurrenceBasis is threaded through the unified accept
unit (CandidateDecisionContext -> acceptCandidateOccurrences ->
insertOccurrenceNoSteal both H2 MERGE and PG ON CONFLICT arms, plus
upsertCandidateOccurrence); every existing caller defaults to user_confirmed.
The shadow auto-accept writes auto_assigned for absorbed faces, so they are
never Confirmed and never anchors. Stash decisions become insert-only
(allowMove=false) - a stash never steals a face from another instance (409 on
an all-conflict stale candidate). Old-worker re-proposals of already
auto-assigned faces are superseded silently instead of producing rejected
candidate + review-decision spam. Tests cover the write paths, the skew
absorption, dissolve lifecycle safety on auto_assigned rows, and a churn-loop
guard proving no recognition re-enqueue per tentative assignment.
chore(spec): mark spec 045 tasks T001-T003 done
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 9s
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
b711c08594
docs(api,people-worker): document spec 045 assignment basis and payload contract
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 7m51s
639efb9823
Clarify the unified accept unit's occurrenceBasis propagation and the
owner_face_recognition payload split (assignments vs tentativeAssignments) in
the handler docstring. No behavior change.
docs(adr): record basis-flip ruling for manual moves (ADR 0097)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 4s
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
fa33250a24
Merge remote-tracking branch 'origin/main' into spec/045-stash-shadow-auto-assign-with-tentative-basis-and-no-anchor-growth
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 6s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 2m33s
33515e8399
chris merged commit 4bc6a42156 into main 2026-09-25 22:33:50 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!876
No description provided.