fix(sync): history derives from lifecycle state — legacy deletions visible, honest attribution #1367

Merged
chris merged 1 commit from fix/sync-history-legacy into main 2026-10-24 23:52:46 +02:00
Owner

Fixes the production bug from the 0.10.5 AVD report: the backup screen correctly showed "18 bewusst nicht in Memories" (change feed = pure lifecycle_state) but the view history answered "Momentan ist nichts aus der Mediathek entfernt". Also fixes oracle finding F5 (bulk-attribution lie). Lane 3 of 4; base = origin/main 05c432db, no overlap with the other lanes' files.

Root cause (verified, not re-diagnosed)

RemovedLibraryHistoryQueryService.PAGE_SQL built its base set from FROM media_deletion_decisions d JOIN media_items mi … — an INNER JOIN on the V178 provenance table. The user's items were deleted 2026-10-23, before the sync server (PR-D/V178) deployed 2026-10-24 → no decision rows → history empty while the change feed correctly reported permanently_deleted and the client shadowed all 18 rows.

A. Server — lifecycle-driven history, provenance as enrichment

  • Base set = the item's live state: owner-scoped media_items with lifecycle_state IN ('trashed','permanently_deleted'); media_deletion_decisions is LEFT JOINed for attribution only. A missing decision row is a legacy entry, still listed.
  • removedAt COALESCE: decision.decided_at → permanently_deleted_at → trashed_at → updated_at — legacy rows carry the item's real removal timestamp; keyset pagination orders on the same coalesced expression (cursor wire format unchanged; cursors are ephemeral per pull/poll anyway).
  • Restore/reinstate unchanged: flips lifecycle to active, entry drops out (verified by the existing restore test, still green).

B. Attribution ruling (F5)

  • emptyTrash actor: TrashService.emptyTrash(ownerId, actorUserId, confirm) now records the authenticated session user as actor_user_id on every purged item, decided_by='user' (C6 arbitration unchanged). Ruling, documented on the method: the original NULL rationale ("no single actor in endpoint contract") misread the contract — the authenticated owner session IS who pressed the button, exactly as for the single-item permanent delete, which always recorded its actor. actor_user_id NULL is reserved for "no human decided this" (retention sweep/unknown), never "a human decided this but we collapsed who".
  • No automated purge path exists today: trash_retention_until is displayed, never enforced (V178 header says the same); no scheduled sweeper calls emptyTrash or any purge path. retention_sweep is defined for a future sweeper.
  • Attribution discriminator (add-only, S5 discipline extended): user requires POSITIVE evidence (decided_by='user' AND actor recorded); sweep on the retention_sweep marker; legacy for no decision row / user decision with unrecorded actor / unknown future decided_by. displayName non-null exactly when user.

C. Android copy (de/en/da, intent-not-failure register)

Attribution → copy mapping (attributionShapeFor, pure + tested):

attribution copy
user (+name) „Entfernt von " / "Removed by " / "Fjernet af " (existing)
sweep „Automatisch entfernt" / "Removed automatically" / "Fjernet automatisk" (existing)
legacy „Entfernt" / "Removed" / "Fjernet" (NEW — neutral no-actor fact; claims neither a person nor automation)
null (older server) / unknown future value / user with missing name same neutral no-actor fallback — never a crash

The card's title still shows the live state ("Im Papierkorb"/"Endgültig entfernt"); no "du hast", no false "Automatisch" (S7a/S8). Purged entries keep the no-button + honest copy from PR #1364; trashed entries keep the working "Wiederherstellen" — no regression.

Contracts & specs

  • OpenAPI (specs/001-v1-baseline/contracts/core-api.openapi.yaml): RemovedLibraryHistoryEntry.attribution (user|sweep|legacy, required, add-only + unknown-fallback documented), displayName nullability tightened, endpoint description amended (legacy entries listed; base set = lifecycle).
  • Spec 065 via spec-query (sanctioned writer), following cod-12's pattern:
    • S11: AMENDED (2026-10-24, legacy-history fix + honest attribution) — lifecycle-derived base set, absent-provenance entries stay listed, user|sweep|legacy vocabulary with neutral no-actor client duty, emptyTrash session-actor ruling.
    • S3: PR #1364's drafted auth bullet (media:upload for intents, media:read for the read feeds, 403 request-level) + S5: its request-level scope note.
    • S2: PR #1362's cursor-repair text (client MAY reset cursor to 0, epoch MUST be kept) — drafted from the merged commit (PR body was empty).
    • spec-md/specs/065/spec.md regenerated with the official exporter and verified: my export is byte-identical to a clean-main export apart from spec 065 (the other export drift on main is pre-existing and left to CI, as in #1365).
  • Spec 044 / INV-2 draft untouched (PR-3 lane).

Tests (all green)

  • :apps:api:test --rerun-tasks — 198 suites / 2310 tests, 0 failures, 0 errors (9 pre-existing skips). New/extended: legacy deletion (decision row deleted, the pre-V178 shape) → listed with attribution=legacy + coalesced trashed_at; user decision with stripped actor → legacy; mixed legacy/new keyset pagination across two pages; owner scoping (existing); trashed+restored drops out (existing); emptyTrash records the session actor on every row + history renders user + display name; controller pins the actor plumbing (MediaControllerTest), bulk semantics + idempotence + owner scope (TrashServiceTest), change-feed allocation (SyncChangeFeedIntegrationTest).
  • :core:test --rerun-tasks — 12 suites / 143 tests, 0 failures.
  • :app:testDebugUnitTest --rerun-tasks — 62 suites / 669 tests, 0 failures. New: attribution→copy-shape mapping table incl. legacy, older-server null, unknown future value, degraded user-without-name; existing history VM tests stay green (15/15 in the suite).

KNOWN_ISSUES

ISSUE-200 (high/mitigated/area:sync, ref [fix/sync-history-legacy]) — history invisible for pre-provenance deletions + bulk-attribution lie, fixed.

Scope discipline

No core engine/binder changes, no intent-queue changes, no trash screen redesign, no schema migration (V178 shape unchanged — only the query and the recorded actor changed).

Fixes the production bug from the 0.10.5 AVD report: the backup screen correctly showed "18 bewusst nicht in Memories" (change feed = pure `lifecycle_state`) but the view history answered "Momentan ist nichts aus der Mediathek entfernt". Also fixes oracle finding **F5** (bulk-attribution lie). Lane 3 of 4; base = origin/main 05c432db, no overlap with the other lanes' files. ## Root cause (verified, not re-diagnosed) `RemovedLibraryHistoryQueryService.PAGE_SQL` built its base set from `FROM media_deletion_decisions d JOIN media_items mi …` — an INNER JOIN on the V178 provenance table. The user's items were deleted 2026-10-23, **before** the sync server (PR-D/V178) deployed 2026-10-24 → no decision rows → history empty while the change feed correctly reported `permanently_deleted` and the client shadowed all 18 rows. ## A. Server — lifecycle-driven history, provenance as enrichment - **Base set = the item's live state**: owner-scoped `media_items` with `lifecycle_state IN ('trashed','permanently_deleted')`; `media_deletion_decisions` is LEFT JOINed for attribution only. A missing decision row is a **legacy** entry, still listed. - **removedAt COALESCE**: `decision.decided_at → permanently_deleted_at → trashed_at → updated_at` — legacy rows carry the item's real removal timestamp; keyset pagination orders on the same coalesced expression (cursor wire format unchanged; cursors are ephemeral per pull/poll anyway). - Restore/reinstate unchanged: flips lifecycle to `active`, entry drops out (verified by the existing restore test, still green). ## B. Attribution ruling (F5) - **emptyTrash actor**: `TrashService.emptyTrash(ownerId, actorUserId, confirm)` now records the **authenticated session user** as `actor_user_id` on every purged item, `decided_by='user'` (C6 arbitration unchanged). Ruling, documented on the method: the original `NULL` rationale ("no single actor in endpoint contract") misread the contract — the authenticated owner session IS who pressed the button, exactly as for the single-item permanent delete, which always recorded its actor. `actor_user_id NULL` is reserved for "no human decided this" (retention sweep/unknown), never "a human decided this but we collapsed who". - **No automated purge path exists today**: `trash_retention_until` is displayed, never enforced (V178 header says the same); no scheduled sweeper calls emptyTrash or any purge path. `retention_sweep` is defined for a future sweeper. - **Attribution discriminator (add-only, S5 discipline extended)**: `user` requires POSITIVE evidence (`decided_by='user'` AND actor recorded); `sweep` on the `retention_sweep` marker; `legacy` for no decision row / user decision with unrecorded actor / unknown future `decided_by`. `displayName` non-null exactly when `user`. ## C. Android copy (de/en/da, intent-not-failure register) Attribution → copy mapping (`attributionShapeFor`, pure + tested): | attribution | copy | |---|---| | `user` (+name) | „Entfernt von <name>" / "Removed by <name>" / "Fjernet af <name>" (existing) | | `sweep` | „Automatisch entfernt" / "Removed automatically" / "Fjernet automatisk" (existing) | | `legacy` | **„Entfernt"** / **"Removed"** / **"Fjernet"** (NEW — neutral no-actor fact; claims neither a person nor automation) | | `null` (older server) / unknown future value / `user` with missing name | same neutral no-actor fallback — never a crash | The card's title still shows the live state ("Im Papierkorb"/"Endgültig entfernt"); no "du hast", no false "Automatisch" (S7a/S8). Purged entries keep the no-button + honest copy from PR #1364; trashed entries keep the working "Wiederherstellen" — no regression. ## Contracts & specs - **OpenAPI** (`specs/001-v1-baseline/contracts/core-api.openapi.yaml`): `RemovedLibraryHistoryEntry.attribution` (`user|sweep|legacy`, required, add-only + unknown-fallback documented), `displayName` nullability tightened, endpoint description amended (legacy entries listed; base set = lifecycle). - **Spec 065** via `spec-query` (sanctioned writer), following cod-12's pattern: - S11: `AMENDED (2026-10-24, legacy-history fix + honest attribution)` — lifecycle-derived base set, absent-provenance entries stay listed, `user|sweep|legacy` vocabulary with neutral no-actor client duty, emptyTrash session-actor ruling. - S3: PR #1364's drafted auth bullet (`media:upload` for intents, `media:read` for the read feeds, 403 request-level) + S5: its request-level scope note. - S2: PR #1362's cursor-repair text (client MAY reset cursor to 0, epoch MUST be kept) — drafted from the merged commit (PR body was empty). - `spec-md/specs/065/spec.md` regenerated with the official exporter and verified: my export is byte-identical to a clean-main export apart from spec 065 (the other export drift on main is pre-existing and left to CI, as in #1365). - Spec 044 / INV-2 draft untouched (PR-3 lane). ## Tests (all green) - `:apps:api:test --rerun-tasks` — **198 suites / 2310 tests, 0 failures, 0 errors** (9 pre-existing skips). New/extended: legacy deletion (decision row deleted, the pre-V178 shape) → listed with `attribution=legacy` + coalesced `trashed_at`; user decision with stripped actor → legacy; mixed legacy/new keyset pagination across two pages; owner scoping (existing); trashed+restored drops out (existing); `emptyTrash` records the session actor on every row + history renders `user` + display name; controller pins the actor plumbing (MediaControllerTest), bulk semantics + idempotence + owner scope (TrashServiceTest), change-feed allocation (SyncChangeFeedIntegrationTest). - `:core:test --rerun-tasks` — **12 suites / 143 tests, 0 failures**. - `:app:testDebugUnitTest --rerun-tasks` — **62 suites / 669 tests, 0 failures**. New: attribution→copy-shape mapping table incl. `legacy`, older-server `null`, unknown future value, degraded `user`-without-name; existing history VM tests stay green (15/15 in the suite). ## KNOWN_ISSUES **ISSUE-200** (`high`/`mitigated`/`area:sync`, ref `[fix/sync-history-legacy]`) — history invisible for pre-provenance deletions + bulk-attribution lie, fixed. ## Scope discipline No core engine/binder changes, no intent-queue changes, no trash screen redesign, no schema migration (V178 shape unchanged — only the query and the recorded actor changed).
fix(sync): history derives from lifecycle state — legacy deletions visible, honest attribution
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 4s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Deploy review instance (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Successful in 3m47s
Build and Deploy / Run scoped PR tests (push) Successful in 7m59s
f6860f6f57
Production bug (0.10.5 AVD report): the backup screen correctly shadowed the
18 bulk-purged items (change feed = pure lifecycle_state) but the view history
answered "nothing is currently removed" — RemovedLibraryHistoryQueryService
built its base set from an INNER JOIN on the V178 provenance table, and items
deleted on 2026-10-23 (before the sync server deployed on 2026-10-24) have NO
decision rows. Overlaps oracle finding F5.

Server:
- RemovedLibraryHistoryQueryService: base set is now the ITEM's live
  lifecycle_state (owner-scoped trashed/permanently_deleted); provenance is a
  LEFT JOIN for attribution enrichment only. removedAt is COALESCEd
  decision->permanent->trashed->updated_at so legacy rows carry the item's
  real removal timestamp; keyset pagination moves to the coalesced timestamp
  (cursor wire format unchanged). Restore still drops entries (lifecycle flips
  to active).
- Add-only attribution discriminator per entry: user | sweep | legacy —
  user needs POSITIVE provenance (decided_by='user' AND actor recorded);
  sweep on the retention_sweep marker; everything without a WHO (no decision
  row, user decision with unrecorded actor, unknown future decided_by) is
  legacy = the honest third state. displayName non-null exactly when user.
- F5 attribution fix: TrashService.emptyTrash now takes the acting session
  user and records it as actor_user_id on every purged item (decided_by
  stays 'user', C6 arbitration unchanged). Ruling documented on the method:
  the authenticated owner session IS the actor; actor NULL is reserved for
  "no human decided this" — no automated caller of emptyTrash exists today
  (trash_retention_until is displayed, never enforced).

Client:
- RemovedLibraryEntryDto carries the lenient attribution field (absent on
  older servers -> null). RemovedEntryUi.attribution + pure
  attributionShapeFor mapping: user+name -> "Entfernt von <name>", sweep ->
  "Automatisch entfernt", legacy/null/unknown -> NEW neutral no-actor copy
  "Entfernt" (de) / "Removed" (en) / "Fjernet" (da) — claims neither a
  person nor automation (S7a/S8). Purged no-button honesty (ISSUE-193) and
  trashed restore unchanged; unknown values never crash (S5 discipline).

Contracts/specs:
- OpenAPI RemovedLibraryHistoryEntry: attribution enum user|sweep|legacy
  (required, documented add-only), endpoint description amended.
- Spec 065 amended via spec-query (S11 legacy-history fix + attribution
  vocabulary + emptyTrash actor ruling; S2 cursor-repair MAY per PR #1362;
  S3 auth bullet + S5 request-level scope note per PR #1364's drafted
  amendment texts); spec-md/specs/065 regenerated with the official
  exporter and verified as the ONLY export drift (my export is byte-identical
  to a clean-main export apart from 065).

Tests: ReinstateProvenanceIntegrationTest — legacy entry (decision row
deleted, pre-V178 shape) listed with attribution=legacy + coalesced
timestamp, actorless user decision degrades to legacy, mixed
legacy/new keyset pagination, emptyTrash records the session actor with
history-level user attribution; TrashServiceTest + MediaControllerTest pin
the new actor plumbing. Android: attribution->copy-shape mapping table incl.
legacy + older-server null + unknown future value + degraded user-without-
name fallback.

KNOWN_ISSUES: ISSUE-200.

Validation: :apps:api:test --rerun-tasks 198 suites / 2310 tests, 0
failures (9 pre-existing skips); :core:test 12 suites / 143 tests, 0
failures; :app:testDebugUnitTest --rerun-tasks 62 suites / 669 tests, 0
failures.
chris merged commit a8d132d463 into main 2026-10-24 23:52:46 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1367
No description provided.