feat(api): cause-centric processing operations for the activity dialog #1397

Merged
chris merged 4 commits from feat/processing-operations into main 2026-10-25 23:20:45 +01:00
Owner

Summary

Generalizes domain_reprocess_runs (V181) into a cause-centric processing-operation ledger so the activity dialog can show the ACTION that caused processing (upload, settings change, manual reprocess, ...) as a top-level entry instead of process types.

  • V181 migration: adds cause (CHECK-constrained: upload / settings_change / manual_reprocess / domain_onboarding / backfill / scheduled / retry_failed / recluster / domain_reset), requested_by_user_id, label; domain_key now nullable. Existing rows keep their meaning via the manual_reprocess default.
  • New ProcessingOperationService: begin/complete/recordCompleted/progress/finish with the supersede invariant generalized (domain causes supersede per owner+domain; non-domain per owner+cause+label; superseded → failed). UserDiscoveryService delegates through it; run-id addressing fixed for loop heartbeats.
  • Instrumented trigger sites (previously invisible as causes): MediaMaintenanceService owner-wide resets + backfills (settings_change, born completed with real counts), SearchService semantic backfill (backfill), ProcessingIssueService retry-failed (retry_failed), onboarding backfill (domain_onboarding), scheduled reprocess (scheduled), people reclustering (recluster). Uploads stay synthesized; per-media resets record nothing.
  • Read path: /api/processing-summary operations now carry additive cause + causeLabel fields, key reprocess:<domain> / op:<id>; ordering ranks scheduled rows below user-action causes (LIMIT 60 → rank → 10 → newest-first) so per-minute scheduler runs cannot crowd out user actions. Stale-running→failed (2h) and the 7d upload window unchanged. failedTotal stays 0 (TODO; per-op worker_jobs grouping is the historical CPU-spike shape, out of scope by design).

Old web bundles and Android are unaffected (additive fields only; Android does not consume this endpoint).

Validation

  • ./gradlew :apps:api:test — 2336 tests, 0 failures (incl. new ProcessingOperationServiceTest 6, extended ProcessingSummaryServiceTest 20, PeopleClusteringResetPostgresTest validating V181 on real Postgres + full Flyway chain)
## Summary Generalizes `domain_reprocess_runs` (V181) into a cause-centric processing-operation ledger so the activity dialog can show the ACTION that caused processing (upload, settings change, manual reprocess, ...) as a top-level entry instead of process types. - **V181 migration**: adds `cause` (CHECK-constrained: upload / settings_change / manual_reprocess / domain_onboarding / backfill / scheduled / retry_failed / recluster / domain_reset), `requested_by_user_id`, `label`; `domain_key` now nullable. Existing rows keep their meaning via the `manual_reprocess` default. - **New `ProcessingOperationService`**: begin/complete/recordCompleted/progress/finish with the supersede invariant generalized (domain causes supersede per owner+domain; non-domain per owner+cause+label; superseded → failed). `UserDiscoveryService` delegates through it; run-id addressing fixed for loop heartbeats. - **Instrumented trigger sites** (previously invisible as causes): MediaMaintenanceService owner-wide resets + backfills (settings_change, born completed with real counts), SearchService semantic backfill (backfill), ProcessingIssueService retry-failed (retry_failed), onboarding backfill (domain_onboarding), scheduled reprocess (scheduled), people reclustering (recluster). Uploads stay synthesized; per-media resets record nothing. - **Read path**: `/api/processing-summary` operations now carry additive `cause` + `causeLabel` fields, key `reprocess:<domain>` / `op:<id>`; ordering ranks scheduled rows below user-action causes (LIMIT 60 → rank → 10 → newest-first) so per-minute scheduler runs cannot crowd out user actions. Stale-running→failed (2h) and the 7d upload window unchanged. failedTotal stays 0 (TODO; per-op worker_jobs grouping is the historical CPU-spike shape, out of scope by design). Old web bundles and Android are unaffected (additive fields only; Android does not consume this endpoint). ## Validation - `./gradlew :apps:api:test` — 2336 tests, 0 failures (incl. new ProcessingOperationServiceTest 6, extended ProcessingSummaryServiceTest 20, PeopleClusteringResetPostgresTest validating V181 on real Postgres + full Flyway chain)
V181 adds cause/requested_by_user_id/label to domain_reprocess_runs and
makes domain_key nullable, so the table can carry every processing
operation cause instead of only manual reprocesses.

New ProcessingOperationService owns the write path: begin/complete/
recordCompleted/updateProgress/finish with the owner+domain supersede
invariant generalized (domain causes supersede per domain, non-domain
per owner+cause+label; superseded rows become 'failed'). Manual
reprocesses and the onboarding backfill now record through the shared
writer; the reprocess run id is returned from the insert so loop
progress/finish updates address the same row.
Previously invisible causes now create operation rows with real enqueued
counts:
- MediaMaintenanceService owner-wide resets + backfills (settings_change,
  labels 'Face detection reset', 'Thing detection reset', 'Plant detection
  reset', 'Previews reset', 'Missing dimensions backfill', 'Video previews
  backfill', 'Perceptual hashes backfill') — per-media actions and empty
  resets deliberately record nothing
- SearchService.backfillSemanticEmbeddings (backfill)
- ProcessingIssueService.reprocessFailedMedia (retry_failed, same
  transaction as the requeued jobs)
- UserDiscoveryService onboarding backfill (domain_onboarding)
- DiscoveryReprocessScheduler per-owner scheduled batches (scheduled);
  the media selector now also selects mi.owner_id to attribute the batch
- PeopleClusteringResetService (recluster, records even with zero
  enqueues since the reconcile scheduler does the follow-up work)
operations() now selects the cause columns: domain-scoped rows keep the
legacy 'reprocess:<domain>' key and '<display name> reprocess' label,
non-domain rows use 'op:<run id>' keys and carry the stored English
label. New additive fields cause + causeLabel let the web render causes
('Upload von Bildern', 'Einstellung geändert: ...') instead of process
types; old bundles keep working off the unchanged label field.

Ordering: newest-first, but 'scheduled' rows rank below user-action
causes during selection (fetch LIMIT 60 -> rank -> cut 10) so the
per-minute scheduler cannot crowd user actions out. Stale-running->failed
(2h), the synthesized upload batch, and the 7d window are unchanged.
failedTotal stays 0 with a TODO: per-operation worker_jobs groupings are
prohibitively expensive (the historical CPU-spike shape).
test(api): wire ProcessingOperationService into upload test fixtures
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Deploy review instance (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m45s
7b3780fadf
UserDiscoveryService gained a constructor dependency; the upload test
seams construct it explicitly over their H2 fixtures.
chris merged commit 44a941ab20 into main 2026-10-25 23:20:45 +01:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1397
No description provided.