feat(api): stale upload-session detection — spec 070 #1519

Merged
chris merged 5 commits from spec/070-stale-upload-sessions into main 2026-11-15 16:10:07 +01:00
Owner

Implements spec 070: server-side detection of dead upload sessions (client died mid-upload — battery drain, Doze kill, crash) that otherwise keep their non-terminal state forever and render 'Upload läuft…' indefinitely in the web dialog.

Backend job (new UploadSessionStaleScheduler)

  • @Scheduled(fixedDelay = 60000), single-instance conventions like DiscoveryReprocessScheduler.
  • One guarded UPDATE: UPDATE upload_sessions SET status='stale', updated_at=:now WHERE status IN ('created','receiving','committing') AND updated_at < :cutoff (cutoff = now − timeout).
  • Property memories.upload.stale-timeout, default 10m, env MEMORIES_UPLOAD_STALE_TIMEOUT (Upload properties record extended, compact-constructor default like MediaToken.ttl).
  • Logging: one info line per batch that transitioned >0 sessions (count + oldest updated_at age); silent when 0.

Scope corrections vs. spec text

  1. Spec says status IN ('receiving','processing') — no session-level 'processing' status exists, and commit_requested is a derived-only read status (never persisted). The job matches the persisted non-terminal set created/receiving/committing. Legacy file-level hashing recovery stays with UploadRecoveryService (untouched).
  2. The job sets updated_at = now() on transition — no DB trigger maintains the column; every writer updates it manually.

Termination safety & UX paths

  • Active clients never match: every received file refreshes session updated_at (guarded receive in UploadReceiveService), so any matched row saw no activity for the whole timeout.
  • Terminal and already-stale rows are excluded — transition is idempotent (verified by a double-tick test).
  • Cancel enabled for persisted stale: stale joins CANCELLABLE_STATUSES in UploadActivityService — both clients expose discard for stale (web isDiscardableSession, Android cancel list); without this every discard on a persisted-stale session 409s (the incident's main UX path).
  • Commit cannot resurrect stale: stale added to markCommitStarted/markPartialCommitFinished CASE lists and finishSession's NOT IN list; requireCommittableStatus now rejects stale up front with a clean 409 instead of a rollback at finishSession.
  • Android BackupEngine treats stale as terminal (TERMINAL_SESSION_STATUSES): without it the engine reuses a stale session, 409s on first receive, and wastes a run re-uploading received-uncommitted files.

Web — no changes

Recon confirmed the full render chain already handles persisted stale: uploadActivity.ts derivedStatus passthrough → uploadSessionServer 'Unterbrochen' label → stale-only isDiscardableSession → dialog discard button. The derived 15-minute staleness (STALE_AFTER) still covers rendering before the job's next tick — two mechanisms, one outcome.

Schema

No Flyway migration: upload_sessions.status is free TEXT (no CHECK constraint), confirmed in V001__baseline_domain.sql.

Tests

API (all green): UploadSessionStaleSchedulerTest (5: old non-terminal → stale + updated_at refreshed; recent → untouched; terminal/already-stale untouched incl. idempotence; custom timeout boundary with strict < semantics; empty-table no-op) + UploadActivityTest extended (+2: cancel on persisted-stale succeeds; commit on persisted-stale does not resurrect). :apps:api:test --tests '*Upload*': 68 tests, 0 failures.
Android (all green): BackupEngineTest +1 ('stale open batch with server-marked stale status is discarded and replaced'): 50 tests, 0 failures.

Implements spec 070: server-side detection of dead upload sessions (client died mid-upload — battery drain, Doze kill, crash) that otherwise keep their non-terminal state forever and render 'Upload läuft…' indefinitely in the web dialog. ## Backend job (new UploadSessionStaleScheduler) - `@Scheduled(fixedDelay = 60000)`, single-instance conventions like DiscoveryReprocessScheduler. - One guarded UPDATE: `UPDATE upload_sessions SET status='stale', updated_at=:now WHERE status IN ('created','receiving','committing') AND updated_at < :cutoff` (cutoff = now − timeout). - Property `memories.upload.stale-timeout`, default `10m`, env `MEMORIES_UPLOAD_STALE_TIMEOUT` (`Upload` properties record extended, compact-constructor default like `MediaToken.ttl`). - Logging: one info line per batch that transitioned >0 sessions (count + oldest updated_at age); silent when 0. ## Scope corrections vs. spec text 1. Spec says `status IN ('receiving','processing')` — no session-level 'processing' status exists, and `commit_requested` is a derived-only read status (never persisted). The job matches the persisted non-terminal set **created/receiving/committing**. Legacy file-level `hashing` recovery stays with UploadRecoveryService (untouched). 2. The job sets `updated_at = now()` on transition — no DB trigger maintains the column; every writer updates it manually. ## Termination safety & UX paths - Active clients never match: every received file refreshes session updated_at (guarded receive in UploadReceiveService), so any matched row saw no activity for the whole timeout. - Terminal and already-stale rows are excluded — transition is idempotent (verified by a double-tick test). - **Cancel enabled for persisted stale**: `stale` joins CANCELLABLE_STATUSES in UploadActivityService — both clients expose discard for stale (web isDiscardableSession, Android cancel list); without this every discard on a persisted-stale session 409s (the incident's main UX path). - **Commit cannot resurrect stale**: `stale` added to markCommitStarted/markPartialCommitFinished CASE lists and finishSession's NOT IN list; requireCommittableStatus now rejects stale up front with a clean 409 instead of a rollback at finishSession. - Android **BackupEngine treats stale as terminal** (TERMINAL_SESSION_STATUSES): without it the engine reuses a stale session, 409s on first receive, and wastes a run re-uploading received-uncommitted files. ## Web — no changes Recon confirmed the full render chain already handles persisted stale: uploadActivity.ts derivedStatus passthrough → uploadSessionServer 'Unterbrochen' label → stale-only isDiscardableSession → dialog discard button. The derived 15-minute staleness (STALE_AFTER) still covers rendering before the job's next tick — two mechanisms, one outcome. ## Schema No Flyway migration: upload_sessions.status is free TEXT (no CHECK constraint), confirmed in V001__baseline_domain.sql. ## Tests API (all green): UploadSessionStaleSchedulerTest (5: old non-terminal → stale + updated_at refreshed; recent → untouched; terminal/already-stale untouched incl. idempotence; custom timeout boundary with strict `<` semantics; empty-table no-op) + UploadActivityTest extended (+2: cancel on persisted-stale succeeds; commit on persisted-stale does not resurrect). `:apps:api:test --tests '*Upload*'`: 68 tests, 0 failures. Android (all green): BackupEngineTest +1 ('stale open batch with server-marked stale status is discarded and replaced'): 50 tests, 0 failures.
Scheduled job (60s fixedDelay, DiscoveryReprocessScheduler conventions)
marks non-terminal upload sessions stale when updated_at is older than
memories.upload.stale-timeout (default 10m, env
MEMORIES_UPLOAD_STALE_TIMEOUT). Matches persisted statuses
created/receiving/committing — the spec text said receiving/processing,
but no session-level 'processing' status exists and 'commit_requested'
is derived-only — and refreshes updated_at (no DB trigger maintains it;
every writer does so manually).

Safety: active clients refresh updated_at per received file (guarded
receive), so they never match; terminal and already-stale rows are
excluded, making the transition idempotent. Legacy file-level 'hashing'
recovery stays with UploadRecoveryService. A persisted 'stale' passes
through deriveStatus unchanged, so the web dialog renders interrupted;
the derived 15-minute staleness still covers rendering before the job's
next tick (two mechanisms, one outcome).

The Upload properties record gains staleTimeout (compact-constructor
default 10m, mirroring MediaToken.ttl); fixture call sites updated.
'cancelled' joins CANCELLABLE_STATUSES so owner discard keeps working on
a persisted-stale session — both clients expose discard for stale (web
isDiscardableSession, Android cancel list) and without this the
interrupted-upload UX 409s, the incident's main path.

Commit guards extended so a racing commit cannot resurrect a dead
session: markCommitStarted/markPartialCommitFinished CASE lists and
finishSession's NOT IN list keep 'stale' sticky; requireCommittableStatus
now rejects stale up front with a clean 409 instead of discovering the
conflict at finishSession (which would roll the commit transaction back).

Tests: cancel on persisted-stale succeeds; commit on persisted-stale
conflicts and leaves the session stale.
fix(android): treat stale upload sessions as terminal in backup engine
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Deploy review instance (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m49s
Build and Deploy / Android unit tests and debug APK (push) Successful in 4m44s
0245c7c442
TERMINAL_SESSION_STATUSES gains 'stale': without it the backup engine
reuses a session the server's stale job has marked interrupted (spec
070), gets 409 on the first file receive, and wastes a whole run
re-uploading received-uncommitted files. A stale probe now discards the
batch and starts a fresh session. UploadActivityViewModel needs no
change (its cancel lists already include stale).
docs(spec): spec 070 T001-T003 done; T004 manual acceptance pending post-merge
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Deploy review instance (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m26s
Build and Deploy / Android unit tests and debug APK (push) Successful in 2m2s
c80e4fde5e
chris merged commit ba9f60d740 into main 2026-11-15 16:10:07 +01:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1519
No description provided.