ADR 0117: Server-authoritative change feed with commit-ordered sequence cursor
Status: accepted Date: 2026-10-24
Context
Clients (Android app, web) and the backend currently have no defined sync contract: the Android app (spec 044) reconciles purely against local Room state, so a server-side trash/purge of a backed-up item is invisible to the device and the file gets silently re-uploaded on the next run (resurrection, violating spec 065 INV-2). Sharing library state across one admin plus N family members (specs 055, 065) needs every device to converge on the same authoritative state without interactive conflict handling.
Options Considered
| Option | Summary | Benefits | Costs / Risks |
|---|---|---|---|
| A | Clients are authoritative (multi-master, CRDT/merge) | Offline writes everywhere | Huge complexity; ownership/deletion semantics (ADR 0009: deletion/trash/permanent-delete authority is Core-API-owned) become unenforceable; unbounded reconciliation surfaces |
| B | Event-log table (append-only domain events), clients replay events | Full history, natural feed | Duplicates truth: the row's current state already lives in media_items; two sources drift; retention questions for an event log whose only consumer is sync |
| C | Per-device server-side sync state (server tracks each device's cursor) | Server can push exact deltas | Server-side per-device bookkeeping for what is client-local state; device lifecycle (reinstall, second device, lost device) becomes server state |
| D | Server-authoritative state feed: single authority (Postgres/Core API per ADR 0009), clients hold a cursor into a commit-ordered per-row change_seq (chosen) | Boring: one counter column, keyset pagination, replay-from-0 always possible via soft-delete rows; deletion authority stays enforceable | Every media-state writer must allocate a seq in-transaction; no push, clients poll |
Decision follows ADR 0009: the Core API is the non-pluggable authority for deletion/trash/permanent-delete semantics and authoritative metadata acceptance - so library state cannot be client-authored. RFC 8620's /changes + SetError model (5.2/5.3/6.1), RFC 6578's sync-token resync rules and RFC 7162's monotonic-counter rule supply the protocol shape; Memories is a REST-native profile of them, not a JMAP implementation (upload-session pipeline and RFC 9457 conventions are kept).
Decision
Option D. The backend is ALWAYS the source of truth for the user's library state (Postgres/Core API per ADR 0009); clients hold only a cursor and a projection.
- Cursor, not clocks: every media_items row carries a BIGINT change_seq allocated in the SAME transaction as every media-state mutation (commit-ordered, global, monotonic, never wall-clock per RFC 7162 §3.1). GET /api/sync/changes?cursor=N serves keyset pagination over it.
- State, not events: the row's LATEST change_seq entry is the truth. Transition history stays in audit_events; NO event-log table is introduced for sync.
- No per-device server sync state: cursors are client-held; the server is stateless with respect to which device has consumed what. A per-owner epoch counter (bumped on clear-library/DB-restore) invalidates all cursors at once; an epoch mismatch or invalid cursor sends the client into an automatic full re-list from 0 (RFC 6578 §3.1).
- Tombstones forever: no sync-specific tombstone table and no retention window; the existing lifecycle_state soft-delete rows ARE the tombstones (trash = UPDATE, permanent delete is the only entry that leaves the library and still emits a change). Replay from cursor 0 remains possible indefinitely.
- No CRDT / multi-master (rejects Option A): conflict handling is the explicit spec 065 matrix (CAS via expectedLifecycleState, dedupe-on-identical-bytes per RFC 8620 §6.1, filename auto-resolve on the backup path), not convergence machinery.
- No push as correctness mechanism (rejects implicit Option C extension): polling IS the contract; SSE is only a later accelerator that triggers an earlier pull, never the carrier of required state.
- Per-item rejects, request-level ProblemDetail: intents answer JMAP-style (RFC 8620 §5.3) with {clientIntentId, outcome, reason, state} inside HTTP 200; the reason-code vocabulary is an ADD-ONLY FOREVER contract that clients translate to plain German copy, unknown codes falling back to generic text.
Consequences
- INV-1 (stated consequence): sync and server-side deletions NEVER modify local original files - MediaStore is untouched, ever, including after purge; server state changes only move the app's shadow state (spec 065 S6). A client is a projection; it has no authority to touch originals.
- HIGHEST-CONSEQUENCE IMPLEMENTATION DETAIL: every media-state writer must allocate a change_seq in its transaction - both the DB trigger (belt) and every service write path (braces: upload commit, trash, restore, permanent delete, accepted metadata edits). A writer that forgets allocation produces an invisible gap: clients never learn about that mutation until a full re-list. Covered by a transactionality test per write path (spec 065 PR-A acceptance).
- The reject-reason vocabulary (spec 065 S5) is an add-only contract: codes are never renamed or removed; clients must render unknown codes as generic fallback copy, so the server can extend the vocabulary without breaking old clients.
- This is a JMAP-PROFILE, not conformance: RFC 8620 provides the /changes and SetError models we adapt; media data model, trash-vs-purge, reinstate and name-conflict semantics are self-defined (no existing standard covers them).
- Polling cost is bounded by keyset pagination + small per-owner change rates; a later SSE accelerator must remain optional - correctness may never depend on event delivery.
- Tombstones are permanent rows: media_items grows monotonically; purge reclaims bytes, not rows. Acceptable for a household-scale library; revisit trigger would be a documented archive/compaction requirement (ADR 0006 territory).
- Epoch bumps (clear-library, DB restore per ADR 0003) force full re-lists on every device; acceptable because they are rare, destructive events where re-listing is the only safe behavior.
- Deletion provenance is REQUIRED for both display and override arbitration: every trash/purge writes a durable decision record (acting user or retention sweep, with when). The same record feeds the per-user removed-from-library history (spec 065 S11: WHO + when, neutral attribution, per-entry revert) and arbitrates reinstates (override_rejected_explicit_deletion). Without it, neither a trustworthy user-visible deletion history nor defensible override rejection is implementable.
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference