ADR 0006: v1 Storage Backends, Layout, Integrity, and Garbage Collection
Status: proposed Date: 2026-05-10
Context
Memories must preserve original media bytes while staying operable for self-hosters. Users may place the library on SSD, HDD, CIFS, or NFS, and future deployments may want S3/object storage. At the same time, v1 should avoid turning storage abstraction into broad operational complexity before the local filesystem path is correct.
Decision
v1 models storage through a StorageBackend boundary but implements only the filesystem backend.
SSD, HDD, CIFS, and NFS are supported as filesystem deployments when the operator provides required semantics: read-after-write consistency, atomic rename within a directory, stable metadata/locking behavior, and durable writes.
S3/object storage is modeled as a future backend behind backend-neutral storage keys, not a v1 implementation requirement.
Stored Object Layout
SHA-256 is the v1 canonical content hash.
Filesystem stored object keys use:
stored-objects/sha256/<first-2>/<next-2>/<sha256>.<normalized-ext>
Example:
stored-objects/sha256/ab/cd/abcdef0123456789abcdef0123456789abcdef0123456789abcdef0123456789.jpg
Rules:
contentHashAlgorithm = sha256.contentHashis lowercase 64-character hex.- Extension is operator/tooling friendly, not identity.
- Dedup key is
(storageBackendId, contentHashAlgorithm, contentHash, byteSize). - Original filename and source path remain owner/provenance metadata, not storage identity.
Write Rules
Core API writes upload bytes into upload-staging first. During import commit, Core API writes to a temporary file in the target final directory, verifies size and SHA-256, atomically renames to the final key where supported, fsyncs where practical, and only then records authoritative database state.
Existing final stored object keys are never overwritten. Final stored object bytes are never mutated.
The accepted failure mode is an orphaned unreferenced file. A database reference to missing original bytes is a preservation issue.
Integrity
The system supports integrity checks for:
- existence;
- byte size;
- SHA-256 hash;
- invalid stored object paths/keys;
- orphan files;
- quarantined files.
Preservation-impacting issues create visible ProcessingIssue records with storage_integrity class.
Garbage Collection
StoredObject bytes can be physically deleted only when no active, trashed/restorable, pending-delete, or active-worker references remain. GC marks objects gc_pending, waits a grace period, then performs a final reference recheck before unlinking.
Orphan final files are quarantined or retained for operator inspection before deletion. v1 does not auto-adopt orphan files into the database.
Derived Assets
Derived assets are cache and use generation-versioned keys:
derived/v1/<derived-type>/<media-id-prefix>/<media-id>/<generation-version>/<variant>.<ext>
Derived records include source stored object/hash snapshots so they can be marked stale and regenerated when generation rules change.
Workers do not write final derived paths directly in v1. Binary worker outputs are uploaded as Core API job-scoped artifacts; Core API validates and promotes accepted artifacts into final derived/v1/... keys.
Consequences
- v1 remains boring and inspectable for self-hosters.
- Future S3/object storage can be added without changing domain identity.
- CIFS/NFS support is possible but documented as dependent on filesystem semantics, not guaranteed for every mount configuration.
- Integrity and GC become explicit operational workflows.
Revisit Triggers
Revisit this decision when adding object storage, multi-library storage tiers, remote workers writing artifacts directly, cloud backup targets, or content-hash migration.
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference