1 ADR 0008 v1 deployment topology service communication
Forgejo Actions edited this page 2026-09-25 16:38:07 +00:00

ADR 0008: v1 Deployment Topology, Service Communication, and ML Model Profiles

Status: proposed Date: 2026-05-10

Context

Memories is intended to run as a self-hosted/local-first photo and video system, but Christian explicitly wants a microservice-based deployment shape rather than one large application process. The service split must support separate operational concerns for the web UI, Core API authority, media processing, machine learning, and PostgreSQL while keeping the system understandable for a self-hoster.

Immich uses a client/server design where clients talk to the backend via REST APIs. Its machine-learning service is a separate Python/FastAPI container because model runtimes, hardware acceleration, memory use, and model caching differ substantially from the main server. Memories should learn from that separation without copying Immich's full operational stack or adding queue infrastructure prematurely.

v1 also requires both face recognition and object/scene/thing detection. Christian wants model choice for face and object recognition, similar in spirit to Immich, and does not want those capabilities hardcoded to one fixed model.

Decision

v1 deploys as a Docker Compose microservice stack with these first-class services:

Service Responsibility
memories-web SvelteKit/PWA web UI and public web entrypoint or static/server-rendered frontend container
memories-api Java/Spring Boot Core API, auth, authorization, domain state, upload commit, worker leasing, result acceptance, export/backup semantics
memories-worker-media Non-ML media work: metadata extraction, thumbnails, previews, video preview/transcode, artifact upload
memories-worker-ml ML work: face detection, face clustering/recognition, and object/scene/thing detection
memories-postgres Authoritative PostgreSQL runtime state

Optional deployment helpers such as Traefik/Caddy/reverse proxy, metrics, or backup sidecars may be documented later, but Docker Compose is the official v1 deployment path.

The public UI, browser clients, API/CLI clients, and future mobile clients communicate only with the Core API boundary. They do not call workers, ML services, PostgreSQL, or filesystem storage directly.

Service Communication

v1 uses boring, inspectable communication:

  • Browser/PWA and CLI/API clients call Core API over HTTP REST/JSON using documented OpenAPI contracts.
  • memories-web calls or proxies to memories-api; the frontend does not encode authorization or persistence truth.
  • memories-worker-media and memories-worker-ml use the Core API worker protocol over HTTP REST/JSON.
  • memories-api talks to PostgreSQL via JDBC.
  • Workers read media only through job-scoped Core API grants: Core API download for enforceable remote access, or read-only library/stored-objects mounts as trusted same-host optimization.
  • Workers upload binary outputs as Core API job-scoped artifacts; Core API validates and promotes final derived assets or metadata records.

v1 does not require Redis, RabbitMQ, Kafka, Temporal, Kubernetes, or direct worker database access. A PostgreSQL-backed job table with Core API polling/claim is the accepted v1 queue boundary.

ML Model Profiles

Model choice is a first-class Core API configuration concept.

v1 supports Super Admin/operator-configurable ML model profiles for at least these task classes:

  • face_detection
  • owner_face_recognition / group_face_recognition Core-owned grouping profiles
  • visual_concept_detection for objects, scenes, and things

A model profile records task class, model name, model version, provider/runtime, model source, enabled/default state, and structured configuration. Worker jobs reference the selected model profile. Worker results include model profile/version provenance so historical results remain inspectable when the operator later changes models.

ML model artifacts and runtime caches live in worker-local model cache volumes. They are not authoritative backup state. The authoritative state is the selected model profile configuration, job/result provenance, and accepted metadata stored by the Core API.

Changing a model profile does not silently rewrite historical metadata. New or explicitly reprocessed jobs use the new profile and can supersede older candidate/accepted worker metadata with provenance.

Consequences

  • The UI/backend separation is clean: performance must come from good API shape, pagination, preview URLs, caching, and same-origin routing/proxying rather than bypassing the Core API.
  • ML dependencies, model caches, and hardware acceleration can evolve independently from the Core API and media worker.
  • Object/scene/thing detection becomes v1 scope, increasing implementation and validation cost.
  • Model selection is not hardcoded; it is inspectable, backed up, exported, and represented in worker results.
  • A dedicated queue service is deferred, reducing operational burden but requiring careful PostgreSQL job indexing and lease handling.

Deferred

v1 defers Kubernetes, Redis/RabbitMQ/Kafka/Temporal, public internet hardening as the baseline, direct browser-to-worker communication, untrusted worker execution, per-user model selection, automatic model download UI, multi-GPU scheduling, and polished smart-search/object-browsing UX beyond the validation scope.

Revisit Triggers

Revisit this decision when worker throughput exceeds PostgreSQL polling comfortably, GPU scheduling becomes necessary, object storage is introduced, public internet exposure becomes a supported baseline, or model management requires user-facing download/update workflows.