7 Operator Play Review Instance
Forgejo Actions edited this page 2026-10-22 20:21:23 +00:00
This file contains ambiguous Unicode characters

This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.

Google Play review instance (memories-review.wiwie.dk)

A small, permanently running, publicly reachable demo deployment of Memories with a seeded play-review account and generated demo media. It exists for exactly one reason: Google Play reviewers must be able to sign in. They cannot create accounts, and a "self-hosted, bring your own server, no credentials" answer to the sign-in declaration gets the app setup rejected and blocks every future update. The Play listing answer in docs/play-store-listing.md ("keine Demo-Zugangsdaten möglich") is superseded by this instance for the sign-in declaration itself.

  • Compose file: deploy/compose.review.yaml (separate stack, name memories-review, own volumes/network, no host ports; routing to the instance is handled by the operator's edge setup).
  • Review credentials live in the deploy env file (${MEMORIES_DEPLOY_ENV_FILE}, see the review section in deploy/example.env).
  • Reviewer-facing login: [email protected] with MEMORIES_REVIEW_SEED_PASSWORD from the deploy env file.
  • Scope wiring: scripts/ci/detect-scopes.sh sets review=true for changes to deploy/compose.review.yaml and scripts/review/** (and marks the api/web/postgres/worker-media/worker-ml/worker-people scopes, whose images the review stack reuses); .forgejo/workflows/build-deploy.yml runs the Deploy review instance job on main when the review scope fires.

Networking (no manual step)

The instance is reachable at https://memories-review.wiwie.dk. Edge routing/TLS and LAN access are pre-configured by the operator; details are deliberately kept out of this public repository (see the operator's private infrastructure notes). No DNS or routing step is needed when recreating the stack.

Everything else — containers, database, account, demo media — is fully automated by CI.

How seeding works

On every review deploy (the Deploy review instance CI job, running on the memories-deploy runner):

  1. review-postgres, review-api, review-web, review-worker-media, review-worker-ml, review-worker-people start (prebuilt production images, docker compose up -d --wait). Starting either model-hungry worker first runs review-model-seed (their service_completed_successfully dependency), and the job re-runs that one-shot explicitly afterwards.
  2. review-model-seed copies the ML model files and the people (face-detection/embedding) model files from the production model volumes into the review-owned model volumes (idempotent; see the ML worker and people worker sections below).
  3. review-reset-admin runs the repo's supported password-reset CLI (PasswordResetCli) inside the memories-api image to pin the bootstrap admin's password (MEMORIES_REVIEW_ADMIN_PASSWORD). On a fresh database the API's admin bootstrap (MEMORIES_ADMIN_USERNAME) created that super-admin with a random one-time password at first startup; the reset makes it usable by automation.
  4. review-reset-reviewer pins the play-review account's password the same way (no-op with a logged warning until the account's first creation by the seed).
  5. review-seed (Python + Pillow, stdlib HTTP + CDN downloads) talks to the API only via the public API:
    • configures the 7 library storage paths under /srv/library (fresh review databases start unconfigured; uploads would 503-gate),
    • flips registration to open, registers [email protected], flips registration back to closed (reviewers must not be able to self-register),
    • uploads ~30 photos through the same endpoints the web client uses (POST /api/upload-sessions → /{id}/files → /{id}/commit). By default these are real photographs (see "Photo source" below), each re-encoded with EXIF DateTimeOriginal (dates spread over ~2 years) and GPS coordinates around Kerteminde/Fyn (Denmark) so timeline, map, and places show content.
    • Idempotent: reads GET /api/library-stats first; when the library already holds the target photo count it exits 0 with "already seeded". After a partial seed it uploads only the missing photos (fresh file salt, no dedup collisions).

Photo source (real Unsplash photos, synthetic fallback)

Since the curated-list change the seed default (MEMORIES_REVIEW_PHOTO_SOURCE=unsplash) is real photography, not generated gradients:

  • scripts/review/photo_ids.txt ships a curated list of 32 Unsplash photos (<photoId> <cdnPath> per line; ~10 people/faces shots — family, friends, portraits — plus landscape/nature/city/pets variety, visually varied, nothing suggestive). License: the Unsplash License — free to use for commercial and non-commercial purposes, no attribution required (https://unsplash.com/license). Each download is attributed implicitly through the photo page https://unsplash.com/photos/<photoId> listed in that file.
  • The seed downloads each photo from the public CDN (images.unsplash.com, no API key), then re-encodes it through Pillow: the CDN metadata is stripped, EXIF orientation is baked into the pixels, and the demo EXIF is injected (DateTimeOriginal/DateTimeDigitized spread over ~2 years, GPS around Fyn, plausible camera Make/Model — iPhone 14 Pro / Pixel 7 / Canon EOS R6). Output JPEG quality ~85, max dimension 2560 px, so upload and worker processing stay fast.
  • Fallback: per-photo download failures are skipped; when fewer than min(20, photos needed) downloads succeed (offline runner, CDN blocked), the whole batch falls back to the original deterministic Pillow generator so the review stack still seeds (log line falling back to SYNTHETIC).
  • Knobs: MEMORIES_REVIEW_PHOTO_SOURCE=unsplash|synthetic (default unsplash), MEMORIES_REVIEW_PHOTO_IDS (path override for the curated list; the deploy mounts it read-only next to seed.py via the same CI staging step), MEMORIES_REVIEW_PHOTO_COUNT (default 30).
  • If the CDN list rots (IDs/paths 404 over time), re-verify a replacement set against https://images.unsplash.com/<cdnPath>?... and update photo_ids.txt via PR — the file documents the last verification date.
  • Resolved — people/face processing is live: the review stack now runs review-worker-people (see the people worker section below); face detection/recognition jobs are processed and the People surface populates for the reviewer demo.

EXIF/GPS/thumbnail processing is done by review-worker-media after the commit — which is why the media worker is part of the review stack. Give it a few minutes after the first deploy before judging the map/timeline.

Re-seeding (drain + fresh upload)

When: after switching MEMORIES_REVIEW_PHOTO_SOURCE or changing the seed logic in scripts/review/. The seed is idempotent — it tops up to the target count and reports "already seeded" once reached — so it keeps whatever the previous seed produced; the old library must be drained first.

How: on the deploy host, run scripts/review/reseed.sh where the internal review API resolves (http://review-api:8080, i.e. inside the review compose env; dependencies: bash + curl). It reads the same credentials as the seed from the deploy env file (MEMORIES_REVIEW_ADMIN_PASSWORD, MEMORIES_REVIEW_SEED_PASSWORD; optional MEMORIES_REVIEW_API_URL / MEMORIES_REVIEW_ADMIN_EMAIL overrides), logs in as the admin and as play-review (two separate curl cookie jars — the API rejects requests without the session cookie), then drains the play-review library through the public API: pages the whole timeline, trashes every item, empties the trash, and verifies the timeline is empty (retries once, then fails listing the remaining ids — a repeated /api/media/trash/empty can report {"purged":0} while items stay active). Then run the seed one-shot:

docker compose -f deploy/compose.review.yaml up --exit-code-from review-seed review-seed

The ML models do NOT need re-seeding: review-model-seed is idempotent and now asserts the required model files exist after the copy (it fails the deploy loudly instead of letting review-worker-ml crash-loop on missing models). Wiring the library drain into the deploy workflow is deliberately NOT automated — re-seeding is a rare operator action.

ML worker (semantic search + content detection)

review-worker-ml processes the semantic_embedding and visual_concept_detection jobs the API queues for seeded and uploaded demo photos. Without it those jobs queue forever and the review UI shows stuck pending work; semantic search queries degrade to plain lexical search (the API falls back when the query-embedding endpoint, MEMORIES_ML_QUERY_EMBEDDING_URL — now http://review-worker-ml:8081/embed-text — is unreachable).

The worker mirrors the production memories-worker-ml service:

  • Models: ~600 MB (CLIP 582M + object detection 13M; the facial-recognition files in the production ML cache are deliberately not copied — this worker does not use them). The worker does NOT download models — it hard-fails per job when files are missing. The one-shot review-model-seed copies them from the production model volume (memories_memories-ml-model-cache, referenced as an external volume) into the review-owned memories-review_review-ml-model-cache volume. The copy enumerates each required model file and skips files already present, so it is idempotent and near-free on every deploy once seeded; the copy runs once (~600 MB + 184 MB within the same disk) and then never again unless the volume is deleted. On a host without the production stack the compose up itself fails at setup ("external volume ... not found") — loudly, before anything starts — which is the right failure mode for a reviewer-facing instance.
  • Memory: the worker is the second-largest consumer in the review stack after the API (ONNX runtime + CLIP); it runs alongside the operator's production instance, and resource caps keep the review footprint small. If the host gets tight, this is the review service to stop first — semantic search in the demo degrades to plain text search, which is acceptable for a review window.
  • Storage scoping: reads /srv/memories-review read-only with rslave propagation — the review library floor only, not the whole /srv tree production binds.
  • Auth: the shared review worker token (MEMORIES_REVIEW_WORKER_TOKEN); the review API aliases MEMORIES_WORKER_TOKEN_ML to it, so all review workers authenticate against one registered credential (the full prod capability list on one credential is the established review pattern).

People worker (face detection + recognition)

review-worker-people processes the discovery_people (face detection) and owner_face_recognition / owner_face_embedding_backfill jobs the API queues after the media worker extracts face scopes. Without it those jobs queue forever and the People surface stays empty — the photos still show on timeline/map, but the reviewer demo loses an entire app area. The seed photos deliberately include ~10 people/faces shots so this surface has content.

The worker mirrors the production memories-worker-people service, with the same three review adaptations as review-worker-ml:

  • Models: ~184 MB (face detection 17M + face embedding 167M). The worker does NOT download models — it hard-fails per job when files are missing. The same review-model-seed one-shot copies them from the production memories_memories-people-model-cache volume (external reference) into the review-owned memories-review_review-people-model-cache volume and asserts both files (face-detection/det_10g.onnx, face-embedding/w600k_r50.onnx) exist after the copy.
  • Memory: OMP/OPENBLAS/MKL threads are capped at 2 (matching the ONNX session caps in the worker), but ONNX face detection/embedding is not free — this is the third-largest memory consumer in the review stack. If the host gets tight after the ML worker, stopping this one degrades the demo least (People surface drains, everything else keeps working).
  • Storage scoping / auth: identical to the ML worker — /srv/memories-review read-only with rslave propagation, shared MEMORIES_REVIEW_WORKER_TOKEN (the review API aliases MEMORIES_WORKER_TOKEN_PEOPLE to it).

Videos: deliberately not seeded

The seed container is python:3.12-slim (no ffmpeg). Adding a second one-shot ffmpeg service for 3 short clips was judged not worth the moving parts — photos are the review must-have. If reviewers ever need video, generate clips with an ffmpeg-image one-shot service alongside review-seed and upload them with the same endpoints (.mp4 is a supported format).

Rotating the review password

Change the value in the deploy env file (MEMORIES_REVIEW_SEED_PASSWORD, optionally MEMORIES_REVIEW_ADMIN_PASSWORD) on the deploy runner, then re-run the review deploy (CI: re-run the Deploy review instance job, or push any scripts/review/**/deploy/compose.review.yaml change). The reset containers pin the new passwords before the seed runs; the seed's login then uses the new value. Reviewer-visible login in the Play Console declaration must be updated in the same sitting.

Play Console: sign-in declaration (paste-ready)

App access → Authentication → Add credentials:

  • Any reviewers are granted access to all parts of the app? → Yes

  • Credential 1:

    • Username: [email protected]
    • Password: current value of MEMORIES_REVIEW_SEED_PASSWORD (deploy env file — do not paste the placeholder from deploy/example.env)
  • Name (credentials/instructions label): Memories demo server – review account

  • Instructions (steps field):

    Sign in with the username/password above on the login screen. The app connects to our demo server at https://memories-review.wiwie.dk, which is pre-loaded with a small demo photo library (timeline, map and places views contain sample content). No account creation is needed and no payment or personal data is involved. All app features (timeline, search, albums, map, settings) are usable read-mostly; uploads from the reviewer session are optional and land in the same demo library.

  • Server URL (if the form asks for the backend separately): https://memories-review.wiwie.dk

  • Full-access note: the account is a normal (non-admin) user account with full access to the demo library on that server — sufficient for every reviewer-reachable screen.

After merge + deploy, verify https://memories-review.wiwie.dk loads, log in once with the reviewer credentials, and only then submit/refresh the Play sign-in declaration.

Threat/privacy notes

  • The demo library contains free-license stock photography (Unsplash License, no personal data) re-encoded with synthetic EXIF; when the CDN is unreachable the seed falls back to fully synthetic generated gradients. No real personal photos anywhere.
  • The review API/DB/worker tokens are independent of production values; review uses its own Postgres volume and library floor (/srv/memories-review on the host).
  • Registration is closed outside the brief seed window; the reviewer account is a plain user (the super_admin bootstrap account is not exposed to reviewers).