Forgejo Deployment Pipeline
Memories uses Forgejo Actions to build Docker Compose service images and deploy
the local self-hosted stack. The workflow lives in
.forgejo/workflows/build-deploy.yml.
Runner
The deployment workflow targets a dedicated Forgejo runner label:
memories-deploy
This runner is intentionally separate from the general docker/DIND runner. It
has access to the host Docker daemon so it can build images and restart the same
Docker Compose stack that the operator runs manually.
Current local runner setup on the deployment host:
container: memories-deploy-runner
image: data.forgejo.org/forgejo/runner:11
runner name: memories-deploy
label: memories-deploy:docker://docker:28-cli
scope: repo-specific for chris/memories
config directory: /home/chris/git/memories-deploy-runner
workspace: /home/chris/git/memories-deploy-runner/workspace
host Docker socket: /var/run/docker.sock
host source mount: /home/chris/git:/home/chris/git
The runner job containers use network: host. Workflow checkouts therefore use
the public Forgejo URL (https://git.wiwie.dk/...) rather than the internal
Compose hostname gitea, which is not resolvable from host networking.
Important runner config details:
- Keep
container.docker_host: unix:///var/run/docker.sock. - Do not also mount
/var/run/docker.sockthroughcontainer.options; the runner already mounts it for the job container whendocker_hostis set. - Keep
/home/chris/gitmounted into job containers so host Docker can access the checked-out Compose build context. - If the Docker socket group changes, restart the runner with the matching
--group-add <docker-socket-gid>.
Useful runner checks:
docker ps --filter 'name=memories-deploy-runner'
docker logs --tail 80 memories-deploy-runner
stat -c '%g' /var/run/docker.sock
Workflow Triggers
The workflow runs on:
- every branch push
- manual
workflow_dispatch - nightly schedule (
02:17 UTC)
Branch pushes run scoped tests for CI feedback. Deployments and Docker image
builds are gated to main only. Manual and nightly runs execute the full test
suite without building or deploying images.
Change Detection And Test Scopes
The changes job runs scripts/ci/detect-scopes.sh, computes changed paths from
the pushed commit range, and emits booleans for each deployable service:
apiwebworker_mediaworker_ml
Service-specific changes build only that service image on main. Broad
operational or shared files build all images, including:
.forgejo/workflows/build-deploy.yml- deploy runtime configuration such as
deploy/compose*.yamlanddeploy/example.env - root Gradle settings/build files
gradle/*- app version touchpoints such as
apps/web/package.json
The same script also emits test-scope booleans. Branch pushes use these scoped checks instead of running the full suite every time:
- People API changes compile the API and run People API tests.
- General API changes run API tests.
- Web changes run
npm run checkand Playwright tests. - CLI changes run CLI build/tests.
- ML worker changes run ML unit tests.
- Docs/spec-only changes run no runtime tests.
Risky or broad paths fall back to broad smoke checks on branch pushes, not the full suite. These paths include workflow/script changes, Compose/runtime deploy configuration, root Gradle settings, database migrations, API auth/security/sharing/worker-protocol paths, and unknown paths. Deploy markdown runbooks are treated as docs-only. The fallback checks compile JVM services, run the web type/build check, run CLI build/tests, and run ML unit tests. Full JVM and Playwright suites are reserved for nightly/manual runs.
App version touchpoints such as apps/web/package.json mark all service images
as affected for main deployment, but run web-scoped checks on branch pushes so
normal runtime PRs do not lose the benefit of scoped CI solely because they bump
the release version.
On main, the workflow runs fast smoke checks for changed service areas before
building and deploying affected images. The full suite is intentionally reserved
for nightly/manual runs so main deploys stay fast while the slower confidence
gate still runs regularly.
Build Jobs
Build jobs are separate so Forgejo shows per-service status:
Build API imageBuild web imageBuild media worker imageBuild ML worker image
Each build uses Docker Compose from deploy/ with the production environment
file path supplied by the workflow:
docker compose --env-file "$MEMORIES_DEPLOY_ENV_FILE" -f compose.yaml build <service>
MEMORIES_DEPLOY_ENV_FILE currently points to:
/home/chris/git/memories-deploy-runner/.ci-secrets/deploy.env
This is a stable host path shared by both deploy runners
(memories-deploy, memories-deploy-runner-2). Runner workspaces are
ephemeral per run, so the env file must not live inside a checkout.
The Compose file still defaults to local .env when the variable is not set, so
manual local commands keep working.
Android CI Image
The android job runs from a dedicated image instead of downloading the
Android SDK and Node.js on every run:
- Dockerfile:
scripts/ci/android-image/Dockerfile - Base
gradle:8.13-jdk21plus Node.js 20 and the Android SDK (cmdline-tools, platform 37, build-tools 37, licences accepted) baked in. - Registry:
git.wiwie.dk/chris/memories/android-ci(:<version>and:latest).
detect-scopes.sh emits android_ci_image=true only when
scripts/ci/android-image/* changes; the build-android-ci-image job then
rebuilds and pushes the image on main. The debug keystore is deliberately NOT
part of the image (ISSUE-121) — the job provisions it at run time from
ANDROID_DEBUG_KEYSTORE_BASE64.
Forgejo pulls the job container image before any step runs, so the android
job authenticates that pull through container.credentials
(PACKAGE_PUSH_TOKEN); a docker login step could never do it. Caveat:
act_runner reuses a locally present image instead of pulling, so a stale
host-side :latest would shadow a rebuilt image for up to 7 days (until the
cleanup prune drops it) — build-android-ci-image therefore docker rmis all
local tags of the image after a successful push to force the next job to pull
(best-effort, with a warning on failure). To force a rebuild without touching
the Dockerfile, change any file under scripts/ci/android-image/ — every path
in that directory sets android_ci_image=true.
Version And Image Tags
The pipeline derives the application version from apps/web/package.json. Keep
that version aligned with the release-policy touchpoints before merge.
Tag rules:
main:MEMORIES_APP_VERSION=<version>- branch builds:
MEMORIES_APP_VERSION=<version>-<branch>-<shortsha>
Branch tags are still computed for inspectability, but branch pushes no longer
build service images by default. main builds use the release version tag.
The API container also receives MEMORIES_APP_VERSION as an environment override
from Compose so runtime version metadata matches the deployed image tag instead
of an older value in deploy/.env.
Memories service images built by Compose carry Docker labels:
dev.memories.project=memoriesdev.memories.service=<compose-service-name>
These labels let the cleanup job prune only unused Memories images created by this pipeline instead of deleting unrelated self-hosted service images.
Deploy Jobs
Deploy jobs run only on main and only for services that were rebuilt.
Deployment order:
Deploy APIstarts or verifiesmemories-postgres, then recreatesmemories-apiand waits for health.Deploy web,Deploy media worker, andDeploy ML workerdepend on the API deploy and recreate their own containers with--no-deps --no-build.
The workflow uses Compose like this for service deployment:
docker compose --env-file "$MEMORIES_DEPLOY_ENV_FILE" -f compose.yaml up -d \
--force-recreate --no-deps --no-build --wait --wait-timeout 300 <service>
Expected Status Patterns
On a branch push:
Detect changed servicessucceeds.Run scoped PR testsruns the selected scoped checks, or broad smoke fallback checks for risky paths.- Build and deploy jobs are skipped.
On a main push:
Detect changed servicessucceeds.Run main smoke checksruns fast checks for affected service areas.- Affected build jobs run after smoke checks.
- Matching deploy jobs run.
- Unaffected service deploy jobs are skipped.
On a nightly or manual run:
Run full test suiteruns JVM, web, CLI, and ML tests.- Build and deploy jobs are skipped.
Docker Cleanup
The Clean Docker build artifacts job runs after successful affected builds and
deploys. It is intentionally conservative:
- It prints
docker system dfbefore and after cleanup. - It prunes unused Memories images older than seven days using the
dev.memories.project=memorieslabel. - It prunes BuildKit cache older than 72 hours while keeping up to
8GBof cache. - It does not run
docker system prune. - It does not prune volumes.
- It cannot remove images used by running containers.
Current old Memories images that predate the labels are not removed by the workflow automatically. Operators may clean those manually after verifying which release tags they want to keep.
Troubleshooting
If a job fails before the first step, inspect runner logs and Forgejo action logs. Local action logs are stored compressed by Forgejo, for example:
/data/gitea/actions_log/chris/memories/<prefix>/<task-id>.log.zst
Common failures seen during setup:
Duplicate mount point: /var/run/docker.sock: remove the explicit socket mount from runnercontainer.options; keepcontainer.docker_hostonly.Could not resolve host: gitea: job container is using host networking; use the public Forgejo URL for checkout.- Build succeeds but deploy does not change the running container tag: confirm the
deploy job exported
MEMORIES_APP_VERSION=<release_version>and that Compose is using the expected env file.
Check the deployed stack after a main run:
docker ps --filter 'name=memories-' --format 'table {{.Names}}\t{{.Image}}\t{{.Status}}'
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference