Table of contents
- ADR 0044: People as Generic Class-Occurrence Domain + Separate People Worker
- Context
- Decision
- 1. People becomes a generic class_occurrence domain
- 2. Region markers and embeddings become first-class discovery concepts
- 3. Separate People Worker
- 4. Backward compatibility
- Migration Path
- Phase 1: People Worker (this ADR)
- Phase 2: Discovery → Class-Instance Promotion
- Phase 3: Legacy Cleanup
- Consequences
ADR 0044: People as Generic Class-Occurrence Domain + Separate People Worker
Status: accepted Date: 2026-06-04
Context
The People pipeline currently uses two separate systems:
-
Legacy face detection pipeline (
face_detection→face_annotations→face_embeddings→class_instance_occurrences): A separate, non-discovery path that produces face annotations with bounding boxes and embeddings, then feeds them into the generic identity layer. -
Generic discovery platform (
discovery_results→class_instances): The target architecture per ADR 0020, where all domains use the same pipeline.
The legacy face detection pipeline exists because when it was built, the generic discovery platform didn't support region markers (bounding boxes) or embeddings. Now that the platform has matured, the separation is arbitrary:
plants_imageusesclass_occurrencewithwhole_image_classifier— a worker classifies the whole image and submits results through the discovery pipeline.peopleusesface_detectionas a completely separate capability that bypasses the discovery pipeline entirely, even though face detection is conceptually just anotherclass_occurrencedomain where the "class" is "person" and the worker also produces bounding boxes and embeddings.
This creates several problems:
- Two parallel systems for the same conceptual operation (detecting things in photos)
- The ML worker carries face detection code that could be in a dedicated people worker
- New domains that need region markers (plant detection, pet detection, object detection) would each need to build their own separate pipeline instead of reusing the discovery platform
- The
face_annotationsandface_embeddingstables are tightly coupled to the face detection pipeline, making it hard to generalize
Decision
1. People becomes a generic class_occurrence domain
The people domain moves from the legacy face detection pipeline to the generic discovery platform:
Current:
face_detection job → face_annotations → class_instance_occurrences (manually linked)
owner_face_recognition job → face_embeddings → class_instances (manually linked)
Target:
discovery_people job → discovery_candidate (with bounding box + embedding payload)
→ Core validates against allowedClasses (person)
→ Core stores discovery_results
→ Class-instance promotion: discovery_results → class_instances (generic lifecycle)
The discovery result payload for people includes:
{
"domainKey": "people",
"schemaVersion": 1,
"candidates": [
{
"classKey": "person",
"confidence": 0.92,
"scope": "region",
"boundingBox": {"x": 0.1, "y": 0.2, "width": 0.3, "height": 0.4},
"embedding": [0.01, 0.02, ...],
"inferenceStrategy": "face_detector"
}
]
}
2. Region markers and embeddings become first-class discovery concepts
The discovery platform gains support for:
scope: "region"— result includes a bounding box (x, y, width, height normalized to 0-1)embedding— optional float array for similarity matching (face recognition, duplicate detection)
These are not face-specific — any domain can use region markers (plant detection, pet detection, object detection) or embeddings (visual similarity, duplicate detection).
3. Separate People Worker
A new workers/people/ worker replaces the face detection capabilities in the ML worker:
| ML Worker (current) | People Worker (target) |
|---|---|
face_detection |
discovery_people with inferenceStrategy: "face_detector" |
owner_face_recognition |
discovery_people with inferenceStrategy: "face_recognizer" |
The ML worker retains only generic ML capabilities:
semantic_embeddingvisual_concept_detectionalbum_shape_proposal_trainalbum_shape_proposal_score
4. Backward compatibility
During migration:
- The
face_annotationsandface_embeddingstables remain readable but are no longer written to by the new pipeline - The legacy
face_detectionandowner_face_recognitioncapabilities remain available but are deprecated - The People Worker can run alongside the ML worker during transition
- Once the People Worker is verified, the legacy capabilities are removed from the ML worker
Migration Path
Phase 1: People Worker (this ADR)
- Create
workers/people/withdiscovery_peoplecapability - Worker detects faces → submits
discovery_candidatewith bounding box + embedding - Core stores in
discovery_results(existing generic table) - Existing face_annotations→class_instance_occurrences path continues to work
Phase 2: Discovery → Class-Instance Promotion
- Generic promotion logic:
discovery_resultswithclassKey=person→class_instances - Replaces the manual linking in
PeopleService - Auto-accept, merge/split, sharing all work through generic discovery lifecycle
Phase 3: Legacy Cleanup
- Remove
face_detectionandowner_face_recognitionfrom ML worker - Deprecate
face_annotationsandface_embeddingstables - All People data flows through generic discovery pipeline
Consequences
- The discovery platform becomes the single pipeline for all recognition domains (plants, people, future pets/objects)
- Region markers and embeddings are reusable for any domain, not just faces
- The ML worker is simplified — it only handles generic ML tasks
- The People Worker is domain-specific and can evolve independently
- Existing People functionality is preserved during migration through backward-compatible coexistence
Memories Wiki
Using Memories
- Accounts, the Library Hub, and Data Export
- Albums and Automatic Suggestions
- Explore and Discovery Domains
- The Media Viewer
- People and Faces
- Places and Map
- Plants (In Development)
- Datenschutzerklärung (Privacy Policy)
- Fast, Progressive Image Loading
- Search
- Sharing and Households
- Timeline and Photos
- Uploading and Import
Self-Hosting
- Backup and Restore
- CLI Password Reset
- Deployment Architecture
- Known Issues and Operator Diagnostics
- Map Tiles and Place Data Privacy
- Storage Layout
- Upgrades and Versioning
- Workers
Reference