refactor(workers/ml): remove dormant legacy face stack (spec 014 phase 2, worker slice) #702

Merged
chris merged 2 commits from feature/ml-face-code-removal into main 2026-09-05 10:08:32 +02:00
Owner

What

Remove the dormant legacy people/face code from workers/ml (spec 014 phase 2, worker slice). Net: 760 lines deleted, 9 added.

Deleted from workers/ml:

  • FaceDetectionConfig, InsightFaceRetinaFaceDetector, _normalized_embedding_or_none, _owner_face_embeddings, _owner_face_assignments, _best_existing_person_match, _density_clusters, _neighborhood, _cosine_distance, _landmarks_at, FACE_THUMBNAIL_SIZES, _face_thumbnail_jpeg
  • face_detection + owner_face_recognition handlers and their dispatch entries; face_detector ctor param
  • core_api.py: fetch_owner_face_embeddings() and the face_thumbnail idempotency special-case
  • main.py: face_detection,owner_face_recognition removed from default advertised capabilities
  • pyproject.toml: insightface + opencv-python-headless deps; description reworded
  • compose.yaml: MEMORIES_ML_FACE_DETECTION_MODEL_PATH / MEMORIES_ML_FACE_RECOGNITION_MODEL_PATH env lines removed; capabilities env updated
  • deploy/README.md: face detection row in ML profile table + service description reworded
  • workers/ml/README.md: face section deleted; people section rewritten (ML worker handles no face-domain jobs; both face job types → unsupported)

Kept (verified by grep before deleting): _box_iou, _non_max_suppressed_concepts, _clamp (visual-concepts L774-777 + album-shape L1284/L1294 callers), _embedding_hash (semantic_embedding ~L1193), _normalize, _input_size, display-oriented image helpers, math/hashlib imports, all other handlers (semantic_embedding, visual_concept_detection, discovery_test_image_labels, album_shape_proposal_train/score).

Why (dormancy evidence)

  • Core ML credential bootstrap excludes face job types, so a face_detection/owner_face_recognition job can never be leased to this worker.
  • The dedicated people worker owns discovery_people + owner_face_recognition (compose capabilities L236).
  • No enqueue path exists for face_detection in the default Compose deployment.
  • Net effect: the ML worker advertised two capabilities it could never serve, plus dead face-model env vars pointing at models the worker would never load.

Out of scope (untouched)

  • Core acceptFaceCandidate (apps/api)
  • workers/people/ (owns discovery_people + owner_face_recognition)
  • Media worker generate_face_thumbnail (compose L131)
  • People worker compose block (L221-253, L238-244)
  • deploy/example.env + deploy/tests/restore-validation.sh still mention MEMORIES_ML_FACE_* (bootstrap-hint vars for the Core model-profile system, not consumed by the ML worker; flagged for follow-up cleanup)

Verification

Step A grep results (pre-deletion):

  1. _clamp in jobs.py: face callers (deleted) + visual-concepts L774-777 + album-shape L1284/L1294 → kept
  2. _embedding_hash: face caller (deleted) + semantic_embedding L1193 → kept
  3. insightface: only inside deleted InsightFaceRetinaFaceDetector → gone
  4. import cv2: zero direct imports anywhere under workers/ml → opencv-python-headless removed (InsightFace transitive only)
  5. MEMORIES_ML_FACE: jobs.py (deleted) + compose.yaml L164-165 (deleted); deploy/example.env + restore-validation.sh leftovers noted above
  6. face_thumbnail/fetch_owner_face_embeddings in core_api.py: only face handlers used them → both removed

Post-change: rg -i "face|people|person|insightface|retinaface|arcface" workers/ml/ → only false positives: HEIC/HEIF substring in an unrelated comment, plus the new README section explaining people-worker ownership. No code hits.

Tests: full ML suite green, 19 tests (was 34): removed FakeRetinaFace, insightface module stubs (onnxruntime stub kept), FakeFaceDetector/FailingFaceDetector, FakeClient face-embedding fields, and 12 face/owner-face tests incl. legacy face_recognition/face_clustering fallback tests. All album-shape, visual-concepts, semantic, display-oriented EXIF, and claim-drain tests kept (claim-drain fixture capability renamed to semantic_embedding). ruff check workers/ml/ F401/F821/F841: 3 baseline errors → 1 (pre-existing undefined LOGGER in main.py, untouched region); total ruff findings 19 → 15 (all pre-existing style rules; zero new).

Known unrelated: 8 known Playwright failures on main, unrelated to this change.

No schema changes, no API changes.

## What Remove the dormant legacy people/face code from `workers/ml` (spec 014 phase 2, worker slice). Net: **760 lines deleted, 9 added**. **Deleted from `workers/ml`:** - `FaceDetectionConfig`, `InsightFaceRetinaFaceDetector`, `_normalized_embedding_or_none`, `_owner_face_embeddings`, `_owner_face_assignments`, `_best_existing_person_match`, `_density_clusters`, `_neighborhood`, `_cosine_distance`, `_landmarks_at`, `FACE_THUMBNAIL_SIZES`, `_face_thumbnail_jpeg` - `face_detection` + `owner_face_recognition` handlers and their dispatch entries; `face_detector` ctor param - `core_api.py`: `fetch_owner_face_embeddings()` and the `face_thumbnail` idempotency special-case - `main.py`: `face_detection,owner_face_recognition` removed from default advertised capabilities - `pyproject.toml`: `insightface` + `opencv-python-headless` deps; description reworded - `compose.yaml`: `MEMORIES_ML_FACE_DETECTION_MODEL_PATH` / `MEMORIES_ML_FACE_RECOGNITION_MODEL_PATH` env lines removed; capabilities env updated - `deploy/README.md`: face detection row in ML profile table + service description reworded - `workers/ml/README.md`: face section deleted; people section rewritten (ML worker handles no face-domain jobs; both face job types → unsupported) **Kept (verified by grep before deleting):** `_box_iou`, `_non_max_suppressed_concepts`, `_clamp` (visual-concepts L774-777 + album-shape L1284/L1294 callers), `_embedding_hash` (semantic_embedding ~L1193), `_normalize`, `_input_size`, display-oriented image helpers, `math`/`hashlib` imports, all other handlers (semantic_embedding, visual_concept_detection, discovery_test_image_labels, album_shape_proposal_train/score). ## Why (dormancy evidence) - Core ML credential bootstrap excludes face job types, so a face_detection/owner_face_recognition job can never be leased to this worker. - The dedicated people worker owns `discovery_people` + `owner_face_recognition` (compose capabilities L236). - No enqueue path exists for `face_detection` in the default Compose deployment. - Net effect: the ML worker advertised two capabilities it could never serve, plus dead face-model env vars pointing at models the worker would never load. ## Out of scope (untouched) - Core `acceptFaceCandidate` (apps/api) - `workers/people/` (owns discovery_people + owner_face_recognition) - Media worker `generate_face_thumbnail` (compose L131) - People worker compose block (L221-253, L238-244) - `deploy/example.env` + `deploy/tests/restore-validation.sh` still mention `MEMORIES_ML_FACE_*` (bootstrap-hint vars for the Core model-profile system, not consumed by the ML worker; flagged for follow-up cleanup) ## Verification **Step A grep results (pre-deletion):** 1. `_clamp` in jobs.py: face callers (deleted) + visual-concepts L774-777 + album-shape L1284/L1294 → kept 2. `_embedding_hash`: face caller (deleted) + semantic_embedding L1193 → kept 3. `insightface`: only inside deleted InsightFaceRetinaFaceDetector → gone 4. `import cv2`: zero direct imports anywhere under workers/ml → opencv-python-headless removed (InsightFace transitive only) 5. `MEMORIES_ML_FACE`: jobs.py (deleted) + compose.yaml L164-165 (deleted); deploy/example.env + restore-validation.sh leftovers noted above 6. `face_thumbnail`/`fetch_owner_face_embeddings` in core_api.py: only face handlers used them → both removed **Post-change:** `rg -i "face|people|person|insightface|retinaface|arcface" workers/ml/` → only false positives: HEIC/HEIF substring in an unrelated comment, plus the new README section explaining people-worker ownership. No code hits. **Tests:** full ML suite green, 19 tests (was 34): removed FakeRetinaFace, insightface module stubs (onnxruntime stub kept), FakeFaceDetector/FailingFaceDetector, FakeClient face-embedding fields, and 12 face/owner-face tests incl. legacy face_recognition/face_clustering fallback tests. All album-shape, visual-concepts, semantic, display-oriented EXIF, and claim-drain tests kept (claim-drain fixture capability renamed to semantic_embedding). `ruff check workers/ml/` F401/F821/F841: 3 baseline errors → 1 (pre-existing undefined `LOGGER` in main.py, untouched region); total ruff findings 19 → 15 (all pre-existing style rules; zero new). **Known unrelated:** 8 known Playwright failures on main, unrelated to this change. No schema changes, no API changes.
refactor(workers/ml): remove dormant legacy face stack
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m51s
cbfd9d0fee
Delete the dormant InsightFace face-detection stack from the ML worker:
FaceDetectionConfig, InsightFaceRetinaFaceDetector, owner-face
embedding/clustering helpers, face_thumbnail generation, the
face_detection/owner_face_recognition handlers and dispatch entries,
plus their tests. Drop insightface and opencv-python-headless
dependencies (cv2 was only transitive via InsightFace), remove the
MEMORIES_ML_FACE_* compose env lines, and remove face job types from
the default advertised capabilities. The core_api.py
face_thumbnail idempotency special-case and
fetch_owner_face_embeddings helper go with the handlers that were
their only callers.

Kept: _box_iou/_non_max_suppressed_concepts (visual concepts NMS),
_clamp (visual concepts + album shape), _embedding_hash
(semantic_embedding), _input_size, display-oriented image helpers,
and all other job handlers.

Face-domain jobs (discovery_people, owner_face_recognition) are owned
by the dedicated people worker; the ML worker now treats
face_detection and owner_face_recognition as unsupported job types.
chore: bump version to 1.21.25, log ISSUE-024 (dormant ML face capabilities removed)
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m46s
c2658d7984
chris merged commit d591bd28ec into main 2026-09-05 10:08:32 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!702
No description provided.