fix(people): embedding-only backfill to migrate legacy buffalo_l occurrences into arcface-r50 space #763
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/legacy-embedding-space-backfill"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Fixes the data half of ISSUE-049: accepted people face occurrences carrying legacy buffalo_l/arcface-v1 embeddings (pre-#702 worker stack) are invisible to cross-person comparison — matching, merge suggestions and similarity ranking group by embedding space and only pair within a shared space, so legacy-only clusters (115 of 116 active unnamed instances) can never be merge-suggested against named persons (all 21 named persons are arcface-r50-only).
What
Embedding-only backfill — NO face re-detection (stored bboxes: no bbox drift, no duplicate face rows):
WorkerJobEnqueueService.enqueueOwnerFaceEmbeddingBackfillJobs): oneowner_face_embedding_backfilljob per affected media item, payloadfacesarray [{faceAnnotationId, boundingBox}...] capped at 500/job. Target query: accepted people face_annotation occurrences with a buffalo_l coe row and no arcface-r50/w600k-v1 row; photo-only; skips permanently_deleted media; face-level anti-join on active backfill jobs ⇒ repeated invocations drain the backlog resumably. Batching choice: per-media jobs instead of one owner-scoped job, because media download is authorized via per-jobworker_media_access_grantscreated only fromworker_jobs.media_item_id(owner-scoped jobs could not download pixels), and per-photo payloads are naturally bounded.POST /api/admin/people/backfill-legacy-embeddings, superadmin, returns {jobsEnqueued}). Deliberately NOT auto-scheduled.main.pynew dispatch branch): downloads the original once per job, crops each stored bbox viaFaceDetector.embedding_from_normalized_box(same 10%-pad/112×112/[-1,1] preprocessing as detection time), stamps modelName/modelVersion from the sameMEMORIES_FACE_EMBEDDING_MODEL_NAME/VERSIONenv vars as detection so the space label matches exactly, submitsowner_face_embedding_backfill_resultwith per-face embeddings.WorkerResultService): routes the new result type through the existingupsertOccurrenceEmbeddingpath with explicit faceAnnotationId (V118 UNIQUE(occurrence_id, model_name, model_version) allows coexistence; dedup ⇒ idempotent). Faces whose occurrence is missing or no longer accepted are skipped + counted, never failing the result; superseded occurrences do NOT buffer pending rows (would re-arm recognition for settled faces). On acceptance,owner_face_recognitionis enqueued for the owner (idempotent via the enqueue's own active-job guard) so matching/merge suggestions converge over the now-comparable snapshot.Test plan
rtk ./gradlew :apps:api:compileJavagreenWorkerJobEnqueueServiceTestcases: buffalo-only occurrence selected, arcface-covered skipped, other domains skipped, active-job anti-join + resumable drain, photo/permanently_deleted filters, payload shapeWorkerResultBackfillEmbeddingTest: arcface coe row written + recognition enqueued, idempotent replay, superseded/no-embedding skipped without failing, owner mismatch + wrong job type rejectedPeopleBackfillControllerTest: superadmin 403 + jobsEnqueued for the new endpointrtk ./gradlew :apps:api:testgreenpython -m py_compileon the worker; pytest in workers/people: 71 passed, 8 pre-existing failures unrelated to this change (reproduced with the worker changes stashed — numpy-version issues in mock-based detection tests)Accepted people face occurrences carrying legacy buffalo_l/arcface-v1 class_occurrence_embeddings (pre-PR#702 worker-ml stack) are invisible to all cross-person comparison: matching, merge suggestions and similarity ranking group by embedding space and only pair within a shared space, so legacy-only clusters can never be merge-suggested against named persons (ISSUE-049). Embedding-only backfill (no re-detection — stored bboxes, no bbox drift, no duplicate face rows): - WorkerJobEnqueueService.enqueueOwnerFaceEmbeddingBackfillJobs(): one owner_face_embedding_backfill job per affected media item (faces array with stored bboxes, cap 500/job). Media-scoped batching chosen because media download is authorized via per-job worker_media_access_grants created only from worker_jobs.media_item_id — an owner-scoped job could not download pixels — and per-photo payloads are naturally bounded. Target query: accepted people face_annotation occurrences with a buffalo_l row and no arcface-r50/w600k-v1 row, photo-only, skipping permanently_deleted media, with a face-level anti-join on active backfill jobs so repeated endpoint invocations drain the backlog. - PeopleBackfillController POST /api/admin/people/backfill-legacy- embeddings (superadmin, {jobsEnqueued}). Not auto-scheduled. - Worker: new dispatch branch downloads the original once per job, crops each stored bbox with the detection-time preprocessing (FaceDetector.embedding_from_normalized_box) and stamps modelName/modelVersion from the same MEMORIES_FACE_EMBEDDING_MODEL_* env vars as detection, so the space label matches exactly. - WorkerResultService.acceptOwnerFaceEmbeddingBackfillResult(): writes coe rows via the existing upsertOccurrenceEmbedding path (dedup => idempotent); faces whose occurrence is missing or no longer accepted are skipped + counted, never failing the result (no pending buffer for superseded occurrences — that would re-arm recognition for settled faces). On acceptance, owner_face_recognition is enqueued for the owner (idempotent via the enqueue's own active-job guard) so matching and merge suggestions converge over the now-comparable snapshot. - V127 migration: register the job type for the people worker credential (discovery_domains.job_types + worker_credentials allowed_job_types, V084/V111 pattern); compose capabilities updated. - Follow-up (comment only): legacy buffalo_l rows can be deleted once arcface coverage is verified. Tests: enqueue selection (buffalo-only selected, arcface-covered/plants domain/other-owner skipped, active-job anti-join + resumable drain, photo/permanently-deleted filters); result acceptance (arcface row written + recognition enqueued, idempotent replay, superseded/no- embedding skipped without failing, owner mismatch + wrong job type rejected); controller superadmin checks.