fix(discovery): make retroactive policy apply work — fix embedding table + query #682

Merged
chris merged 1 commit from fix/retroactive-apply into main 2026-07-21 21:03:45 +02:00
Owner

Summary

Fixes the 'Apply retroactively returns 0' bug in the face recognition auto-approval system.

Root causes

  1. Wrong embeddings table: matchCandidateToExistingInstance queried class_occurrence_embeddings (0 rows in prod) for the candidate's own embedding. Pending candidates' face embeddings are buffered in pending_face_embeddings (5,864 rows in prod) during candidate creation.

  2. Impossible state filter: Retrospective SQL queries in genericNamedAutoApprovalCandidates and genericClassInstanceUnnamedCandidateAssignments looked for state='pending' AND existing_instance_id IS NOT NULL — a combination that never exists. Pending candidates have existing_instance_id IS NULL.

Changes

  • ClassInstanceDiscoveryService.matchCandidateToExistingInstance: Changed step 3 to query pending_face_embeddings by face_annotation_id instead of joining through class_occurrence_embeddings → class_instance_occurrences.

  • ClassInstanceService.genericNamedAutoApprovalCandidates: Changed WHERE to existing_instance_id IS NULL, removed JOIN to class_instances (no existing instance to join to).

  • ClassInstanceService.applyAutoApprovalRetrospectively: Now calls matchCandidateToExistingInstance for each candidate, converts cosine distance to confidence, and auto-approves high-confidence matches. Unmatched candidates are counted as unnamed retargeted.

  • Test schema: Added pending_face_embeddings and class_occurrence_embeddings tables.

  • Tests: Added 3 tests: high-confidence match approval, low-confidence pending retention, empty dataset handling.

## Summary Fixes the 'Apply retroactively returns 0' bug in the face recognition auto-approval system. ### Root causes 1. **Wrong embeddings table**: `matchCandidateToExistingInstance` queried `class_occurrence_embeddings` (0 rows in prod) for the candidate's own embedding. Pending candidates' face embeddings are buffered in `pending_face_embeddings` (5,864 rows in prod) during candidate creation. 2. **Impossible state filter**: Retrospective SQL queries in `genericNamedAutoApprovalCandidates` and `genericClassInstanceUnnamedCandidateAssignments` looked for `state='pending' AND existing_instance_id IS NOT NULL` — a combination that never exists. Pending candidates have `existing_instance_id IS NULL`. ### Changes - **ClassInstanceDiscoveryService.matchCandidateToExistingInstance**: Changed step 3 to query `pending_face_embeddings` by `face_annotation_id` instead of joining through `class_occurrence_embeddings` → `class_instance_occurrences`. - **ClassInstanceService.genericNamedAutoApprovalCandidates**: Changed WHERE to `existing_instance_id IS NULL`, removed JOIN to `class_instances` (no existing instance to join to). - **ClassInstanceService.applyAutoApprovalRetrospectively**: Now calls `matchCandidateToExistingInstance` for each candidate, converts cosine distance to confidence, and auto-approves high-confidence matches. Unmatched candidates are counted as unnamed retargeted. - **Test schema**: Added `pending_face_embeddings` and `class_occurrence_embeddings` tables. - **Tests**: Added 3 tests: high-confidence match approval, low-confidence pending retention, empty dataset handling.
fix(discovery): make retroactive policy apply work — fix embedding table + query
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 4s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 2m20s
9565a325e9
Closes the 'Apply retroactively returns 0' bug.

Two root causes fixed:

1. matchCandidateToExistingInstance queried class_occurrence_embeddings
   (0 rows in prod) for candidate's own embedding. Changed to query
   pending_face_embeddings (5,864 rows in prod) which is where pending
   candidates' face embeddings are buffered during candidate creation.

2. Retrospective SQL queries in genericNamedAutoApprovalCandidates and
   genericClassInstanceUnnamedCandidateAssignments looked for
   state='pending' AND existing_instance_id IS NOT NULL — a combination
   that never exists. Pending candidates have existing_instance_id IS NULL.
   Changed to IS NULL and removed the JOIN to class_instances.

The fix processes all pending candidates, attempts to match each against
existing named people via matchCandidateToExistingInstance (now using the
correct embeddings table), and auto-approves high-confidence matches.
Unmatched candidates are counted as unnamed retargeted.

Version: 1.21.19
chris merged commit 238019a657 into main 2026-07-21 21:03:45 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!682
No description provided.