fix(people): restore worker main-loop dispatch inside polling loop (regression from #763) #764

Merged
chris merged 1 commit from fix/people-worker-dispatch-indentation into main 2026-09-19 01:38:48 +02:00
Owner

CRITICAL production regression from #763: the job-type dispatch block in workers/people main.py was accidentally moved OUTSIDE the while RUNNING polling loop (indentation 4 vs 8). Since the #763 deploy the people worker claims a job, lazily loads models, then loops forever without dispatching — every claimed job (discovery_people, owner_face_recognition, owner_face_embedding_backfill) sits 'running' until lease expiry and gets requeued by the recovery service, never processed.

Verified on prod with a canary owner_face_embedding_backfill job: claimed 01:18:58, zero handler logs / media downloads / results, lease expired 01:28:58, worker emits only empty claim polls.

Fix: restore the dispatch block to its pre-#763 indentation (inside the loop). Pure indentation change, no logic touched. py_compile green.

CRITICAL production regression from #763: the job-type dispatch block in workers/people main.py was accidentally moved OUTSIDE the while RUNNING polling loop (indentation 4 vs 8). Since the #763 deploy the people worker claims a job, lazily loads models, then loops forever without dispatching — every claimed job (discovery_people, owner_face_recognition, owner_face_embedding_backfill) sits 'running' until lease expiry and gets requeued by the recovery service, never processed. Verified on prod with a canary owner_face_embedding_backfill job: claimed 01:18:58, zero handler logs / media downloads / results, lease expired 01:28:58, worker emits only empty claim polls. Fix: restore the dispatch block to its pre-#763 indentation (inside the loop). Pure indentation change, no logic touched. py_compile green.
fix(people): restore worker main-loop dispatch inside polling loop (regression from #763)
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
da28f07c36
#763 accidentally moved the job-type dispatch block out of the
while RUNNING polling loop (4-space vs 8-space indent): the worker
claimed a job, lazily loaded models, then looped forever without ever
dispatching the claimed job — jobs sat 'running' until their lease
expired and the WorkerJobRecoveryService requeued them, endlessly.

Symptom verified on prod: canary owner_face_embedding_backfill job
claimed at 01:18:58, zero handler logs, zero media downloads, lease
expired at 01:28:58, worker shows only empty claim polls afterwards.

Restores the dispatch block to the pre-#763 indentation so all three
job types (owner_face_recognition, owner_face_embedding_backfill,
discovery_people) execute inside the loop again.
chris merged commit 105331b512 into main 2026-09-19 01:38:48 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!764
No description provided.