perf(api): speed up face-suggestion lookups + regression guard #1252

Merged
chris merged 3 commits from fix/face-suggest-slow into main 2026-10-16 00:23:07 +02:00
Owner

Problem

The person-assignment dialog's suggestion dropdown (GET /discovery/domains/{domainKey}/instance-suggestions → suggestSimilarInstances) fires on every ~300 ms-debounced keystroke and had become slow again. Three measured/argued cost drivers on main (17d5147b):

  1. Seq scans on the source-face probes. Steps 0/1 filter class_instance_occurrences on occurrence_key ALONE. The only index containing occurrence_key is the V074 unique constraint (domain_key, owner_id, occurrence_type, occurrence_key) — fourth column, unusable for a leading-occurrence_key predicate → parallel seq scan of the whole occurrences table, twice per request, on a table that grows with one row per accepted/tentative face. (Sixteen other occurrence_key probes in the service carry domain_key/owner_id beside it and are served by the constraint prefix; the suggestion probes are the uncovered shapes.)
  2. Correlated per-row occurrence-count probe. The candidate-rows statement carried (SELECT COUNT(*) FROM class_instance_occurrences cio2 WHERE cio2.class_instance_id = ci.id ...) — executed once per candidate embedding row, the exact ISSUE-140 class (PR #1144: thumbnail subquery → ~4,675 executions for ~350 persons → ~39 s). The PR#1146 fix batched the thumbnail probe but left the count probe per-row.
  3. Nothing pinned suggestSimilarInstances against a statement-count/probe regression, so the class kept recurring.

Commits

test(api): pin instance-suggestion statement count (+ fix)

  • New guard suggestSimilarInstancesRunsConstantNumberOfStatementsRegardlessOfCandidateCount (mirrors personRankingRunsConstantNumberOfStatementsRegardlessOfFaceCount): identical statement budget at 1/3/50 candidate faces, plus an alias-independent textual pin of the candidate-rows statement (identified by its load-bearing AS instance_id alias) forbidding a scalar (SELECT COUNT occurrence probe — the per-row probe lives inside ONE statement and is invisible to a statement count. Mutation-checked: the test fails against the unfixed code and passes with the fix.
  • Behaviour-preserving fix: the correlated scalar probe becomes a LEFT JOIN to a once-per-instance grouped aggregate over the same live-state rows scoped to the same candidate-instance predicates. Same values (the IN scope is implied by the ci join), 1:1 join via GROUP BY so the row set is unchanged, COALESCE(..., 0) reproduces COUNT(*) = 0 for unmatched instances. No ranking/score/ordering/fallback changes — all 23 pre-existing suggestion tests pass unchanged.

perf(api): index occurrence_key lookups for face suggestions

  • V167 (additive, non-destructive): plain CREATE INDEX btree on class_instance_occurrences (occurrence_key) — Flyway runs migrations in transactions, so CONCURRENTLY is unavailable (V140 convention).
  • EXPLAIN reasoning: btree scans left-to-right; a leading-column equality on occurrence_key descends the new index directly, while the V074 constraint would need a full index scan (columns 1-3 unbound) which the planner rejects in favour of a seq scan. Measured on PG16, 455k occurrences, matched key:
    • step 0: parallel seq scan 43.9 ms → index scan 0.031 ms
    • step 1 (with production V101 index on coe.occurrence_id): nested loop, two index scans, 0.061 ms
  • OccurrenceKeyIndexPostgresTest (Testcontainers, full Flyway chain) pins usability of the index for both exact statement shapes via the established enable_seqscan = off EXPLAIN pattern.

Validation

  • rtk ./gradlew :apps:api:check — green (before the base rebase; discovery package + Postgres chain re-run green after).
  • discovery package: 721 tests, 0 failures. Postgres-backed people suites (full Flyway chain incl. V167): green.
  • The new guard was demonstrated RED on pre-fix code, GREEN after.

Notes

  • V166 was taken by PR #1251 mid-flight; the migration is renumbered V167 (never applied outside throwaway containers).
  • No caching layers, no ranking/threshold/q-filter changes, no frontend or thumbnail-selection changes.
## Problem The person-assignment dialog's suggestion dropdown (`GET /discovery/domains/{domainKey}/instance-suggestions` → `suggestSimilarInstances`) fires on every ~300 ms-debounced keystroke and had become slow again. Three measured/argued cost drivers on `main` (17d5147b): 1. **Seq scans on the source-face probes.** Steps 0/1 filter `class_instance_occurrences` on `occurrence_key` ALONE. The only index containing `occurrence_key` is the V074 unique constraint `(domain_key, owner_id, occurrence_type, occurrence_key)` — fourth column, unusable for a leading-`occurrence_key` predicate → parallel seq scan of the whole occurrences table, **twice per request**, on a table that grows with one row per accepted/tentative face. (Sixteen other `occurrence_key` probes in the service carry `domain_key`/`owner_id` beside it and are served by the constraint prefix; the suggestion probes are the uncovered shapes.) 2. **Correlated per-row occurrence-count probe.** The candidate-rows statement carried `(SELECT COUNT(*) FROM class_instance_occurrences cio2 WHERE cio2.class_instance_id = ci.id ...)` — executed once per candidate **embedding row**, the exact ISSUE-140 class (PR #1144: thumbnail subquery → ~4,675 executions for ~350 persons → ~39 s). The PR#1146 fix batched the *thumbnail* probe but left the *count* probe per-row. 3. Nothing pinned `suggestSimilarInstances` against a statement-count/probe regression, so the class kept recurring. ## Commits ### `test(api): pin instance-suggestion statement count` (+ fix) - New guard `suggestSimilarInstancesRunsConstantNumberOfStatementsRegardlessOfCandidateCount` (mirrors `personRankingRunsConstantNumberOfStatementsRegardlessOfFaceCount`): identical statement budget at 1/3/50 candidate faces, **plus** an alias-independent textual pin of the candidate-rows statement (identified by its load-bearing `AS instance_id` alias) forbidding a scalar `(SELECT COUNT` occurrence probe — the per-row probe lives inside ONE statement and is invisible to a statement count. **Mutation-checked: the test fails against the unfixed code and passes with the fix.** - Behaviour-preserving fix: the correlated scalar probe becomes a `LEFT JOIN` to a once-per-instance grouped aggregate over the same live-state rows scoped to the same candidate-instance predicates. Same values (the IN scope is implied by the `ci` join), 1:1 join via GROUP BY so the row set is unchanged, `COALESCE(..., 0)` reproduces `COUNT(*) = 0` for unmatched instances. No ranking/score/ordering/fallback changes — all 23 pre-existing suggestion tests pass unchanged. ### `perf(api): index occurrence_key lookups for face suggestions` - **V167** (additive, non-destructive): plain `CREATE INDEX` btree on `class_instance_occurrences (occurrence_key)` — Flyway runs migrations in transactions, so `CONCURRENTLY` is unavailable (V140 convention). - EXPLAIN reasoning: btree scans left-to-right; a leading-column equality on `occurrence_key` descends the new index directly, while the V074 constraint would need a full index scan (columns 1-3 unbound) which the planner rejects in favour of a seq scan. Measured on PG16, 455k occurrences, matched key: - step 0: parallel seq scan **43.9 ms → index scan 0.031 ms** - step 1 (with production V101 index on `coe.occurrence_id`): **nested loop, two index scans, 0.061 ms** - `OccurrenceKeyIndexPostgresTest` (Testcontainers, full Flyway chain) pins usability of the index for both exact statement shapes via the established `enable_seqscan = off` EXPLAIN pattern. ## Validation - `rtk ./gradlew :apps:api:check` — green (before the base rebase; discovery package + Postgres chain re-run green after). - `discovery` package: 721 tests, 0 failures. Postgres-backed people suites (full Flyway chain incl. V167): green. - The new guard was demonstrated RED on pre-fix code, GREEN after. ## Notes - V166 was taken by PR #1251 mid-flight; the migration is renumbered V167 (never applied outside throwaway containers). - No caching layers, no ranking/threshold/`q`-filter changes, no frontend or thumbnail-selection changes.
suggestSimilarInstances (the assignment dropdown) had no guard against the
ISSUE-140 class of regression: a correlated per-row probe re-introduced
inside the candidate-rows statement is invisible to a statement count. Pin
both failure shapes at 1/3/50 candidate faces - a fixed statement budget
(mirror of personRankingRunsConstantNumberOfStatementsRegardlessOfFaceCount)
plus an alias-independent textual pin of the candidate-rows statement
("AS instance_id" filter) forbidding a scalar "(SELECT COUNT" occurrence
probe. The new test failed against the then-current code, which still
carried a per-row COUNT(*) (cio2) in the candidate statement.

Fix, behaviour-preserving: replace the correlated scalar probe with a LEFT
JOIN to a once-per-instance grouped aggregate over the same live-state rows
scoped to the same candidate-instance predicates. Values are identical
(same counted rows; the IN scope is implied by the ci join), the GROUP BY
makes the join 1:1 so the row set is unchanged, and unmatched instances
keep COALESCE(..., 0) = the old probe's COUNT(*) = 0. No ranking, score,
ordering or fallback changes; all 23 existing suggestion tests pass
unchanged.
suggestSimilarInstances resolves the source face with two statements
filtering class_instance_occurrences on occurrence_key ALONE (step 0's
live-assignment probe and step 1's embedding join). The only index
containing occurrence_key is the V074 unique constraint (domain_key,
owner_id, occurrence_type, occurrence_key) - occurrence_key is its fourth
column, so a leading-occurrence_key predicate cannot use it and every
request seq-scans the table twice, on a table that grows with one row per
accepted/tentative face. Sixteen further occurrence_key probes in this
service are served by the constraint's (domain_key, owner_id) prefix; the
suggestion probes are the uncovered shapes.

V166 adds a plain btree on (occurrence_key) (Flyway transactional, no
CONCURRENTLY - V140 convention). EXPLAIN (ANALYZE) on PG16 with 455k
occurrences: step 0 drops from a parallel seq scan at 43.9 ms to an index
scan at 0.031 ms; step 1 becomes a two-index nested loop at 0.061 ms.
Additive, non-destructive; OccurrenceKeyIndexPostgresTest pins the index's
usability for both exact statement shapes via the enable_seqscan-off
EXPLAIN pattern.
fix(api): renumber face-suggestion index migration to V167 (V166 taken on main)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 8m6s
6c0d9bd087
PR #1251 landed V166__worker_job_history_purge.sql on main while this
branch was in flight; the file had not been applied anywhere outside the
throwaway Postgres containers, so a renumber to V167 keeps the chain
linear (same situation as 38ebffa8).
chris merged commit 4b9ccebae6 into main 2026-10-16 00:23:07 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1252
No description provided.