perf(api): batch ranking candidate-face lookups and defer person-side scans #1089

Merged
chris merged 3 commits from fix/person-ranking-picker-latency into main 2026-10-10 15:13:23 +02:00
Owner

Problem

Opening the "choose an existing person" picker in discovery suggestions showed "Loading people…" for seconds. Measured against the live DB:

  • personRanking issued per-face SQL (N+1): up to 391 faces × ~4.2 ms = 1.6–3.3 s
  • plus a full person-side embedding scan (22 MB / 3561-row JSONB, ~505 ms inside Postgres)

Fix

  1. Batch the N+1: candidate-face lookups are collected and issued as bounded WHERE face_id IN (...) chunks (RANKING_IN_CHUNK_SIZE) instead of one query per face.
  2. Defer the person-side scan: occurrence embedding buckets / person embedding spaces are loaded lazily, only when a ranking merge actually needs them, so the common path (representatives-first) never triggers the scan.

ADR 0109 multi-representant pessimistic ranking semantics are preserved exactly — no test tolerances were loosened.

Verification

  • rtk ./gradlew :apps:api:test → BUILD SUCCESSFUL, 144 suites / 1672 tests / 0 failures / 9 skipped (incl. PgVectorEmbeddingPostgresTest 6/6, ClassInstanceDiscoveryServiceTest 291/291)
  • New statement-count guard test: 1 face 7→7, 3 faces 11→7, 50 faces 105→7 statements.
  • 4 new tests cover batching, chunking, and deferral exactness.
  • Oracle implementation review: approved (minor findings fixed in 807852a9).

Scope / follow-ups

  • Residual [ISSUE-111] is low/open and honestly scoped: only mergeSuggestion can change an outcome; mergeTargetRankings only over-fetches.
  • Out of scope (separate cleanup): dead embeddingKnnService field, stashed_at IS NULL filter inconsistency, two stale single-centroid comments.
  • No schema changes / migrations.
## Problem Opening the "choose an existing person" picker in discovery suggestions showed "Loading people…" for seconds. Measured against the live DB: - `personRanking` issued **per-face SQL (N+1)**: up to 391 faces × ~4.2 ms = 1.6–3.3 s - plus a full person-side embedding scan (22 MB / 3561-row JSONB, ~505 ms inside Postgres) ## Fix 1. **Batch the N+1**: candidate-face lookups are collected and issued as bounded `WHERE face_id IN (...)` chunks (`RANKING_IN_CHUNK_SIZE`) instead of one query per face. 2. **Defer the person-side scan**: occurrence embedding buckets / person embedding spaces are loaded lazily, only when a ranking merge actually needs them, so the common path (representatives-first) never triggers the scan. ADR 0109 multi-representant pessimistic ranking semantics are preserved exactly — no test tolerances were loosened. ## Verification - `rtk ./gradlew :apps:api:test` → **BUILD SUCCESSFUL**, 144 suites / **1672 tests / 0 failures / 9 skipped** (incl. `PgVectorEmbeddingPostgresTest` 6/6, `ClassInstanceDiscoveryServiceTest` 291/291) - New statement-count guard test: 1 face 7→7, 3 faces 11→7, **50 faces 105→7** statements. - 4 new tests cover batching, chunking, and deferral exactness. - Oracle implementation review: approved (minor findings fixed in `807852a9`). ## Scope / follow-ups - Residual `[ISSUE-111]` is `low`/`open` and honestly scoped: only `mergeSuggestion` can change an outcome; `mergeTargetRankings` only over-fetches. - Out of scope (separate cleanup): dead `embeddingKnnService` field, `stashed_at IS NULL` filter inconsistency, two stale single-centroid comments. - No schema changes / migrations.
The 'choose an existing person' picker endpoints
(personRanking/mergeTargetRankings/mergeSuggestion) held a read-only
transaction for seconds: personRanking ran one SQL query PER candidate
face (accepted occurrence + pending fallback, ~4.2 ms/query live; 391
faces = ~780 statements), and all three surfaces ran an unconditional
full-corpus person-side embedding scan (3561 rows / 22 MB JSONB =
505 ms in PostgreSQL alone, plus a 128 ms personRows query).

Phase 1 (personRanking): candidate-side face embeddings now resolve
through two set-based queries (accepted probe over chunked occurrence_key
IN lists, then ONE pending-face_embeddings query over only the faces the
accepted probe left empty) — grouping per face, per-face majority pair
vote and row multiplicity into the buckets are byte-identical to the
former loop; the created_at DESC ordering was inert and is dropped.

Phase 2 (all three surfaces): representatives load FIRST and the
composer's rep direction is evaluated in memory, so the expensive
person-side scans run restricted to the instances whose live buckets the
composer can actually consult:
- personRanking: candidate side never carries reps (Map.of()), so a
  person covered by pessimisticRepresentativeDistance never has its
  buckets read; the scan is restricted to the still-uncovered ids of the
  personRows universe (derived from the composer's own output, never
  rep-table presence — foreign-pair rep sets still fall back to live
  buckets);
- mergeTargetRankings: source buckets always load; per target the
  in-memory viaSecond plus a metadata-only space census (GROUP BY over
  model/version/dims, no embedding payload, flushed+pending UNION ALL,
  sub-millisecond) decides whether the target's buckets can affect any
  composer branch (viaSecond null, or source reps sharing one of the
  target's live spaces so viaFirst can win);
- mergeSuggestion: same census over all instances; an instance's buckets
  load only when some same-class non-rejected partner either reaches its
  buckets through the partner's reps or leaves the live fallback as the
  only reachable direction.

Exactness: bucket maps, spaces, per-instance ordering and the composer
inputs are unchanged; ADR 0109 binding contract (pessimistic-min rep
ranking, live single-centroid fallback, null-similarity handling, nulls
last / id tie-break) is preserved. No schema changes; the dead
embeddingKnnService field and the stashed-filter differences between the
surfaces are intentionally untouched.

Measured by the new guard: personRanking executes a constant 7
statements for 1/3/50 candidate faces (before: 7/11/105, >=2 per face).
Four H2 tests pinning the ranking latency fix:
- personRankingRunsConstantNumberOfStatementsRegardlessOfFaceCount:
  counting-JdbcClient guard asserting a fixed statement budget (<= 7)
  for candidates with 1, 3 and 50 faces — the former per-face N+1 ran
  7/11/105 statements at those sizes;
- personRankingDefersPersonEmbeddingScanByComposerCoverageNotRepPresence:
  rep-covered person, rep-less person and a person whose reps live only
  in a foreign model pair all score their hand-computed pessimistic
  values — proving the deferral set follows the composer's own coverage
  (viaSecond), not rep-table presence;
- personRankingBatchedFaceResolutionKeepsPerFacePairVotesAndRowMultiplicity:
  faces carrying two model pairs keep their per-face majority vote, a
  within-face tie resolves to the larger pair key, and every row of the
  dominant pair enters the candidate bucket (n_source counts rows, not
  faces);
- mergeTargetRankingsSourceRepresentativesStillLoadTargetBucketsWhenViaFirstCanWin:
  a source rep set that disagrees with its live face makes viaFirst win
  for a target with live buckets — the metadata-based deferral must load
  them — while a rep-only control target stays covered.

Also documents the merge-surface deferral caveat (space-key-level
census vs all-zero-norm bucket degeneracy) as ISSUE-111.
docs(api): correct ISSUE-111 trigger and fixture margin comment
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 9m36s
807852a9b9
ISSUE-111's outcome-changing direction needs rows that fail embedding()
parsing or are empty (zero-norm rows still enter buckets), and only
mergeSuggestion can change an outcome -- mergeTargetRankings only
over-fetches. The viaSecond fixture comment computed d=1 for the
+x->-x pair; the cosine distance is 2.
chris merged commit 766b3ce2f1 into main 2026-10-10 15:13:23 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1089
No description provided.