feat(workers): representative density graph prototype + benchmark (ADR 0113 exploration) #1255

Merged
chris merged 3 commits from feat/representative-density-graph-prototype into main 2026-10-16 10:55:04 +02:00
Owner

Summary

Additive exploration companion to ADR 0113 (PR #1254) — implements decision V2 (graph construction) as an INERT prototype and measures it. No production behavior changes.

Contents (3 commits):

  • rep_graph_prototype.py — build_rep_graph() (U×U via production _density_graph corrected tiles + rectangular U×R tile pass, same 192 MiB float32 budget, ISSUE-081 memory invariant; no L-blocks, no R×R; deterministic node order [U pool, R by (person_id, rep index)]) + partition_pool_groups() (components over U×U subgraph only) + RepGraphLayout. NOT registered in CLUSTERING_METHODS, no dispatch path.
  • tests/test_rep_graph_prototype.py — 11 contract tests: node/label invariants, no R–R edges, inclusive threshold, identity separation, rep-edges-never-glue-components, determinism, degenerate inputs.
  • experiments/rep_graph/benchmark.py + README — seeded synthetic generator at production geometry; baseline (_density_graph over L+U) vs prototype.

Measured results (synthetic, 512-dim, reps via production _person_representatives @ 0.50):

scenario baseline prototype speedup max comp
steady L=12k U=500 3.35 s / 733 MiB 0.03 s / 8.5 MiB 111× 102 → 10
mid U=5k 9.78 s 0.45 s 21.7× 113 → 20
bulk U=11k 19.58 s 3.44 s 5.7× (no regression) 30 → 30

Computed pairs steady state: 78.1M → 342k (228×). Identity sanity: 100.0% known-person U faces adjacent to ≥1 own-person rep, 0.0% to foreign reps; same-mode U–U connectivity 100.0% (cross-mode fragments by threshold design — reps carry identity).

Test plan

  • .venv/bin/python -m pytest workers/people/tests/test_rep_graph_prototype.py -q → 11 passed
  • Additivity proven: test_semisupervised_v2_clustering.py (20 + 92 subtests) and test_multi_representant.py (19) stay green
  • Reproduce benchmark: PYTHONPATH=workers/people/src .venv/bin/python workers/people/experiments/rep_graph/benchmark.py

This PR can merge independently or together with #1254; it changes no production paths either way.

## Summary Additive exploration companion to ADR 0113 (PR #1254) — implements decision V2 (graph construction) as an INERT prototype and measures it. No production behavior changes. **Contents (3 commits):** - `rep_graph_prototype.py` — `build_rep_graph()` (U×U via production `_density_graph` corrected tiles + rectangular U×R tile pass, same 192 MiB float32 budget, ISSUE-081 memory invariant; no L-blocks, no R×R; deterministic node order [U pool, R by (person_id, rep index)]) + `partition_pool_groups()` (components over U×U subgraph only) + `RepGraphLayout`. NOT registered in CLUSTERING_METHODS, no dispatch path. - `tests/test_rep_graph_prototype.py` — 11 contract tests: node/label invariants, no R–R edges, inclusive threshold, identity separation, rep-edges-never-glue-components, determinism, degenerate inputs. - `experiments/rep_graph/benchmark.py` + README — seeded synthetic generator at production geometry; baseline (`_density_graph` over L+U) vs prototype. **Measured results (synthetic, 512-dim, reps via production `_person_representatives` @ 0.50):** | scenario | baseline | prototype | speedup | max comp | |---|---|---|---|---| | steady L=12k U=500 | 3.35 s / 733 MiB | 0.03 s / 8.5 MiB | 111× | 102 → 10 | | mid U=5k | 9.78 s | 0.45 s | 21.7× | 113 → 20 | | bulk U=11k | 19.58 s | 3.44 s | 5.7× (no regression) | 30 → 30 | Computed pairs steady state: 78.1M → 342k (228×). Identity sanity: 100.0% known-person U faces adjacent to ≥1 own-person rep, 0.0% to foreign reps; same-mode U–U connectivity 100.0% (cross-mode fragments by threshold design — reps carry identity). ## Test plan - `.venv/bin/python -m pytest workers/people/tests/test_rep_graph_prototype.py -q` → 11 passed - Additivity proven: `test_semisupervised_v2_clustering.py` (20 + 92 subtests) and `test_multi_representant.py` (19) stay green - Reproduce benchmark: `PYTHONPATH=workers/people/src .venv/bin/python workers/people/experiments/rep_graph/benchmark.py` This PR can merge independently or together with #1254; it changes no production paths either way.
experiment(workers): representative density graph benchmark (ADR 0113 G3)
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
bf6da5ef11
chris force-pushed feat/representative-density-graph-prototype from bf6da5ef11
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
to 2d1cd200e2
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 2s
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build Android CI image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
2026-10-16 10:54:48 +02:00
Compare
chris merged commit 5f5c835904 into main 2026-10-16 10:55:04 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1255
No description provided.