test(web): cap Playwright workers to stop full-suite contention flakes #1069

Merged
chris merged 1 commit from test/pw-worker-tuning into main 2026-10-07 21:37:11 +02:00
Owner

Evidence: nproc = 8 and Playwright's default workers is 50% of cores (playwright/lib/common/config.js), so the full suite ran at 4 workers on this host — measured from the run log ("Running 862 tests using 4 workers"). At those settings the full suite flaked: 47 failed / 16 flaky / 776 passed in 9.2m, spread across unrelated specs, while every failing spec passes in isolation (e.g. upload-status 12/12 alone, profile-tabs 36/36 alone) and retries: 2 did not absorb them — i.e. contention on the shared dev host, not product bugs. A candidate cap of min(cpus, 6) = 6 was rejected from measurement because it would have raised concurrency above the setting that failed.

Fix: explicit workers = min(floor(cpus/2), 4) — identical to today's default on ≤8-core hosts, and a hard bound on bigger machines where the 50% default would spawn 8–16 workers and saturate them. fullyParallel: true kept (we cap concurrency, we do not serialize files); no spec files, retries, webServer/port logic, or projects touched.

Verification (full suite npm test, 862 tests / desktop+mobile): baseline at the previous settings — 0 failed / 0 flaky / 839 passed / 23 skipped in 5.8m (flake is load-dependent; host was idle). After the cap, two consecutive clean full runs: 837 passed, 0 failed, 2 flaky absorbed by retries (7.4m) and 832 passed, 0 failed, 7 flaky absorbed by retries (11.2m).

CI: .forgejo/workflows/ runs npm run test with no --workers flag, the Playwright image does not export CI, and no CPU limit is set on the web-test container — so no CI override conflicts; CI inherits the same expression (unchanged vs today's 50% default on ≤8-core runners, capped at 4 on larger ones).

Evidence: `nproc` = 8 and Playwright's default `workers` is `50%` of cores (`playwright/lib/common/config.js`), so the full suite ran at **4 workers** on this host — measured from the run log ("Running 862 tests using 4 workers"). At those settings the full suite flaked: 47 failed / 16 flaky / 776 passed in 9.2m, spread across unrelated specs, while every failing spec passes in isolation (e.g. upload-status 12/12 alone, profile-tabs 36/36 alone) and `retries: 2` did not absorb them — i.e. contention on the shared dev host, not product bugs. A candidate cap of `min(cpus, 6)` = 6 was rejected from measurement because it would have *raised* concurrency above the setting that failed. Fix: explicit `workers = min(floor(cpus/2), 4)` — identical to today's default on ≤8-core hosts, and a hard bound on bigger machines where the 50% default would spawn 8–16 workers and saturate them. `fullyParallel: true` kept (we cap concurrency, we do not serialize files); no spec files, `retries`, webServer/port logic, or projects touched. Verification (full suite `npm test`, 862 tests / desktop+mobile): baseline at the previous settings — 0 failed / 0 flaky / 839 passed / 23 skipped in 5.8m (flake is load-dependent; host was idle). After the cap, two consecutive clean full runs: **837 passed, 0 failed, 2 flaky absorbed by retries (7.4m)** and **832 passed, 0 failed, 7 flaky absorbed by retries (11.2m)**. CI: `.forgejo/workflows/` runs `npm run test` with no `--workers` flag, the Playwright image does not export `CI`, and no CPU limit is set on the web-test container — so no CI override conflicts; CI inherits the same expression (unchanged vs today's 50% default on ≤8-core runners, capped at 4 on larger ones).
test(web): cap Playwright workers to stop full-suite contention flakes
All checks were successful
Wiki Sync / Sync wiki from docs (push) Has been skipped
Build and Deploy / Detect changed services and test scopes (push) Successful in 5s
Build and Deploy / Android unit tests and debug APK (push) Has been skipped
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Smoke people worker (push) Has been skipped
Build and Deploy / Build postgres image (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Deploy postgres (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 9m24s
7cb1560fa0
The full suite (fullyParallel, desktop+mobile) flakes from resource
contention on the shared dev host: with the 50%-of-cores default (8
workers here) a run produced 47 failures across unrelated specs, every
one of which passes in isolation. retries: 2 did not absorb them.

Cap workers at min(floor(cpus/2), 4). Verified with two consecutive
clean full-suite runs at the cap (0 failed; 2 and 7 flaky absorbed by
retries). CI runs the same expression — identical behavior on <=8-core
runners, capped on bigger ones.
chris merged commit c0b62c5955 into main 2026-10-07 21:37:11 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!1069
No description provided.