fix(web): stabilize flaky album/timeline/selection E2E specs #785

Merged
chris merged 5 commits from fix/web-ci-stability into main 2026-09-20 23:24:04 +02:00
Owner

Follow-up to #784. Reproduced the new-base flakes locally and fixed the deterministic races rather than bumping timeouts. Final scope is test-only (4 spec files).

Root causes and fixes

albums.spec.ts:1308 (failed 3/3 in runs 2577+2578) — heavy test, not a layout bug
Passed 3/3 locally but ~10s locally and >30s on loaded CI runners: 65 pattern albums × up to 6 rows, each row polled with two boundingBox() round-trips (~200 polls). Replaced with a wait for all rows reporting data-row-justified="true" plus one batched in-browser width comparison per pattern (~7s locally, ~10s CI), with a 60s budget surfacing regressions sooner.

albums.spec.ts:1778 (mobile flaky in 2578; reproduced 3/12 locally) — test race, not app bug
Snapshot showed "6 selected" instead of 5: Playwright's coordinate click on "Select loaded" scrolls the review container to an edge, triggering the app's scroll-based lazy day reveal and growing the loaded window mid-test. The click is now dispatched on the already-enabled button, so no synthetic scroll can fire. 24/24 local repeats green.

timeline.spec.ts:302 (failed 3/3 in 2577; reproduced 2/8 locally) — CSS hover race
The section selection control is hover-revealed; a coordinate click after hover can land after a section re-render drops the hover state. Controls are now hover-revealed (asserted visible) and clicked via dispatch, with a re-hover before the final visibility assertion. 12/12 local repeats green.

discovery-domain-selection.spec.ts:129 (timed out at 180s in runs 2577+2578) — test cost
505 sequential coordinate clicks each scrolled and re-rendered the 505-card grid (~60s locally, >180s on CI). Controls are now toggled in-page via their real click handlers (cap logic unchanged) and the budget tightened 180s -> 60s. Runtime 1.0m -> 1.1s locally, ~3s in CI.

processing-issues.spec.ts:54 (flaky in 2578, not reproducible locally 12/12) — env timing
A single menu-toggle click can be lost while the page settles; the menu is now re-opened until its items render, bounded by expect.poll (15s, 250ms interval). No sleeps.

CI infrastructure (superseded)

An earlier revision of this branch also isolated the Playwright preview port, after run 2580 failed 433 tests with net::ERR_CONNECTION_REFUSED (concurrent web-test containers share the host network on the shared runner). Main now carries the equivalent fix for this (3f1a084b, PR #779), including the root cause that CI is unset inside the Playwright image so reuseExistingServer was active in CI. During the merge of main this branch dropped its own port change in favour of main's implementation; the PR diff is now test-only.

Verification

  • Local: npx playwright test tests/albums.spec.ts tests/timeline.spec.ts tests/discovery-domain-selection.spec.ts tests/processing-issues.spec.ts -> 102 tests, 0 failures (99 passed, 3 skipped)
  • Repeats: albums:1308 3/3, albums:1778 24/24, timeline:302 12/12, processing-issues:54 12/12
  • npm run check -> 0 errors (20 pre-existing warnings)
  • CI run 2589 (final SHA, post-merge of main): scoped web suite 449 passed / 0 failed; all 10 target invocations (5 tests × desktop+mobile) green; the 5 flaky retries were unrelated specs, all passed on retry
Follow-up to #784. Reproduced the new-base flakes locally and fixed the deterministic races rather than bumping timeouts. Final scope is test-only (4 spec files). ## Root causes and fixes **albums.spec.ts:1308 (failed 3/3 in runs 2577+2578) — heavy test, not a layout bug** Passed 3/3 locally but ~10s locally and >30s on loaded CI runners: 65 pattern albums × up to 6 rows, each row polled with two `boundingBox()` round-trips (~200 polls). Replaced with a wait for all rows reporting `data-row-justified="true"` plus one batched in-browser width comparison per pattern (~7s locally, ~10s CI), with a 60s budget surfacing regressions sooner. **albums.spec.ts:1778 (mobile flaky in 2578; reproduced 3/12 locally) — test race, not app bug** Snapshot showed "6 selected" instead of 5: Playwright's coordinate click on "Select loaded" scrolls the review container to an edge, triggering the app's scroll-based lazy day reveal and growing the loaded window mid-test. The click is now dispatched on the already-enabled button, so no synthetic scroll can fire. 24/24 local repeats green. **timeline.spec.ts:302 (failed 3/3 in 2577; reproduced 2/8 locally) — CSS hover race** The section selection control is hover-revealed; a coordinate click after hover can land after a section re-render drops the hover state. Controls are now hover-revealed (asserted visible) and clicked via dispatch, with a re-hover before the final visibility assertion. 12/12 local repeats green. **discovery-domain-selection.spec.ts:129 (timed out at 180s in runs 2577+2578) — test cost** 505 sequential coordinate clicks each scrolled and re-rendered the 505-card grid (~60s locally, >180s on CI). Controls are now toggled in-page via their real click handlers (cap logic unchanged) and the budget tightened 180s -> 60s. Runtime 1.0m -> 1.1s locally, ~3s in CI. **processing-issues.spec.ts:54 (flaky in 2578, not reproducible locally 12/12) — env timing** A single menu-toggle click can be lost while the page settles; the menu is now re-opened until its items render, bounded by `expect.poll` (15s, 250ms interval). No sleeps. ## CI infrastructure (superseded) An earlier revision of this branch also isolated the Playwright preview port, after run 2580 failed 433 tests with `net::ERR_CONNECTION_REFUSED` (concurrent web-test containers share the host network on the shared runner). Main now carries the equivalent fix for this (3f1a084b, PR #779), including the root cause that `CI` is unset inside the Playwright image so `reuseExistingServer` was active in CI. During the merge of main this branch dropped its own port change in favour of main's implementation; the PR diff is now test-only. ## Verification - Local: `npx playwright test tests/albums.spec.ts tests/timeline.spec.ts tests/discovery-domain-selection.spec.ts tests/processing-issues.spec.ts` -> 102 tests, 0 failures (99 passed, 3 skipped) - Repeats: albums:1308 3/3, albums:1778 24/24, timeline:302 12/12, processing-issues:54 12/12 - `npm run check` -> 0 errors (20 pre-existing warnings) - CI run 2589 (final SHA, post-merge of main): scoped web suite 449 passed / 0 failed; all 10 target invocations (5 tests × desktop+mobile) green; the 5 flaky retries were unrelated specs, all passed on retry
test(web): harden flaky album/timeline/selection E2E specs
Some checks failed
Build and Deploy / Detect changed services and test scopes (push) Successful in 4s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Failing after 12m7s
0fd377f4d4
- albums:1308: replace per-row expect.poll loops (2 boundingBox round-trips
  each, ~200 polls) with a justify-complete wait plus one batched browser
  measurement per pattern; bounded 90s budget for the 65-pattern test.
- albums:1778: dispatch the "Select loaded" click directly. A coordinate click
  makes Playwright scroll the review container to an edge, triggering the
  app's scroll-based lazy day reveal and growing the loaded window mid-test.
- timeline:302: dispatch the hover-revealed section selection controls instead
  of coordinate clicks that race section re-renders dropping the CSS hover.
- discovery-domain-selection:129: toggle the 505 controls in-page via their
  real click handlers (60s -> ~1s) instead of coordinate clicks that scroll
  and re-render the 505-card grid per click.
- processing-issues:54: re-open the page actions menu until its items render,
  bounded via expect.poll, for occasionally lost toggle clicks on CI runners.
fix(ci): isolate Playwright preview port per web test job
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m23s
0270af1421
Web test containers run with --network host on the shared memories-deploy
runner and Playwright served the preview build on a fixed port 4173. Two
concurrent jobs (e.g. PR scoped tests plus the daily full suite or another
PR) collide: the second preview fails to bind, while Playwright's readiness
probe can be satisfied by the other job's server, which then disappears
mid-run. Run 2580 failed 433 tests with net::ERR_CONNECTION_REFUSED on
http://127.0.0.1:4173/ as a result.

Derive the port from PLAYWRIGHT_PORT (default 4173 for local dev) and pass a
randomized per-run port to the web test containers in both workflows.
test(web): fall back to port 4173 when PLAYWRIGHT_PORT is invalid
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 3s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 4m3s
c5a49f1251
Guards against a malformed or empty PLAYWRIGHT_PORT producing an
unbindable 127.0.0.1:NaN baseURL.
Merge remote-tracking branch 'origin/main' into fix/web-ci-stability
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 4s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 1m28s
48b5658920
# Conflicts:
#	apps/web/playwright.config.ts
test(web): tighten heavy album preview test budget to 60s
All checks were successful
Build and Deploy / Detect changed services and test scopes (push) Successful in 5s
Build and Deploy / Smoke API and media worker (push) Has been skipped
Build and Deploy / Smoke web (push) Has been skipped
Build and Deploy / Smoke ML worker (push) Has been skipped
Build and Deploy / Build plants worker image (push) Has been skipped
Build and Deploy / Build people worker image (push) Has been skipped
Build and Deploy / Build converter worker image (push) Has been skipped
Build and Deploy / Build media worker image (push) Has been skipped
Build and Deploy / Build API image (push) Has been skipped
Build and Deploy / Build ML worker image (push) Has been skipped
Build and Deploy / Build web image (push) Has been skipped
Build and Deploy / Deploy API (push) Has been skipped
Build and Deploy / Deploy web (push) Has been skipped
Build and Deploy / Deploy media worker (push) Has been skipped
Build and Deploy / Deploy ML worker (push) Has been skipped
Build and Deploy / Deploy plants worker (push) Has been skipped
Build and Deploy / Deploy people worker (push) Has been skipped
Build and Deploy / Deploy converter worker (push) Has been skipped
Build and Deploy / Clean Docker build artifacts (push) Has been skipped
Build and Deploy / Run scoped PR tests (push) Successful in 3m59s
74960ad19f
The optimized batched measurement runs in ~10s in CI (observed 9.5-10.9s),
so 60s keeps ample headroom while surfacing regressions sooner.
chris merged commit 1a5a719c2c into main 2026-09-20 23:24:04 +02:00
Sign in to join this conversation.
No reviewers
No labels
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
chris/memories!785
No description provided.