How to fix flaky visual tests

A flaky visual test is one that reports a difference when nothing in your code changed. It is the number one reason visual suites get disabled, and the instinct it produces — raise the threshold until the noise stops — is precisely the wrong move, because it raises it past the real regressions too.

Flake is nearly always non-determinism in the page, not imprecision in the comparison. Work through these in order; each one removes more noise than the one after it.

1. Fonts loading after the capture

The most common cause by a wide margin. A web font arrives after first paint, every line of text reflows, and a page that was correct a moment ago produces a full-page diff. Worse, it is timing-dependent, so it fails perhaps one run in six.

// Wait for every declared face to be loaded and applied.
await page.evaluate(() => document.fonts.ready);

• Self-host fonts so a third-party CDN is not in the critical path of your test.
• Preload the faces used above the fold.
• Use font-display: optional or swap deliberately, and know which one your baseline captured.

2. Animation and transition frames

Capturing mid-transition gives you an element at 60% opacity or 8px off its final position. Disabling CSS animation is table stakes, but JavaScript-driven motion needs handling separately — charts that ease in, numbers that count up, carousels, skeleton shimmers, and anything using requestAnimationFrame.

await page.addStyleTag({
  content: `*, *::before, *::after {
    animation: none !important;
    transition: none !important;
    scroll-behavior: auto !important;
  }`,
});

// For JS-driven motion, give the app an explicit off switch.
await page.goto("/dashboard?motion=off");

That query-string flag is worth building. Respecting prefers-reduced-motion in your application gets you most of it for free, and it is an accessibility improvement you owed your users anyway.

3. Time, dates, and randomness

Any rendered clock is a scheduled failure. Relative timestamps ('3 minutes ago'), date pickers defaulting to today, expiry countdowns, greetings that change at noon, and randomised IDs or placeholder content all drift between runs.

await page.clock.setFixedTime(new Date("2026-01-15T12:00:00Z"));

// Make randomness reproducible before the app's own scripts run.
await page.addInitScript(() => {
  let seed = 42;
  Math.random = () => {
    seed = (seed * 1103515245 + 12345) % 2147483648;
    return seed / 2147483648;
  };
});

4. Live data

A dashboard reading from production data will differ on every run, and no threshold fixes that. The options, best first:

1. 01Seed a fixed dataset for the test environment. Highest fidelity — real rendering paths, real edge cases.
2. 02Stub the network responses at the request layer, so the page renders from a fixture.
3. 03Mask the region and accept that you are no longer testing it.

Masking is a last resort that people reach for first. Every masked region is coverage you have given up, and the masks tend to grow until the screenshot is mostly magenta rectangles.

5. Capturing before the page is ready

A fixed waitForTimeout is both too slow on a fast run and too short on a slow one. Wait for a condition instead — the specific element that indicates the page has real content.

// Not this.
await page.waitForTimeout(3000);

// This: wait for the state that means the page is genuinely ready.
await expect(page.getByRole("table")).toBeVisible();
await expect(page.getByTestId("skeleton")).toHaveCount(0);
await page.waitForLoadState("networkidle");

6. Scrollbars and viewport chrome

Scrollbar rendering differs by operating system and by whether content overflows at all. A page that is 2px taller than the viewport on one run gains a scrollbar, which narrows the content area and shifts every centred element — a full-width diff from a two-pixel cause.

/* Applied only in the visual-test environment. */
html { scrollbar-width: none; }
html::-webkit-scrollbar { display: none; }

7. The rendering environment itself

Identical code renders differently across operating systems, GPU availability, and browser versions. If baselines are recorded on a developer's machine and compared on a Linux CI runner, antialiasing alone will produce constant low-level noise.

• Record and compare in the same container image, pinned to the same browser version.
• Pin timezoneId, locale, colorScheme, and deviceScaleFactor explicitly rather than inheriting the machine's.
• Re-record baselines deliberately after a browser or font-package upgrade, and do it in its own commit so the diff is reviewable.

8. Only now, tune the threshold

With the first seven addressed, a small tolerance absorbs the irreducible antialiasing noise. Coming to it last means you are tuning against a quiet baseline instead of hiding real problems.

expect: {
  toHaveScreenshot: {
    threshold: 0.2,           // per-pixel colour tolerance
    maxDiffPixelRatio: 0.01,  // fail above 1% of pixels differing
  },
}

Prefer maxDiffPixelRatio over an absolute maxDiffPixels count: a ratio scales sensibly across viewport sizes, where a fixed count is too strict on mobile and too lax on a wide desktop capture.

A triage order for an already-noisy suite

1. 01Make it non-blocking today, so it stops costing the team trust while you fix it.
2. 02Run the suite three times against an unchanged commit. Anything that differs across those runs is flake, not a regression — that is your work queue.
3. 03Group the flaky tests by cause rather than by test. One font fix usually resolves dozens at once.
4. 04Fix by cause, in the order above.
5. 05Re-record baselines once the suite is quiet, then restore blocking on the routes that have proven stable.

That three-run experiment is worth doing before any other work, and the approval rate it produces is the health metric worth reporting upward — see visual testing for engineering leaders. It separates 'the page is non-deterministic' from 'the baseline is stale', and those need completely different fixes.