Visual regression testing in GitHub Actions
A working GitHub Actions workflow for visual regression testing — matching baselines to the CI container, sharding, caching browsers, posting results to the PR, and handling fork secrets safely.
Most visual testing setups in GitHub Actions fail for the same two reasons, and neither is the test code: baselines recorded somewhere other than the CI container, and results that land where nobody reads them. This is a workflow that addresses both.
The pipeline-design reasoning behind these choices — preview deployments, blocking policy, budgets — is in visual testing in CI/CD pipelines. This is the implementation.
The workflow
name: Visual
on:
pull_request:
push:
branches: [main]
# A new push supersedes an in-flight run on the same PR.
concurrency:
group: visual-${{ github.ref }}
cancel-in-progress: true
jobs:
visual:
runs-on: ubuntu-latest
# Render in the same container that recorded the baselines.
container:
image: mcr.microsoft.com/playwright:v1.56.0-noble
options: --user 1001
strategy:
fail-fast: false
matrix:
shard: [1, 2, 3, 4]
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with:
node-version: 22
cache: npm
- run: npm ci
- run: npx playwright test --project=visual --shard=${{ matrix.shard }}/4
env:
BASE_URL: http://localhost:3000
- uses: actions/upload-artifact@v4
if: ${{ !cancelled() }}
with:
name: visual-report-${{ matrix.shard }}
path: |
playwright-report/
test-results/
retention-days: 7Recording baselines that CI will accept
Developers must record baselines in the same image. Make it a script so nobody does it by hand:
{
"scripts": {
"visual:update": "docker run --rm --network host -v \"$PWD\":/work -w /work mcr.microsoft.com/playwright:v1.56.0-noble npx playwright test --project=visual --update-snapshots"
}
}Pin the image tag to the @playwright/test version in package.json. When those drift, a browser upgrade changes rendering and you get a wave of diffs that look like product regressions.
Posting results where people read them
A red X in the checks list is not a review surface. Post a single comment and update it in place — a bot that appends a comment per run trains everyone to collapse it.
- name: Comment results on the PR
if: ${{ github.event_name == 'pull_request' && !cancelled() }}
uses: marocchino/sticky-pull-request-comment@v2
with:
header: visual-review # updates in place, one comment
path: visual-summary.mdGenerate visual-summary.md from the Playwright JSON reporter: which routes changed, how much, and a link to the uploaded artifact. Keep it short — a reviewer should be able to decide whether to look closer in about five seconds.
Speed
Cache browsers, not just node_modules
- uses: actions/cache@v4
with:
path: ~/.cache/ms-playwright
key: pw-${{ runner.os }}-${{ hashFiles('package-lock.json') }}
- run: npx playwright install --with-deps chromium
if: steps.cache.outputs.cache-hit != 'true'Unnecessary when using the container: image above, which ships the browsers — but essential if you run on a bare runner.
Only test what changed
- name: Determine affected routes
id: affected
run: |
CHANGED=$(git diff --name-only origin/${{ github.base_ref }}...HEAD)
# Map changed files to routes via your own manifest, then:
echo "routes=$(node scripts/affected-routes.mjs $CHANGED)" >> "$GITHUB_OUTPUT"
- run: npx playwright test --project=visual --grep "${{ steps.affected.outputs.routes }}"
if: steps.affected.outputs.routes != ''That affected-routes.mjs is the hard part and the reason most teams skip this step — mapping a diff to the routes it can reach needs the module graph, the route definitions, and an honest answer about global CSS. It is also the largest available saving, typically 80% or more of capture volume.
Shard, with fail-fast off
fail-fast: false matters. With the default, one shard failing cancels the others and the reviewer sees a partial picture of what changed.
Forks and secrets
Pull requests from forks do not receive your secrets, and that is correct. Do not reach for pull_request_target to work around it — that runs untrusted code with access to your secrets and is a well-known way to leak them.
visual:
# Authenticated visual runs only for branches in this repository.
if: ${{ github.event.pull_request.head.repo.full_name == github.repository }}
# ...
visual-public:
# Forks get the routes that need no credentials.
if: ${{ github.event.pull_request.head.repo.full_name != github.repository }}
# ...runs the marketing-only projectKeeping main's baseline current
update-baseline:
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
runs-on: ubuntu-latest
container:
image: mcr.microsoft.com/playwright:v1.56.0-noble
steps:
- uses: actions/checkout@v4
- uses: actions/setup-node@v4
with: { node-version: 22, cache: npm }
- run: npm ci
- run: npx playwright test --project=visual --update-snapshots
- uses: actions/upload-artifact@v4
with:
name: baseline-${{ github.sha }}
path: tests/**/*-snapshots/**Recapture on every merge, not on a schedule only loosely related to when the product changes. A pull request compared against a stale baseline is comparing against a state of the world that no longer exists.
Frequently asked questions
How do you run visual regression tests in GitHub Actions?
Run the job inside the same container image used to record baselines — for Playwright, mcr.microsoft.com/playwright pinned to your installed version. Shard across runners with fail-fast disabled, upload the report as an artifact whenever the job is not cancelled, and post a single sticky comment to the pull request.
Why do visual tests fail in GitHub Actions but pass locally?
Font rendering and antialiasing differ between operating systems, so baselines recorded on macOS will not match an Ubuntu runner. Use the container: key so the job runs in the same image you record baselines in, and pin that image tag to your Playwright version.
How do you handle secrets for visual tests on fork pull requests?
Forks do not receive repository secrets, by design. Gate the authenticated visual job on the pull request head repository matching your own, and run a credential-free subset for forks. Do not use pull_request_target to work around it — that exposes secrets to untrusted code.