Skip to content

perf(browser): reduce matching screenshot overhead#10278

Open
kasperpeulen wants to merge 5 commits intovitest-dev:mainfrom
kasperpeulen:optimize-screenshot-matcher-fast-path
Open

perf(browser): reduce matching screenshot overhead#10278
kasperpeulen wants to merge 5 commits intovitest-dev:mainfrom
kasperpeulen:optimize-screenshot-matcher-fast-path

Conversation

@kasperpeulen
Copy link
Copy Markdown

@kasperpeulen kasperpeulen commented May 5, 2026

Summary

  • Adds a raw PNG byte-equality fast path for matching screenshots with the built-in comparator.
  • Supports expect(page).toMatchScreenshot() by passing an explicit page target through the browser matcher so providers can use native page-level screenshots instead of body locator screenshots.
  • Reuses captured PNG buffers when writing reference/actual artifacts, avoiding unnecessary re-encoding on update and mismatch paths.

Why

This came from profiling a downstream browser visual-regression suite where toMatchScreenshot had become the dominant local-test cost. The suite captures every known theme/viewport variant, so a single test file can produce many screenshots and small per-screenshot overhead quickly turns into seconds of local feedback time.

The key profiling result was that the comparator was not the bottleneck. Most time was spent before comparison: asking Playwright for screenshots and decoding PNGs that often turned out to be byte-identical to the stored reference.

For the common passing case, if the browser returns the exact same PNG bytes as the reference, Vitest does not need to decode both images or run pixel comparison. This PR short-circuits only that known-safe built-in comparator path while preserving the existing fallback behavior for mismatches, updates, unstable screenshots, and custom comparators.

Profiling Notes

Before this patch, the downstream suite was spending about 31.97s wall time with screenshot assertions enabled versus about 16s with screenshot assertions disabled.

Instrumentation around the Vitest matcher and Playwright provider showed this distribution for the full browser run:

Segment Count / total Average
Screenshot assertions 160 screenshots (40 tests x 4 variants) -
Total matcher time 84.9s summed 530ms / screenshot
Playwright element.screenshot() 68.7s summed 429ms / screenshot
PNG decode 11.5s summed 72ms / screenshot
BlazeDiff compare 95.7ms summed 0.6ms / screenshot
Stable retries 0 -

A smaller dedicated probe showed the same shape: compare was usually 0-7ms, while the useful work was dominated by roughly 90-110ms of Playwright capture plus 35-75ms of PNG decode per screenshot.

A separate Playwright probe also showed that, for explicit page captures, going through the native page.screenshot() API was faster than routing the same capture through a body/document locator. That is the motivation for the explicit expect(page) / target: 'page' path instead of treating page screenshots as another element screenshot.

This PR does not make expect(page).toMatchScreenshot() imply full-page screenshots. It preserves the caller/configured screenshotOptions.fullPage value, so apps that want full-document captures can opt into fullPage: true themselves.

That is why this patch targets the matching/reference path, page-level capture path, and PNG work rather than the diff algorithm itself.

Local Performance

Measured in the downstream app that motivated this change, using 4 workers against the same visual suite:

Scenario Wall time
Before these changes ~31s
After these changes ~20-21s
Screenshot expect disabled ~16s

So the visual assertion overhead dropped to roughly 4-5s over the same suite without screenshot assertions. Compared with the original ~31s run, this removes about 10s of local feedback time while still always running the visual assertions.

These numbers are from an app-level suite, not Vitest's own fixtures, but they are the reason for the shape of the patch: avoid decode/compare work when the captured PNG is already byte-identical to the reference, use the faster native page capture path for explicit page screenshots, and avoid extra encode work when writing screenshots that were already captured as PNG buffers.

Safety Notes

  • The byte fast path is intentionally limited to the built-in comparator path. Custom comparators are still called even if PNG bytes match, because they may have custom semantics or side effects.
  • expect(page).toMatchScreenshot() uses explicit page-target handling, but it does not force fullPage: true; full-page screenshots remain controlled by screenshotOptions.fullPage.
  • Existing element screenshot behavior and plain page.screenshot() body fallback are kept separate.
  • Mismatch and update paths still produce the same reference/actual/diff artifacts; they can reuse the captured PNG buffer for actual/reference writes instead of encoding it again.

Test Plan

  • pnpm typecheck
  • CI=true PROVIDER=playwright TEST_BROWSER=chromium pnpm -C test/browser run test-expect-dom toMatchScreenshot.test.ts
  • CI=true PROVIDER=webdriverio TEST_BROWSER=chrome pnpm -C test/browser run test-expect-dom toMatchScreenshot.test.ts

Avoid decoding byte-identical PNG screenshots for built-in comparisons and add page-target screenshot support so full-page assertions can use the browser provider directly.
@netlify
Copy link
Copy Markdown

netlify Bot commented May 5, 2026

Deploy Preview for vitest-dev ready!

Built without sensitive environment variables

Name Link
🔨 Latest commit 1ae498e
🔍 Latest deploy log https://app.netlify.com/projects/vitest-dev/deploys/69fa3521dec3110008a008ea
😎 Deploy Preview https://deploy-preview-10278--vitest-dev.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.
🤖 Make changes Run an agent on this branch

To edit notification comments on pull requests, go to your Netlify project configuration.

Document that byte-level short circuiting only applies to the built-in comparator and make the WebdriverIO page screenshot fallback explicit.
@kasperpeulen kasperpeulen changed the title perf(browser): speed up screenshot matcher perf(browser): reduce screenshot matcher overhead May 5, 2026
@kasperpeulen kasperpeulen changed the title perf(browser): reduce screenshot matcher overhead perf(browser): reduce matching screenshot overhead May 5, 2026
@sheremet-va sheremet-va requested review from AriPerkkio and macarie May 7, 2026 08:08
macarie
macarie previously approved these changes May 7, 2026
Copy link
Copy Markdown
Member

@macarie macarie left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I love the idea behind this change, great work! 🙌🏼

sheremet-va
sheremet-va previously approved these changes May 7, 2026
Copy link
Copy Markdown
Member

@sheremet-va sheremet-va left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me, just a small nitpick 😄

// element and plain `page.screenshot()` calls keep the existing body fallback.
const buffer = options.target === 'page'
? await context.browser.saveScreenshot(normalizedSavePath)
: await (await context.browser.$(options.element?.selector ?? 'body')).saveScreenshot(
Copy link
Copy Markdown
Member

@sheremet-va sheremet-va May 7, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think the double await is required here; context.browser.$ is chainable

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, updated to use the chainable element directly in 5334ae3.

@kasperpeulen kasperpeulen dismissed stale reviews from sheremet-va and macarie via 5334ae3 May 7, 2026 17:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants