DEVPROD-24935: Migrate from Cypress to Playwright for Parsley by minnakt · Pull Request #1483 · evergreen-ci/ui

minnakt · 2026-03-26T19:17:34Z

DEVPROD-24935

Description

Migrates tests from Cypress to Playwright. Sorry for how big this PR is 🧍‍♂️

Instructions:

Have your backend and frontend server running
Run pnpm playwright:ui, which will open the Playwright UI
Click on any test suite or specific test you want to run to see results

Cypress configuration still exists since Spruce still has Cypress tests.

Screenshots

* install ESLint plugin

sophstad

yay 😍 maybe when Spruce is also done we can migrate away from data-cy and the dataCy utils

sophstad · 2026-03-26T22:12:50Z

    Tasks.CheckCodegen,
    Tasks.Compile,
-    Tasks.E2E,
+    Tasks.E2EPlaywright,


[nit] Maybe we can just name this E2E and the other to e2e_cypress?

sophstad · 2026-03-26T22:50:08Z

+  use: {
+    baseURL: "http://localhost:5173",
+    viewport: { width: 1280, height: 800 },
+    video: process.env.CI ? "retain-on-failure" : "off",


Maybe we should use on-first-retry instead of retain-on-failure, it seems possible that recording video by default would be slightly slower

sophstad · 2026-03-26T22:50:48Z

+  forbidOnly: !!process.env.CI, // Fail if implementer accidentally left test.only in any file.
+  retries: process.env.CI ? 2 : 0,
+  workers: 1, // Disable parallelism - run tests serially like Cypress.
+  use: {


Let's try enabling traces, this looks freaking sick

So I think it is possible to get traces to be served from the HTML report, but the amount of things I have to upload would be kind of a lot. (I wish s3.put would just infer content_type?)

Instead I attached a .zip file which contains the traces and can be served locally via pnpm exec playwright show-report playwright-html-report

sophstad · 2026-03-26T22:57:13Z

    "storybook": "NO_HTTPS=true storybook dev -p 6007",
    "storybook:build": "env-cmd -e local storybook build",
-    "test": "vitest --typecheck=false",
+    "test": "vitest --typecheck=false --exclude 'playwright/**'",


Would it maybe be preferable to scope Vitest using include to only look at the src directory? Either way let's move the option to Vitest config instead of the script

sophstad · 2026-03-27T16:22:55Z

+  for (let i = 0; i < conditions.length; i++) {
+    if (!conditions[i]) {
+      hasOutOfBoundsValue = true;
+      console.log(`Out of bounds value: ${i} ${conditions[i]}`);


Not sure if we really need this log, looks slightly awk in the logs (or could somehow log on failure only? idk)

https://parsley.corp.mongodb.com/evergreen/evergreen_ui_parsley_e2e_patch_ea915a386dace88758de7f0351768bc1ec20d457_69c6a766d05ffb00078d3a53_26_03_27_15_51_12/0/task?bookmarks=0%2C839&selectedLineRange=L819&shareLine=819

Sorry yeah this is slop

ha you're okay I think it was copypasta from a cy.log()

sophstad · 2026-03-27T16:25:41Z

+  await page.locator(`[aria-label="Collapse navigation"]`).click();
+};
+
+export const validateToast = async (


Would be nice if we can somehow share these utils when Spruce is added

sophstad · 2026-03-27T16:26:27Z

@@ -0,0 +1 @@
+export * from "./commands";


I'd be okay having all the helper functions here, seems like this is maybe Cypress cruft

sophstad · 2026-03-27T16:31:30Z

+};
+const toastDataCy = "toast";
+
+export const getByDataCy = (page: Page, value: string) =>


Hm do you think we should use a custom test attribute instead of requiring this helper everywhere? Or maybe just followup work?

sophstad · 2026-03-27T16:35:44Z

+    - command: s3.put
+      type: system
+      params:
+        role_arn: ${ASSUME_ROLE_ARN}
+        local_files_include_filter: ["ui/${app_dir}/bin/playwright/html/data/**/*.png"]
+        remote_file: ${build_variant}/${task_id}/${execution}/playwright/html/data/
+        bucket: evg-bucket-evergreen-ui
+        content_type: image/png
+        permissions: public-read
+    - command: s3.put
+      type: system
+      params:
+        role_arn: ${ASSUME_ROLE_ARN}
+        local_files_include_filter: ["ui/${app_dir}/bin/playwright/html/data/**/*.webm"]
+        remote_file: ${build_variant}/${task_id}/${execution}/playwright/html/data/
+        bucket: evg-bucket-evergreen-ui
+        content_type: video/webm
+        permissions: public-read
+    - command: s3.put
+      type: system
+      params:
+        role_arn: ${ASSUME_ROLE_ARN}
+        local_files_include_filter: ["ui/${app_dir}/bin/playwright/html/data/**/*.md"]
+        remote_file: ${build_variant}/${task_id}/${execution}/playwright/html/data/
+        bucket: evg-bucket-evergreen-ui
+        content_type: text/markdown
+        permissions: public-read


I think maybe we should be grabbing these assets from bin/playwright/test-results instead of /html. That makes them not have hashed asset names, and allows us to directly download the trace.zip files which is ideal. We could also use view remote traces except for a CORS issue I haven't figured out yet 😿

I tried it on this patch (just look at changes in .evergreen/attach.yml)

sophstad · 2026-03-27T16:42:09Z

+        permissions: private
+        visibility: signed
+        display_name: "Playwright HTML Report (zip)"
+    - command: s3.put


Similar changes in the patch I linked -- it is slightly easier imo to just upload the whole /html directory and use preserive_path... and I added a special display name upload so the index file is clear. What do you think?

sophstad · 2026-03-27T16:45:36Z

+    // Intercept GraphQL queries to detect mutations; match both relative and absolute paths.
+    await page.route("**/graphql/query", async (route) => {
+      const request = route.request();
+      try {
+        const postData = request.postDataJSON();
+        if (postData?.query?.startsWith("mutation")) {
+          mutationDispatched = true;
+        }
+      } catch (e) {
+        console.error("Failed to parse GraphQL request:", e);
+      }
+      await route.continue();
+    });


Nice (but I want to rip it out 😈)

sophstad · 2026-03-27T16:47:12Z

 body.txt-e
+
+# Playwright
+/test-results/


I believe this is being written to /bin so can be removed (playwright-report and blob-report may be the same)

sophstad · 2026-03-27T16:48:24Z

    "postversion": "pnpm deploy-utils-postversion",
    "prepare": "pnpm --dir ../.. prepare",
-    "preview": "serve -s dist -p 4173",
+    "preview": "serve -s dist -p 5173",


I think this should also be updated in env-cmdrc.json

sophstad

This is looking pretty good! Main question is whether there's any way to include the test name instead of just the filename for failed assets 🤔 if not that's okay too

minnakt · 2026-04-09T00:32:16Z

added test names to uploaded files

sophstad

WAHOOOO

sophstad · 2026-04-09T13:58:46Z

+        remote_file: ${build_variant}/${task_id}/${execution}/report/
+        bucket: evg-bucket-evergreen-ui
+        content_type: text/html
+        visibility: none


YAY this is geniussss

sophstad · 2026-04-09T13:59:25Z

+        bucket: evg-bucket-evergreen-ui
+        content_type: image/png
+        permissions: public-read
+        display_name: "screenshot:"


[super nit] The names here seem prefixed with the type so could remove if you want, up to you

Suggested change

display_name: "screenshot:"

sophstad · 2026-04-09T13:59:46Z

+        bucket: evg-bucket-evergreen-ui
+        content_type: video/webm
+        permissions: public-read
+        display_name: "video:"


Here too (giving you github suggestions to make it easy to apply)

Suggested change

display_name: "video:"

sophstad · 2026-04-09T13:59:54Z

+        bucket: evg-bucket-evergreen-ui
+        content_type: application/zip
+        permissions: public-read
+        display_name: "trace:"


Suggested change

display_name: "trace:"

minnakt added 6 commits March 25, 2026 16:29

feat: translate Cypress tests into Playwright

ae2595a

* install ESLint plugin

delete Cypress files

f3119e4

feat: add to Evergreen CI pipeline

ff20fdf

fix: import

fadfeb8

Merge branch 'main' into DEVPROD-24935_parsley

195ec70

fix: no more cookies in Parsley

c7e950d

minnakt added the parsley label Mar 26, 2026

minnakt added 3 commits March 26, 2026 15:29

fix: don't include playwright tests in test command

8bc474d

fix: change permissions

9e30e3b

try to make less flaky

eefe872

minnakt marked this pull request as ready for review March 26, 2026 20:01

minnakt requested a review from a team as a code owner March 26, 2026 20:01

sophstad reviewed Mar 26, 2026

View reviewed changes

minnakt added 2 commits March 27, 2026 00:19

fix: CR

fddab1c

attach trace zip files

3b7ba7e

sentry bot reviewed Mar 27, 2026

View reviewed changes

Comment thread .evergreen/attach.yml Outdated

provide zip file with traces

3b1057a

minnakt requested a review from sophstad March 27, 2026 14:29

sophstad requested changes Mar 27, 2026

View reviewed changes

sophstad mentioned this pull request Mar 30, 2026

DEVPROD-27495 Add ability to exclude timestamps from logs #1344

Merged

minnakt added 8 commits March 30, 2026 20:03

Merge branch 'main' into DEVPROD-24935_parsley

1e5fe43

get rid of console log

dd62e1c

use getByTestId

0bd101c

use recommended setup and teardown pattern

87dfa7c

fix unmigrated assertions

3e4aaf4

update .gitignore

4783505

update URL

a31a2f3

apply suggestions for uploads

86ebad2

minnakt requested a review from sophstad March 31, 2026 01:47

sophstad reviewed Apr 2, 2026

View reviewed changes

minnakt added 4 commits April 3, 2026 13:32

Merge branch 'main' into DEVPROD-24935_parsley

6e4588c

fix: errors from merge conflict

bd804c4

create custom reporter

e9b508a

fix: errors from merge conflicts

7f5c8ab

sentry bot reviewed Apr 3, 2026

View reviewed changes

Comment thread .evergreen/shared.yml

minnakt added 2 commits April 8, 2026 20:05

Merge branch 'main' into DEVPROD-24935_parsley

1586ace

use visibility

8d9c792

minnakt requested a review from sophstad April 9, 2026 00:35

sophstad approved these changes Apr 9, 2026

View reviewed changes

minnakt added 2 commits April 9, 2026 11:52

clean up reporter code and add tickets

0cef249

Merge branch 'main' into DEVPROD-24935_parsley

2edbf6f

minnakt merged commit 8214059 into evergreen-ci:main Apr 9, 2026
12 checks passed

minnakt deleted the DEVPROD-24935_parsley branch April 9, 2026 19:25

Conversation

minnakt commented Mar 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Screenshots

Uh oh!

sophstad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sophstad Mar 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sophstad left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

minnakt commented Apr 9, 2026

Uh oh!

sophstad left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

minnakt commented Mar 26, 2026 •

edited

Loading

sophstad Mar 27, 2026 •

edited

Loading