fix: Prepare CI to properly run Electron on ubuntu-latest (24.04) #34238

agg23 · 2025-01-07T15:50:45Z

GitHub Actions updated the ubuntu-latest tag to point to Ubuntu 24.04 without much of an announcement (see actions/runner-images#10636). This included removing several required packages for running Electron, causing our Electron smoke tests to fail.

.github/workflows/tests_primary.yml

mxschmitt · 2025-01-08T17:53:08Z

.github/actions/run-test/action.yml

+    - name: Setup Ubuntu Binary Installation
+      if: ${{ runner.os == 'Linux' }}
+      run: |
+        if grep -q "Ubuntu 24" /etc/os-release; then


Lets add a todo to remove it after electron/electron#42510 has been fixed?

Also maybe do it only if command contains itest or etest?

github-actions · 2025-01-08T19:14:50Z

Test results for "tests 1"

5 flaky

⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-ubuntu-22.04-node18
⚠️ [installation tests] › tests/playwright-test-package-managers.spec.ts:54:5 › npm: uninstalling ct removes playwright bin @package-installations-windows-latest
⚠️ [chromium-library] › tests/library/video.spec.ts:581:5 › screencast › should capture static page in persistent context @smoke @ubuntu-20.04-chromium-tip-of-tree
⚠️ [webkit-library] › tests/library/selector-generator.spec.ts:321:7 › selector generator › should prioritize attributes correctly › name @webkit-ubuntu-22.04-node18
⚠️ [playwright-test] › tests/ui-mode-test-watch.spec.ts:145:5 › should watch all @windows-latest-node18-1

37546 passed, 649 skipped
✔️✔️✔️

Merge workflow run.

github-actions · 2025-01-08T20:00:28Z

Test results for "tests 2"

2 fatal errors, not part of any test
13 failed
❌ [chromium-library] › tests/library/proxy.spec.ts:93:11 › should proxy local network requests › with other bypasses › link-local @chrome-beta-macos-latest
❌ [chromium-library] › tests/library/proxy.spec.ts:125:3 › should authenticate @chrome-beta-macos-latest
❌ [chromium-library] › tests/library/browsertype-connect.spec.ts:667:5 › run-server › should fulfill with global fetch result @chrome-macos-latest
❌ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @chrome-macos-latest
❌ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:466:7 › cli codegen › should save assets via SIGINT @chrome-macos-latest
❌ [chromium-library] › tests/library/downloads-path.spec.ts:105:5 › downloads path › should delete downloads when persistent context closes @msedge-beta-macos-latest
❌ [chromium-library] › tests/library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-dev-macos-latest
❌ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-dev-macos-latest
❌ [chromium-library] › tests/library/proxy.spec.ts:93:11 › should proxy local network requests › by default › loopback address @msedge-dev-macos-latest
❌ [installation tests] › tests/playwright-electron-should-work.spec.ts:21:5 › electron should work @package-installations-ubuntu-latest-node20
❌ [installation tests] › tests/playwright-electron-should-work.spec.ts:31:5 › electron should work with special characters in path @package-installations-ubuntu-latest-node20
❌ [installation tests] › tests/playwright-electron-should-work.spec.ts:44:5 › should work when wrapped inside @playwright/test and trace is enabled @package-installations-ubuntu-latest-node20
❌ [webkit-library] › tests/library/trace-viewer.spec.ts:1598:1 › should show a popover @webkit-macos-13-xlarge

109 flaky

⚠️ [chromium-library] › tests/library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @channel-chromium-macos-latest
⚠️ [chromium-library] › tests/library/browsertype-launch-server.spec.ts:23:5 › launch server › should work @channel-chromium-macos-latest
⚠️ [chromium-library] › tests/library/browsertype-launch-server.spec.ts:50:5 › launch server › should work when wsPath is missing leading slash @channel-chromium-macos-latest
⚠️ [chromium-library] › tests/library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @channel-chromium-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @channel-chromium-windows-latest
⚠️ [chromium-library] › tests/library/trace-viewer.spec.ts:1534:1 › canvas clipping @channel-chromium-windows-latest
⚠️ [chromium-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @channel-chromium-windows-latest
⚠️ [chromium-library] › tests/library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-fetch.spec.ts:1229:3 › should work with connectOverCDP @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/inspector/pause.spec.ts:23:3 › should resume when closing inspector @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/proxy.spec.ts:31:3 › should use proxy @smoke @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/proxy.spec.ts:146:3 › should work with authenticate followed by redirect @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @chrome-beta-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @chrome-beta-windows-latest
⚠️ [chromium-library] › tests/library/browser.spec.ts:54:5 › should dispatch page.on(close) upon browser.close and reject evaluate @chrome-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-fetch-algorithms.spec.ts:77:9 › algorithms › br decompression › should not fail with an empty response without content-length header (Z_BUF_ERROR) @chrome-macos-latest
⚠️ [chromium-library] › tests/library/browsertype-connect.spec.ts:735:5 › run-server › setInputFiles should preserve lastModified timestamp @chrome-macos-latest
⚠️ [chromium-library] › tests/library/chromium/connect-over-cdp.spec.ts:317:5 › should connect via https @chrome-macos-latest
⚠️ [chromium-library] › tests/library/fetch-proxy.spec.ts:21:3 › context request should pick up proxy credentials @chrome-macos-latest
⚠️ [chromium-page] › tests/page/page-mouse.spec.ts:244:3 › should tween mouse movement @chromium-headed-ubuntu-24.04
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-3.spec.ts:171:7 › cli codegen › should generate frame locators (4) @chromium-tip-of-tree-macos-13
⚠️ [chromium-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @chromium-tip-of-tree-macos-13
⚠️ [chromium-library] › tests/library/trace-viewer.spec.ts:1221:1 › should pick locator in iframe @chromium-tip-of-tree-macos-13--headed
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-3.spec.ts:498:7 › cli codegen › should generate getByAltText @chromium-tip-of-tree-ubuntu-20.04--headed
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-aria.spec.ts:77:7 › should update aria snapshot highlight @chromium-tip-of-tree-windows-latest--headed
⚠️ [chromium-library] › tests/library/trace-viewer.spec.ts:1534:1 › canvas clipping @chromium-windows-latest
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-beta-ubuntu-20.04
⚠️ [firefox-library] › tests/library/trace-viewer.spec.ts:1551:1 › canvas clipping in iframe @firefox-beta-windows-latest
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-beta-windows-latest
⚠️ [firefox-library] › tests/library/capabilities.spec.ts:252:3 › requestFullscreen @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @firefox-headed-macos-14-xlarge
⚠️ [firefox-library] › tests/library/inspector/cli-codegen-1.spec.ts:930:7 › cli codegen › should clear when recording is disabled @firefox-headed-ubuntu-24.04
⚠️ [firefox-page] › tests/page/page-add-locator-handler.spec.ts:230:5 › should work when owner frame detaches @firefox-headed-ubuntu-24.04
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-headed-ubuntu-24.04
⚠️ [firefox-library] › tests/library/browsercontext-basic.spec.ts:34:3 › should be able to click across browser contexts @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/browsercontext-har.spec.ts:455:3 › should update har.zip for page @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/inspector/cli-codegen-1.spec.ts:828:7 › cli codegen › should record slider @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/inspector/console-api.spec.ts:45:3 › should support playwright.selector @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/inspector/console-api.spec.ts:50:3 › should support playwright.locator.value @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/inspector/console-api.spec.ts:67:3 › should support playwright.locator({ has }) @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/inspector/console-api.spec.ts:78:3 › should support playwright.locator({ hasNot }) @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/selectors-register.spec.ts:29:3 › should work @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/trace-viewer.spec.ts:1551:1 › canvas clipping in iframe @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/tracing.spec.ts:506:5 › should work with multiple chunks @firefox-headed-windows-latest
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-headed-windows-latest
⚠️ [firefox-page] › tests/page/page-event-request.spec.ts:169:3 › should return response body when Cross-Origin-Opener-Policy is set @firefox-headed-windows-latest
⚠️ [firefox-library] › tests/library/trace-viewer.spec.ts:1221:1 › should pick locator in iframe @firefox-macos-13-large
⚠️ [firefox-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @firefox-macos-13-large
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-macos-13-large
⚠️ [firefox-library] › tests/library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @firefox-macos-14-large
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-macos-14-large
⚠️ [firefox-library] › tests/library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @firefox-ubuntu-20.04
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-ubuntu-20.04
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-ubuntu-24.04
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @firefox-windows-latest
⚠️ [chromium-library] › tests/library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-fetch.spec.ts:1229:3 › should work with connectOverCDP @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:86:1 › should reset serviceworker that hangs in importScripts @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsertype-connect.spec.ts:161:5 › launchServer › should be able to visit ipv6 @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/video.spec.ts:360:5 › screencast › should expose video path blank popup @msedge-beta-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-ubuntu-20.04
⚠️ [chromium-library] › tests/library/browsercontext-proxy.spec.ts:107:11 › should proxy local network requests › by default › loopback address @msedge-beta-windows-latest
⚠️ [chromium-library] › tests/library/browsercontext-proxy.spec.ts:294:3 › should exclude patterns @msedge-beta-windows-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-beta-windows-latest
⚠️ [chromium-library] › tests/library/browsercontext-base-url.spec.ts:37:3 › should construct a new URL when a baseURL in browserType.launchPersistentContext is passed to page.goto @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-proxy.spec.ts:27:3 › should work when passing the proxy only on the context level @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:466:7 › cli codegen › should save assets via SIGINT @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-csharp.spec.ts:216:7 › should work with --save-har in nunit @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-java.spec.ts:92:5 › should work with --save-har and --save-har-glob as java-library @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/popup.spec.ts:264:3 › should not throw when click closes popup @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/proxy.spec.ts:93:11 › should proxy local network requests › by default › link-local @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/proxy.spec.ts:93:11 › should proxy local network requests › with other bypasses › localhost @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @msedge-dev-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-ubuntu-20.04
⚠️ [chromium-library] › tests/library/browsercontext-proxy.spec.ts:294:3 › should exclude patterns @msedge-dev-windows-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-dev-windows-latest
⚠️ [chromium-library] › tests/library/beforeunload.spec.ts:20:3 › should close browser with beforeunload page @msedge-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:456:7 › cli codegen › should --save-trace @msedge-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-2.spec.ts:466:7 › cli codegen › should save assets via SIGINT @msedge-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-java.spec.ts:105:5 › should work with --save-har and --save-har-glob as java-junit @msedge-macos-latest
⚠️ [chromium-library] › tests/library/inspector/cli-codegen-javascript.spec.ts:87:5 › should save the codegen output to a file if specified @msedge-macos-latest
⚠️ [chromium-library] › tests/library/browsercontext-reuse.spec.ts:30:1 › should re-add binding after reset @msedge-ubuntu-20.04
⚠️ [chromium-library] › tests/library/video.spec.ts:381:5 › screencast › should capture navigation @tracing-chromium-tip-of-tree
⚠️ [firefox-page] › tests/page/page-evaluate.spec.ts:403:3 › should throw for too deep reference chain @tracing-firefox
⚠️ [webkit-library] › tests/library/browsercontext-device.spec.ts:45:5 › device › should scroll to click @tracing-webkit
⚠️ [webkit-library] › tests/library/screenshot.spec.ts:44:14 › page screenshot › should work with a mobile viewport @tracing-webkit
⚠️ [webkit-library] › tests/library/screenshot.spec.ts:77:14 › page screenshot › should work with device scale factor @tracing-webkit
⚠️ [webkit-library] › tests/library/screenshot.spec.ts:96:14 › page screenshot › should work with device scale factor and scale:css @tracing-webkit
⚠️ [webkit-library] › tests/library/browsertype-connect.spec.ts:609:5 › run-server › should save har @webkit-headed-macos-14-xlarge
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:611:7 › cli codegen › should select @webkit-headed-macos-14-xlarge
⚠️ [webkit-page] › tests/page/elementhandle-misc.spec.ts:27:3 › should hover when Node is removed @webkit-headed-ubuntu-24.04
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:24:7 › cli codegen › should click @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:117:7 › cli codegen › should click after same-document navigation @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:299:7 › cli codegen › should fill japanese text @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:333:7 › cli codegen › should fill textarea @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-1.spec.ts:484:7 › cli codegen › should emit single keyup on ArrowDown @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/inspector/cli-codegen-pick-locator.spec.ts:36:7 › should update locator highlight @webkit-headed-windows-latest
⚠️ [webkit-library] › tests/library/modernizr.spec.ts:34:3 › Safari Desktop @webkit-headed-windows-latest
⚠️ [webkit-page] › tests/page/page-request-fallback.spec.ts:239:5 › post data › should amend binary post data @webkit-macos-15-large
⚠️ [webkit-library] › tests/library/trace-viewer.spec.ts:1524:1 › canvas disabled title @webkit-macos-15-xlarge
⚠️ [webkit-library] › tests/library/tracing.spec.ts:432:14 › should produce screencast frames crop @webkit-macos-15-xlarge
⚠️ [webkit-library] › tests/library/tracing.spec.ts:432:14 › should produce screencast frames scale @webkit-macos-15-xlarge
⚠️ [webkit-page] › tests/page/wheel.spec.ts:70:3 › should dispatch wheel events after context menu was opened @webkit-macos-15-xlarge
⚠️ [webkit-library] › tests/library/browsercontext-viewport-mobile.spec.ts:87:5 › mobile viewport › should support window.orientation emulation @webkit-ubuntu-20.04
⚠️ [webkit-library] › tests/library/browsercontext-viewport-mobile.spec.ts:157:5 › mobile viewport › mouse should work with mobile viewports and cross process navigations @webkit-ubuntu-20.04

245835 passed, 9486 skipped
✔️✔️✔️

Merge workflow run.

github-actions · 2025-01-08T20:35:01Z

Test results for "tests others"

16720 passed, 430 skipped
✔️✔️✔️

Merge workflow run.

fix: Prepare CI to properly run Electron on ubuntu-latest (24.04)

68c7499

mxschmitt reviewed Jan 7, 2025

View reviewed changes

.github/workflows/tests_primary.yml Outdated Show resolved Hide resolved

Try built-in Playwright Chromium deps installation for Ubuntu

5ad5d1a