Skip to content

Playwright fixes and improvements #3844

Playwright fixes and improvements

Playwright fixes and improvements #3844

Triggered via pull request January 8, 2025 09:19
Status Success
Total duration 14m 25s
Artifacts 8

end-to-end-tests.yaml

on: pull_request
Build Element-Web
1m 51s
Build Element-Web
Matrix: playwright
end-to-end-tests
57s
end-to-end-tests
Fit to window
Zoom out
Zoom in

Annotations

16 errors, 30 warnings, and 6 notices
[Chrome] › audio-player/audio-player.spec.ts:199:9 › Audio player › should support downloading an audio file @no-firefox @no-webkit: playwright/e2e/audio-player/audio-player.spec.ts#L125
1) [Chrome] › audio-player/audio-player.spec.ts:199:9 › Audio player › should support downloading an audio file @no-firefox @no-webkit Test timeout of 30000ms exceeded while running "beforeEach" hook. 123 | }; 124 | > 125 | test.beforeEach(async ({ page, app, user }) => { | ^ 126 | await app.client.createRoom({ name: "Test Room" }); 127 | await app.viewRoomByName("Test Room"); 128 | at /home/runner/work/element-web/element-web/playwright/e2e/audio-player/audio-player.spec.ts:125:10
[Chrome] › audio-player/audio-player.spec.ts:199:9 › Audio player › should support downloading an audio file @no-firefox @no-webkit: playwright/pages/ElementAppPage.ts#L96
1) [Chrome] › audio-player/audio-player.spec.ts:199:9 › Audio player › should support downloading an audio file @no-firefox @no-webkit Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('tree', { name: 'Rooms' }).locator('[title="Test Room"],[aria-label="Test Room"]').first() at ../pages/ElementAppPage.ts:96 94 | .locator(`[title="${name}"],[aria-label="${name}"]`) 95 | .first() > 96 | .click(); | ^ 97 | } 98 | 99 | public async viewRoomById(roomId: string): Promise<void> { at ElementAppPage.viewRoomByName (/home/runner/work/element-web/element-web/playwright/pages/ElementAppPage.ts:96:14) at /home/runner/work/element-web/element-web/playwright/e2e/audio-player/audio-player.spec.ts:127:19
[Chrome] › share-dialog/share-dialog.spec.ts:19:9 › Share dialog › should share a room @screenshot: playwright/element-web-test.ts#L313
1) [Chrome] › share-dialog/share-dialog.spec.ts:19:9 › Share dialog › should share a room @screenshot Error: expect.toHaveScreenshot(share-dialog-room.png): Test ended. Call log: - expect.toHaveScreenshot(share-dialog-room.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByRole('dialog', { name: 'Share room' }) at ../element-web-test.ts:313 311 | 312 | const screenshotName = sanitizeFilePathBeforeExtension(name); > 313 | await baseExpect(receiver).toHaveScreenshot(screenshotName, options); | ^ 314 | 315 | await style.evaluate((tag) => tag.remove()); 316 | at Object.toMatchScreenshot (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:313:9)
[Chrome] › share-dialog/share-dialog.spec.ts:32:9 › Share dialog › should share a room member @screenshot: playwright/element-web-test.ts#L313
2) [Chrome] › share-dialog/share-dialog.spec.ts:32:9 › Share dialog › should share a room member @screenshot Error: expect.toHaveScreenshot(share-dialog-user.png): Target page, context or browser has been closed at ../element-web-test.ts:313 311 | 312 | const screenshotName = sanitizeFilePathBeforeExtension(name); > 313 | await baseExpect(receiver).toHaveScreenshot(screenshotName, options); | ^ 314 | 315 | await style.evaluate((tag) => tag.remove()); 316 | at Object.toMatchScreenshot (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:313:9)
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot: playwright/element-web-test.ts#L313
3) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot Error: expect.toHaveScreenshot(share-dialog-event.png): Test ended. Call log: - expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms - verifying given screenshot expectation - waiting for getByRole('dialog', { name: 'Share Room Message' }) - locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div> - taking element screenshot - disabled all CSS animations - waiting for fonts to load... - fonts loaded - attempting scroll into view action - waiting for element to be stable at ../element-web-test.ts:313 311 | 312 | const screenshotName = sanitizeFilePathBeforeExtension(name); > 313 | await baseExpect(receiver).toHaveScreenshot(screenshotName, options); | ^ 314 | 315 | await style.evaluate((tag) => tag.remove()); 316 | at Object.toMatchScreenshot (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:313:9)
[Chrome] › sliding-sync/sliding-sync.spec.ts:366:5 › Sliding Sync › should send unsubscribe_rooms for every room switch: playwright/e2e/sliding-sync/sliding-sync.spec.ts#L383
4) [Chrome] › sliding-sync/sliding-sync.spec.ts:366:5 › Sliding Sync › should send unsubscribe_rooms for every room switch Error: expect(received).not.toHaveProperty(path) Matcher error: received value must not be null nor undefined Received has value: undefined 381 | } 382 | expect(body.unsubscribe_rooms).toEqual([unsubRoomId]); > 383 | expect(body.room_subscriptions).not.toHaveProperty(unsubRoomId); | ^ 384 | expect(body.room_subscriptions).toHaveProperty(subRoomId); 385 | }; 386 | at assertUnsubExists (/home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:383:49) at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:404:9
[Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot: playwright/e2e/right-panel/right-panel.spec.ts#L51
1) [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected) Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') Expected string: "mx_EditableItem_item" Received: <element(s) not found> Call log: - expect.toHaveClass with timeout 5000ms - waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#loremIpsumDolorSitAmetConsecteturAdipisicingElitSedDoEiusmodTemporIncididuntUtLaboreEtDoloreMagnaAliqua:localhost') 49 | await localAddresses.getByRole("textbox").fill(ROOM_ADDRESS_LONG); 50 | await localAddresses.getByRole("button", { name: "Add" }).click(); > 51 | await expect(localAddresses.getByText(`#${ROOM_ADDRESS_LONG}:localhost`)).toHaveClass( | ^ 52 | "mx_EditableItem_item", 53 | ); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/right-panel/right-panel.spec.ts:51:87
[Chrome] › room/room-header.spec.ts:54:13 › Room Header › with feature_notifications enabled › should render a very long room name without collapsing the buttons @screenshot: playwright/e2e/room/room-header.spec.ts#L1
2) [Chrome] › room/room-header.spec.ts:54:13 › Room Header › with feature_notifications enabled › should render a very long room name without collapsing the buttons @screenshot Test timeout of 30000ms exceeded.
[Chrome] › room/room-header.spec.ts:54:13 › Room Header › with feature_notifications enabled › should render a very long room name without collapsing the buttons @screenshot: playwright/pages/ElementAppPage.ts#L96
2) [Chrome] › room/room-header.spec.ts:54:13 › Room Header › with feature_notifications enabled › should render a very long room name without collapsing the buttons @screenshot Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('tree', { name: 'Rooms' }).locator('[title="Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."],[aria-label="Lorem ipsum dolor sit amet, consectetur adipisicing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur. Excepteur sint occaecat cupidatat non proident, sunt in culpa qui officia deserunt mollit anim id est laborum."]').first() at ../pages/ElementAppPage.ts:96 94 | .locator(`[title="${name}"],[aria-label="${name}"]`) 95 | .first() > 96 | .click(); | ^ 97 | } 98 | 99 | public async viewRoomById(roomId: string): Promise<void> { at ElementAppPage.viewRoomByName (/home/runner/work/element-web/element-web/playwright/pages/ElementAppPage.ts:96:14) at /home/runner/work/element-web/element-web/playwright/e2e/room/room-header.spec.ts:66:27
[Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread: playwright/e2e/room_options/marked_unread.spec.ts#L51
3) [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread Error: expect(locator).toBeVisible() Locator: getByLabel('The mark unread test room Unread messages.') Expected: visible Received: <element(s) not found> Call log: - expect.toBeVisible with timeout 5000ms - waiting for getByLabel('The mark unread test room Unread messages.') 49 | await page.getByRole("menuitem", { name: "Mark as unread" }).click(); 50 | > 51 | expect(page.getByLabel(TEST_ROOM_NAME + " Unread messages.")).toBeVisible(); | ^ 52 | }); 53 | }); 54 | at /home/runner/work/element-web/element-web/playwright/e2e/room_options/marked_unread.spec.ts:51:71
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected: playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts#L17
4) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected Test timeout of 30000ms exceeded while running "beforeEach" hook. 15 | 16 | test.describe("Message Layout Panel", () => { > 17 | test.beforeEach(async ({ app, user, util }) => { | ^ 18 | await util.createAndDisplayRoom(); 19 | await util.assertModernLayout(); 20 | await util.openAppearanceTab(); at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:17:14
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected: playwright/pages/ElementAppPage.ts#L96
4) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('tree', { name: 'Rooms' }).locator('[title="Test Room"],[aria-label="Test Room"]').first() at ../pages/ElementAppPage.ts:96 94 | .locator(`[title="${name}"],[aria-label="${name}"]`) 95 | .first() > 96 | .click(); | ^ 97 | } 98 | 99 | public async viewRoomById(roomId: string): Promise<void> { at ElementAppPage.viewRoomByName (/home/runner/work/element-web/element-web/playwright/pages/ElementAppPage.ts:96:14) at Helpers.createAndDisplayRoom (/home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/index.ts:155:24) at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:18:13
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected: playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts#L17
4) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Test timeout of 30000ms exceeded while running "beforeEach" hook. 15 | 16 | test.describe("Message Layout Panel", () => { > 17 | test.beforeEach(async ({ app, user, util }) => { | ^ 18 | await util.createAndDisplayRoom(); 19 | await util.assertModernLayout(); 20 | await util.openAppearanceTab(); at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:17:14
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected: playwright/pages/ElementAppPage.ts#L96
4) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected Retry #1 ─────────────────────────────────────────────────────────────────────────────────────── Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('tree', { name: 'Rooms' }).locator('[title="Test Room"],[aria-label="Test Room"]').first() at ../pages/ElementAppPage.ts:96 94 | .locator(`[title="${name}"],[aria-label="${name}"]`) 95 | .first() > 96 | .click(); | ^ 97 | } 98 | 99 | public async viewRoomById(roomId: string): Promise<void> { at ElementAppPage.viewRoomByName (/home/runner/work/element-web/element-web/playwright/pages/ElementAppPage.ts:96:14) at Helpers.createAndDisplayRoom (/home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/index.ts:155:24) at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:18:13
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:49:13 › Appearance user settings tab › Message Layout Panel › should disable compact layout when the modern layout is not selected: playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts#L17
5) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:49:13 › Appearance user settings tab › Message Layout Panel › should disable compact layout when the modern layout is not selected Test timeout of 30000ms exceeded while running "beforeEach" hook. 15 | 16 | test.describe("Message Layout Panel", () => { > 17 | test.beforeEach(async ({ app, user, util }) => { | ^ 18 | await util.createAndDisplayRoom(); 19 | await util.assertModernLayout(); 20 | await util.openAppearanceTab(); at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:17:14
[Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:49:13 › Appearance user settings tab › Message Layout Panel › should disable compact layout when the modern layout is not selected: playwright/pages/ElementAppPage.ts#L96
5) [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:49:13 › Appearance user settings tab › Message Layout Panel › should disable compact layout when the modern layout is not selected Error: locator.click: Test timeout of 30000ms exceeded. Call log: - waiting for getByRole('tree', { name: 'Rooms' }).locator('[title="Test Room"],[aria-label="Test Room"]').first() at ../pages/ElementAppPage.ts:96 94 | .locator(`[title="${name}"],[aria-label="${name}"]`) 95 | .first() > 96 | .click(); | ^ 97 | } 98 | 99 | public async viewRoomById(roomId: string): Promise<void> { at ElementAppPage.viewRoomByName (/home/runner/work/element-web/element-web/playwright/pages/ElementAppPage.ts:96:14) at Helpers.createAndDisplayRoom (/home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/index.ts:155:24) at /home/runner/work/element-web/element-web/playwright/e2e/settings/appearance-user-settings-tab/message-layout-panel.spec.ts:18:13
Slow Test: [Chrome] › timeline/timeline.spec.ts#L1
[Chrome] › timeline/timeline.spec.ts took 2.7m
Slow Test: [Chrome] › threads/threads.spec.ts#L1
[Chrome] › threads/threads.spec.ts took 43.9s
Slow Test: [Chrome] › user-onboarding/user-onboarding-new.spec.ts#L1
[Chrome] › user-onboarding/user-onboarding-new.spec.ts took 34.9s
Slow Test: [Chrome] › widgets/widget-pip-close.spec.ts#L1
[Chrome] › widgets/widget-pip-close.spec.ts took 21.9s
Slow Test: [Chrome] › widgets/stickers.spec.ts#L1
[Chrome] › widgets/stickers.spec.ts took 20.4s
Slow Test: [Chrome] › messages/messages.spec.ts#L1
[Chrome] › messages/messages.spec.ts took 2.2m
Slow Test: [Chrome] › pinned-messages/pinned-messages.spec.ts#L1
[Chrome] › pinned-messages/pinned-messages.spec.ts took 1.7m
Slow Test: [Chrome] › login/login.spec.ts#L1
[Chrome] › login/login.spec.ts took 1.2m
Slow Test: [Chrome] › lazy-loading/lazy-loading.spec.ts#L1
[Chrome] › lazy-loading/lazy-loading.spec.ts took 31.6s
Slow Test: [Chrome] › login/soft_logout.spec.ts#L1
[Chrome] › login/soft_logout.spec.ts took 24.5s
Slow Test: [Chrome] › crypto/event-shields.spec.ts#L1
[Chrome] › crypto/event-shields.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/device-verification.spec.ts#L1
[Chrome] › crypto/device-verification.spec.ts took 54.5s
Slow Test: [Chrome] › integration-manager/kick.spec.ts#L1
[Chrome] › integration-manager/kick.spec.ts took 37.1s
Slow Test: [Chrome] › knock/knock-into-room.spec.ts#L1
[Chrome] › knock/knock-into-room.spec.ts took 37.1s
Slow Test: [Chrome] › editing/editing.spec.ts#L1
[Chrome] › editing/editing.spec.ts took 30.6s
Slow Test: [Chrome] › composer/RTE.spec.ts#L1
[Chrome] › composer/RTE.spec.ts took 1.2m
Slow Test: [Chrome] › crypto/crypto.spec.ts#L1
[Chrome] › crypto/crypto.spec.ts took 59.5s
Slow Test: [Chrome] › crypto/backups.spec.ts#L1
[Chrome] › crypto/backups.spec.ts took 55.9s
Slow Test: [Chrome] › crypto/decryption-failure-messages.spec.ts#L1
[Chrome] › crypto/decryption-failure-messages.spec.ts took 53.1s
Slow Test: [Chrome] › accessibility/keyboard-navigation.spec.ts#L1
[Chrome] › accessibility/keyboard-navigation.spec.ts took 31.0s
Slow Test: [Chrome] › spotlight/spotlight.spec.ts#L1
[Chrome] › spotlight/spotlight.spec.ts took 2.2m
Slow Test: [Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts#L1
[Chrome] › spaces/threads-activity-centre/threadsActivityCentre.spec.ts took 1.5m
Slow Test: [Chrome] › spaces/spaces.spec.ts#L1
[Chrome] › spaces/spaces.spec.ts took 1.0m
Slow Test: [Chrome] › settings/general-room-settings-tab.spec.ts#L1
[Chrome] › settings/general-room-settings-tab.spec.ts took 25.9s
Slow Test: [Chrome] › settings/preferences-user-settings-tab.spec.ts#L1
[Chrome] › settings/preferences-user-settings-tab.spec.ts took 19.1s
Slow Test: [Chrome] › polls/polls.spec.ts#L1
[Chrome] › polls/polls.spec.ts took 55.6s
Slow Test: [Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 44.7s
Slow Test: [Chrome] › right-panel/file-panel.spec.ts#L1
[Chrome] › right-panel/file-panel.spec.ts took 38.7s
Slow Test: [Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts#L1
[Chrome] › settings/appearance-user-settings-tab/theme-choice-panel.spec.ts took 32.1s
Slow Test: [Chrome] › register/register.spec.ts#L1
[Chrome] › register/register.spec.ts took 26.5s
🎭 Playwright Run Summary
2 skipped 45 passed (6.0m)
🎭 Playwright Run Summary
1 skipped 51 passed (7.4m)
🎭 Playwright Run Summary
52 passed (7.6m)
🎭 Playwright Run Summary
1 flaky [Chrome] › audio-player/audio-player.spec.ts:199:9 › Audio player › should support downloading an audio file @no-firefox @no-webkit 51 passed (8.1m)
🎭 Playwright Run Summary
4 flaky [Chrome] › share-dialog/share-dialog.spec.ts:19:9 › Share dialog › should share a room @screenshot [Chrome] › share-dialog/share-dialog.spec.ts:32:9 › Share dialog › should share a room member @screenshot [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot [Chrome] › sliding-sync/sliding-sync.spec.ts:366:5 › Sliding Sync › should send unsubscribe_rooms for every room switch 2 skipped 50 passed (8.9m)
🎭 Playwright Run Summary
5 flaky [Chrome] › right-panel/right-panel.spec.ts:41:13 › RightPanel › in rooms › should handle long room address and long room name @screenshot [Chrome] › room/room-header.spec.ts:54:13 › Room Header › with feature_notifications enabled › should render a very long room name without collapsing the buttons @screenshot [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:42:13 › Appearance user settings tab › Message Layout Panel › should enable compact layout when the modern layout is selected [Chrome] › settings/appearance-user-settings-tab/message-layout-panel.spec.ts:49:13 › Appearance user settings tab › Message Layout Panel › should disable compact layout when the modern layout is not selected 1 skipped 46 passed (9.7m)

Artifacts

Produced during runtime
Name Size
all-blob-reports-Chrome-1 Expired
3.69 MB
all-blob-reports-Chrome-2 Expired
569 KB
all-blob-reports-Chrome-3 Expired
225 KB
all-blob-reports-Chrome-4 Expired
20.7 MB
all-blob-reports-Chrome-5 Expired
14.1 MB
all-blob-reports-Chrome-6 Expired
954 KB
html-report
35.3 MB
webapp Expired
28.5 MB