Fix flaky playwright tests #4002
Annotations
10 errors, 2 warnings, and 1 notice
Run Playwright tests:
playwright/e2e/room-directory/room-directory.spec.ts#L34
1) [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected)
Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
Expected string: "mx_EditableItem_item"
Received: <element(s) not found>
Call log:
- expect.toHaveClass with timeout 5000ms
- waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
32 | await localAddresses.getByRole("textbox").fill("gaming");
33 | await localAddresses.getByRole("button", { name: "Add" }).click();
> 34 | await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item");
| ^
35 |
36 | // Publish into the public rooms directory
37 | const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" });
at /home/runner/work/element-web/element-web/playwright/e2e/room-directory/room-directory.spec.ts:34:73
|
Run Playwright tests:
playwright/e2e/room-directory/room-directory.spec.ts#L34
1) [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected)
Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
Expected string: "mx_EditableItem_item"
Received: <element(s) not found>
Call log:
- expect.toHaveClass with timeout 5000ms
- waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
32 | await localAddresses.getByRole("textbox").fill("gaming");
33 | await localAddresses.getByRole("button", { name: "Add" }).click();
> 34 | await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item");
| ^
35 |
36 | // Publish into the public rooms directory
37 | const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" });
at /home/runner/work/element-web/element-web/playwright/e2e/room-directory/room-directory.spec.ts:34:73
|
Run Playwright tests:
playwright/e2e/room-directory/room-directory.spec.ts#L34
1) [Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: Timed out 5000ms waiting for expect(locator).toHaveClass(expected)
Locator: locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
Expected string: "mx_EditableItem_item"
Received: <element(s) not found>
Call log:
- expect.toHaveClass with timeout 5000ms
- waiting for locator('.mx_SettingsFieldset').filter({ hasText: 'Local Addresses' }).getByText('#gaming:localhost')
32 | await localAddresses.getByRole("textbox").fill("gaming");
33 | await localAddresses.getByRole("button", { name: "Add" }).click();
> 34 | await expect(localAddresses.getByText("#gaming:localhost")).toHaveClass("mx_EditableItem_item");
| ^
35 |
36 | // Publish into the public rooms directory
37 | const publishedAddresses = page.locator(".mx_SettingsFieldset", { hasText: "Published Addresses" });
at /home/runner/work/element-web/element-web/playwright/e2e/room-directory/room-directory.spec.ts:34:73
|
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
2) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
2) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Retry #1 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry1/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/share-dialog/share-dialog.spec.ts#L60
2) [Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
Retry #2 ───────────────────────────────────────────────────────────────────────────────────────
Error: expect(locator).toHaveScreenshot(expected)
265 pixels (ratio 0.01 of all image pixels) are different.
Expected: /home/runner/work/element-web/element-web/playwright/snapshots/share-dialog/share-dialog.spec.ts/share-dialog-event-linux.png
Received: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-actual.png
Diff: /home/runner/work/element-web/element-web/playwright/test-results/share-dialog-share-dialog-Share-dialog-should-share-an-event-Chrome-retry2/share-dialog-event-diff.png
Call log:
- expect.toHaveScreenshot(share-dialog-event.png) with timeout 5000ms
- verifying given screenshot expectation
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- 265 pixels (ratio 0.01 of all image pixels) are different.
- waiting 100ms before taking screenshot
- waiting for getByRole('dialog', { name: 'Share Room Message' })
- locator resolved to <div role="dialog" class="mx_ShareDialog" data-focus-lock-disabled="false" aria-describedby="mx_Dialog_content" aria-labelledby="mx_BaseDialog_title">…</div>
- taking element screenshot
- disabled all CSS animations
- waiting for fonts to load...
- fonts loaded
- attempting scroll into view action
- waiting for element to be stable
- captured a stable screenshot
- 265 pixels (ratio 0.01 of all image pixels) are different.
58 | const dialog = page.getByRole("dialog", { name: "Share Room Message" });
59 | await expect(dialog.getByRole("checkbox", { name: "Link to selected message" })).toBeChecked();
> 60 | await expect(dialog).toMatchScreenshot("share-dialog-event.png", {
| ^
61 | // QRCode and url changes at every run
62 | mask: [page.locator(".mx_QRCode"), page.locator(".mx_ShareDialog_top > span")],
63 | });
at /home/runner/work/element-web/element-web/playwright/e2e/share-dialog/share-dialog.spec.ts:60:30
|
Run Playwright tests:
playwright/e2e/room_options/marked_unread.spec.ts#L51
3) [Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread
Error: expect(locator).toBeVisible()
Locator: getByLabel('The mark unread test room Unread messages.')
Expected: visible
Received: <element(s) not found>
Call log:
- expect.toBeVisible with timeout 5000ms
- waiting for getByLabel('The mark unread test room Unread messages.')
49 | await page.getByRole("menuitem", { name: "Mark as unread" }).click();
50 |
> 51 | expect(page.getByLabel(TEST_ROOM_NAME + " Unread messages.")).toBeVisible();
| ^
52 | });
53 | });
54 |
at /home/runner/work/element-web/element-web/playwright/e2e/room_options/marked_unread.spec.ts:51:71
|
Run Playwright tests:
playwright/e2e/settings/general-room-settings-tab.spec.ts#L47
4) [Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow @no-webkit
Error: Timed out 5000ms waiting for expect(locator).toHaveValue(expected)
Locator: locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
Expected string: "#abcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksdabcasdhjasjhdaj1jh1asdhasjdhajsdhjavhjksd:localhost"
Received string: ""
Call log:
- expect.toHaveValue with timeout 5000ms
- waiting for locator('.mx_Dialog').filter({ has: locator('.mx_RoomSettingsDialog') }).locator('#canonicalAlias')
9 × locator resolved to <select type="text" id="canonicalAlias" label="Main address" placeholder="Main address">…</select>
- unexpected value ""
45 |
46 | // 2. wait for the new setting to apply ...
> 47 | await expect(settings.locator("#canonicalAlias")).toHaveValue(`#${longString}:localhost`);
| ^
48 |
49 | // 3. Check if the dialog overflows
50 | const dialogBoundingBox = await page.locator(".mx_Dialog").boundingBox();
at /home/runner/work/element-web/element-web/playwright/e2e/settings/general-room-settings-tab.spec.ts:47:59
|
Run Playwright tests:
playwright/e2e/sliding-sync/sliding-sync.spec.ts#L86
5) [Chrome] › sliding-sync/sliding-sync.spec.ts:191:5 › Sliding Sync › should not show unread indicators
Test timeout of 30000ms exceeded while running "beforeEach" hook.
84 |
85 | // Load the user fixture for all tests
> 86 | test.beforeEach(({ user }) => {});
| ^
87 |
88 | test("should render the Rooms list in reverse chronological order by default and allowing sorting A-Z", async ({
89 | page,
at /home/runner/work/element-web/element-web/playwright/e2e/sliding-sync/sliding-sync.spec.ts:86:10
|
Run Playwright tests:
playwright/element-web-test.ts#L191
5) [Chrome] › sliding-sync/sliding-sync.spec.ts:191:5 › Sliding Sync › should not show unread indicators
Error: page.waitForSelector: Test timeout of 30000ms exceeded.
Call log:
- waiting for locator('.mx_MatrixChat') to be visible
at ../element-web-test.ts:191
189 | user: async ({ pageWithCredentials: page, credentials }, use) => {
190 | await page.goto("/");
> 191 | await page.waitForSelector(".mx_MatrixChat", { timeout: 30000 });
| ^
192 | await use(credentials);
193 | },
194 |
at Object.user (/home/runner/work/element-web/element-web/playwright/element-web-test.ts:191:20)
|
Run Playwright tests:
[Chrome] › room/room-header.spec.ts#L1
[Chrome] › room/room-header.spec.ts took 34.6s
|
Run Playwright tests:
[Chrome] › settings/account-user-settings-tab.spec.ts#L1
[Chrome] › settings/account-user-settings-tab.spec.ts took 17.4s
|
Run Playwright tests
2 failed
[Chrome] › room-directory/room-directory.spec.ts:18:9 › Room Directory › should allow admin to add alias & publish room to directory @no-webkit
[Chrome] › share-dialog/share-dialog.spec.ts:49:9 › Share dialog › should share an event @screenshot
3 flaky
[Chrome] › room_options/marked_unread.spec.ts:22:9 › Mark as Unread › should mark a room as unread
[Chrome] › settings/general-room-settings-tab.spec.ts:39:9 › General room settings tab › long address should not cause dialog to overflow @no-webkit
[Chrome] › sliding-sync/sliding-sync.spec.ts:191:5 › Sliding Sync › should not show unread indicators
1 skipped
44 passed (5.8m)
|
Loading