Skip to content

Commit

Permalink
feat(axe-4.7): update to use axe-core 4.7.2 (#1061)
Browse files Browse the repository at this point in the history
#### Details

This PR upgrades `axe-core` to 4.7.2 according to the instructions in
CONTRIBUTING.md.

Additionally, it adds a script that uses `@puppeteer/browsers` to
install a specific version of Chrome and move it to the default windows
install location, which is expected by `ChromeDriver`.

It also adds `@axe-core/cli` to dependabot,yml since we want to include
it in axe-core upgrades and not before.

##### Motivation

feature work 🚀 

##### Context

#### Pull request checklist
<!-- If a checklist item is not applicable to this change, write "n/a"
in the checkbox -->
- [x] PR title respects [Conventional
Commits](https://www.conventionalcommits.org) (starts with `fix:`,
`feat:`, etc, and is suitable for user-facing release notes)
- [x] PR contains no breaking changes, **OR** description of both PR
**and final merge commit** starts with `BREAKING CHANGE:`
- [n/a] (if applicable) Addresses issue: #0000
- [n/a] Added relevant unit tests for your changes
- [x] Ran `yarn precheckin`
- [x] Verified code coverage for the changes made
  • Loading branch information
madalynrose authored Aug 4, 2023
1 parent 6bf708c commit ab8306b
Show file tree
Hide file tree
Showing 18 changed files with 108,801 additions and 33 deletions.
3 changes: 3 additions & 0 deletions .github/dependabot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,9 @@ updates:
- dependency-name: "@axe-core/puppeteer"
versions:
- ">= 0"
- dependency-name: "@axe-core/cli"
versions:
- ">= 0"
# Major version of @types/node is pinned to match the version of node we
# use for builds (ideally, latest LTS)
- dependency-name: "@types/node"
Expand Down
24 changes: 23 additions & 1 deletion CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ To update the package and test cases to account for a new axe-core version:
1. In `package.json`, update the version numbers of the following components:

- `devDependencies` entries for `@axe-core/cli` and `@axe-core/puppeteer`
- `resolutions` entries for `axe-core` and `@axe-core/cli/chromedriver`
- `resolutions` entries for `axe-core`
- **NOT** the `dependencies` entry for `@axe-core`!

1. Build the repo with:
Expand All @@ -69,6 +69,28 @@ To update the package and test cases to account for a new axe-core version:
yarn generate-test-resources
```
- If this fails with an error message saying it cannot find the Chrome binary, run:
```
yarn generate-chrome-resources
```
- If this fails with an error message saying "This version of ChromeDriver only supports Chrome version X", then find a version number that starts with X from [this list](https://googlechromelabs.github.io/chrome-for-testing/known-good-versions-with-downloads.json) and then run:
```
yarn generate-chrome-resources [version-number]
yarn generate-test-resources
```
for example:
```
yarn generate-chrome-resources 113.0.5672.0
yarn generate-test-resources
```
This script installs the Chrome binary for the specified version of Chrome in the location that ChromeDriver expects it to be (`C:\Users\[username]\AppData\Local\Google\Chrome\Application` on Windows). This will allow you to successfully run the script to generate test resources.
1. Manually compare the diff of `/src/test-resources/basic-axe-vPREVIOUS.sarif` and `/src/test-resources/basic-axe-vNEW.sarif`; the only differences should be the version numbers.
1. Manually compare the diff of `/src/test-resources/w3citylights-axe-vPREVIOUS.sarif` and `/src/test-resources/w3citylights-axe-vNEW.sarif`; in addition to version number differences, you should see some differences based on new/removed rules between the axe versions.
1. Add test cases involving the new files to the integration tests in `src/index.test.ts` and `src/cli.test.ts`
Expand Down
11 changes: 6 additions & 5 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,14 @@
},
"packageManager": "[email protected]",
"dependencies": {
"@puppeteer/browsers": "^1.5.0",
"@types/sarif": ">=2.1.1 <=2.1.4",
"axe-core": "^3.2.2 || ^4.0.0",
"yargs": "^17.0.0"
},
"devDependencies": {
"@axe-core/cli": "^4.6.0",
"@axe-core/puppeteer": "^4.6.0",
"@axe-core/cli": "^4.7.0",
"@axe-core/puppeteer": "^4.7.0",
"@types/jest": "^27.4.0",
"@types/lodash": "^4.14.136",
"@types/node": "^18.16.2",
Expand All @@ -44,9 +45,8 @@
"typescript": "^4.0.2"
},
"resolutions": {
"@axe-core/cli/chromedriver@^109.0.0": "^111.0.0",
"ansi-regex@^4.1.0": "^5.0.1",
"axe-core": "4.6.3"
"axe-core": "4.7.2"
},
"scripts": {
"prebuild": "yarn clean",
Expand All @@ -63,7 +63,8 @@
"semantic-release": "semantic-release",
"generate-test-resources": "yarn generate-axe-core-resources && yarn generate-axe-cli-resources",
"generate-axe-cli-resources": "node dist/test-resources/generator/generate-axe-cli-resources.js",
"generate-axe-core-resources": "node dist/test-resources/generator/generate-axe-core-resources.js"
"generate-axe-core-resources": "node dist/test-resources/generator/generate-axe-core-resources.js",
"generate-chrome-resources": "node dist/test-resources/generator/install-chrome-for-webdriver.js"
},
"repository": {
"type": "git",
Expand Down
8 changes: 4 additions & 4 deletions src/__snapshots__/index.test.ts.snap
Original file line number Diff line number Diff line change
Expand Up @@ -1388,15 +1388,15 @@ Object {
],
"tool": Object {
"driver": Object {
"downloadUri": "https://www.npmjs.com/package/axe-core/v/4.6.3",
"fullName": "axe for Web v4.6.3",
"downloadUri": "https://www.npmjs.com/package/axe-core/v/4.7.2",
"fullName": "axe for Web v4.7.2",
"informationUri": "https://www.deque.com/axe/axe-for-web/",
"name": "axe-core",
"properties": Object {
"microsoft/qualityDomain": "Accessibility",
},
"rules": Array [],
"semanticVersion": "4.6.3",
"semanticVersion": "4.7.2",
"shortDescription": Object {
"text": "An open source accessibility rules library for automated testing.",
},
Expand All @@ -1407,7 +1407,7 @@ Object {
"name": "WCAG",
},
],
"version": "4.6.3",
"version": "4.7.2",
},
},
},
Expand Down
6 changes: 4 additions & 2 deletions src/cli.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -73,6 +73,8 @@ describe('axe-sarif-converter CLI', () => {
${'w3citylights-axe-v4.4.1.axe-cli-v4.4.2.json'}
${'basic-axe-v4.6.3.axe-cli-v4.6.0.json'}
${'w3citylights-axe-v4.6.3.axe-cli-v4.6.0.json'}
${'basic-axe-v4.7.2.axe-cli-v4.7.3.json'}
${'w3citylights-axe-v4.7.2.axe-cli-v4.7.3.json'}
`(
'supports conversion from axe-cli output $inputFile',
async ({ inputFile }) => {
Expand Down Expand Up @@ -194,11 +196,11 @@ describe('axe-sarif-converter CLI', () => {
const testResultsDir = path.join(__dirname, '..', 'test-results');
const basicAxeV2File = path.join(
testResourcesDir,
'basic-axe-v4.4.1.reporter-v2.json',
'basic-axe-v4.7.2.reporter-v2.json',
);
const basicSarifFile = path.join(
testResourcesDir,
'basic-axe-v4.4.1.sarif',
'basic-axe-v4.7.2.sarif',
);

const mkdir = promisify(fs.mkdir);
Expand Down
6 changes: 5 additions & 1 deletion src/index.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -111,6 +111,10 @@ describe('public convertAxeToSarif API', () => {
${'basic-axe-v4.6.3.reporter-v2.json'} | ${'basic-axe-v4.6.3.sarif'}
${'w3citylights-axe-v4.6.3.reporter-v1.json'} | ${'w3citylights-axe-v4.6.3.sarif'}
${'w3citylights-axe-v4.6.3.reporter-v2.json'} | ${'w3citylights-axe-v4.6.3.sarif'}
${'basic-axe-v4.7.2.reporter-v1.json'} | ${'basic-axe-v4.7.2.sarif'}
${'basic-axe-v4.7.2.reporter-v2.json'} | ${'basic-axe-v4.7.2.sarif'}
${'w3citylights-axe-v4.7.2.reporter-v1.json'} | ${'w3citylights-axe-v4.7.2.sarif'}
${'w3citylights-axe-v4.7.2.reporter-v2.json'} | ${'w3citylights-axe-v4.7.2.sarif'}
`(
'converts pinned v1/v2 input $inputFile to pinned output $outputFile',
({ inputFile, outputFile }) => {
Expand Down Expand Up @@ -157,7 +161,7 @@ describe('public sarifReporter API', () => {
// it isn't very meaningful to test cases that involve old axe versions here.
it.each`
inputFile | outputFile
${'basic-axe-v4.6.3.reporter-raw.json'} | ${'basic-axe-v4.6.3.sarif'}
${'basic-axe-v4.7.2.reporter-raw.json'} | ${'basic-axe-v4.7.2.sarif'}
`(
'converts pinned raw input $inputFile to pinned output $outputFile',
async ({ inputFile, outputFile }) => {
Expand Down
70 changes: 70 additions & 0 deletions src/test-resources/basic-axe-v4.7.2.axe-cli-v4.7.3.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,70 @@
[
{
"testEngine": {
"name": "axe-core",
"version": "4.7.2"
},
"testRunner": {
"name": "axe"
},
"testEnvironment": {
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/113.0.5672.35 Safari/537.36",
"windowWidth": 800,
"windowHeight": 600,
"orientationAngle": 0,
"orientationType": "landscape-primary"
},
"timestamp": "2023-08-03T17:39:25.967Z",
"url": "file:///C:/repos/axe-sarif-converter/src/test-resources/basic.html",
"toolOptions": {
"runOnly": {
"type": "rule",
"values": [
"document-title"
]
},
"reporter": "v1"
},
"inapplicable": [],
"passes": [],
"incomplete": [],
"violations": [
{
"id": "document-title",
"impact": "serious",
"tags": [
"cat.text-alternatives",
"wcag2a",
"wcag242",
"ACT",
"TTv5",
"TT12.a"
],
"description": "Ensures each HTML document contains a non-empty <title> element",
"help": "Documents must have <title> element to aid in navigation",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.7/document-title?application=webdriverjs",
"nodes": [
{
"any": [
{
"id": "doc-has-title",
"data": null,
"relatedNodes": [],
"impact": "serious",
"message": "Document does not have a non-empty <title> element"
}
],
"all": [],
"none": [],
"impact": "serious",
"html": "<html><head></head><body>\n</body></html>",
"target": [
"html"
],
"failureSummary": "Fix any of the following:\n Document does not have a non-empty <title> element"
}
]
}
]
}
]
54 changes: 54 additions & 0 deletions src/test-resources/basic-axe-v4.7.2.reporter-raw.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@
[
{
"id": "document-title",
"result": "failed",
"pageLevel": false,
"impact": "serious",
"tags": [
"cat.text-alternatives",
"wcag2a",
"wcag242",
"ACT",
"TTv5",
"TT12.a"
],
"description": "Ensures each HTML document contains a non-empty <title> element",
"help": "Documents must have <title> element to aid in navigation",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.7/document-title?application=axe-puppeteer",
"inapplicable": [],
"passes": [],
"incomplete": [],
"violations": [
{
"node": {
"selector": [
"html"
],
"source": "<html><head></head><body>\n</body></html>",
"xpath": [
"/html"
],
"ancestry": [
"html"
],
"nodeIndexes": [
0
]
},
"any": [
{
"id": "doc-has-title",
"data": null,
"relatedNodes": [],
"impact": "serious",
"message": "Document does not have a non-empty <title> element"
}
],
"all": [],
"none": [],
"impact": "serious",
"result": "failed"
}
]
}
]
72 changes: 72 additions & 0 deletions src/test-resources/basic-axe-v4.7.2.reporter-v1.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,72 @@
{
"testEngine": {
"name": "axe-core",
"version": "4.7.2"
},
"testRunner": {
"name": "axe"
},
"testEnvironment": {
"userAgent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) HeadlessChrome/101.0.4950.0 Safari/537.36",
"windowWidth": 800,
"windowHeight": 600,
"orientationAngle": 0,
"orientationType": "portrait-primary"
},
"timestamp": "2000-01-02T03:04:05.006Z",
"url": "http://localhost/",
"toolOptions": {
"xpath": true,
"runOnly": {
"type": "rule",
"values": [
"document-title"
]
},
"reporter": "v1"
},
"inapplicable": [],
"passes": [],
"incomplete": [],
"violations": [
{
"id": "document-title",
"impact": "serious",
"tags": [
"cat.text-alternatives",
"wcag2a",
"wcag242",
"ACT",
"TTv5",
"TT12.a"
],
"description": "Ensures each HTML document contains a non-empty <title> element",
"help": "Documents must have <title> element to aid in navigation",
"helpUrl": "https://dequeuniversity.com/rules/axe/4.7/document-title?application=axe-puppeteer",
"nodes": [
{
"any": [
{
"id": "doc-has-title",
"data": null,
"relatedNodes": [],
"impact": "serious",
"message": "Document does not have a non-empty <title> element"
}
],
"all": [],
"none": [],
"impact": "serious",
"html": "<html><head></head><body>\n</body></html>",
"target": [
"html"
],
"xpath": [
"/html"
],
"failureSummary": "Fix any of the following:\n Document does not have a non-empty <title> element"
}
]
}
]
}
Loading

0 comments on commit ab8306b

Please sign in to comment.