Screen Reader Automation Testing

When it comes to Accessibility testing, screen readers paly a major role. Screen readers will help people with cognitive disabilities or blind persons to interact with the web. So, they can run a Screen reader application and listen to what the reader reads and then do the user actions on the web site. So, as developers and testers we need to make sure our application supports and works correctly with screen readers to help our differently abled customers and clients.

As I mentioned above, when we do development and testing we have to support screen readers and also we may have to use an actual screen reader and do the testing. There are different screen readers are available in the market like JAWS (for windows) and some free ones like NVDA (for windows) and some inbuilt ones like VoiceOver (for MacOS). So we can use those and do the manual testing. But when it comes to regression, it’s not feasible to do the manual tetsing for every release, so, it’s good idea to do the regression part automated.

Currently most of the browsers have an “Accessibility Tree” view in devtools. As an example using chrome browser, you can view the accessibility tree like following. If you go to the devtools by pressing F5 or right clike -> Inspect, you can see the normal DOM tree under ‘Elements’ tab. There will be a new man icon on right side as I shown in the below figure, if you click on that your element tree will be turned in to the accessibility tree view. Here you can see what the screen reader see when it interacts with your site. So, if we can access this accessibility tree from code, then we can extract out this text content and do the regression testing comparison,

Selenium Chrome DevTools Protocol (CDP) API grants access to Chrome DevTools directly from automated tests. So, we can use this API and access the accessibility tree like below,

If you are using gradle, then you can add following line to add the seleniumhq library to your repo and it will enable you to execute CDP commands from java.

implementation group: 'org.seleniumhq.selenium', name: 'selenium-devtools-v115', version: '4.11.0'

Then we need to create a session using devtools and enable the accessibility tree for the site. We can do that by using following code snippet,

DevTools devTools = driver.getDevTools();
devTools.createSession();
devTools.send(Accessibility.enable());

After that, you can do the normal selenium user traversals to visit the page you want to get the accessibility tree. One important thing to remember is the landing view of a page has the default accessibility tree, which will not have all the elements. For example, if you want to test a dropdown then you need to set focus first to the drop down by doing the user actions like clicking using selenium and get the accessibility tree. There are two ways to run the CDP commands,

1) Either you can use the driver.executeCdpCommand() method and use it as follows which will return a Map as results,

Map deviceMetrics = new HashMap() {{}};
Map map = driver.executeCdpCommand("Accessibility.getFullAXTree", deviceMetrics);

2) Or else you can use the direct methods avaiable via the Accessibility API and call like below which will return a List as results,

List<AXNode> fullAccessibilityTree = devTools.send(Accessibility.getFullAXTree(Optional.empty(), Optional.empty()));

You can use any of the above two methods and get the full accessibility tree and then manipulate this to get any parts you need like ID, Text, Role or else you can save whole output to a file (if there are no dynamically changing items like IDs) and then use it as a base results to do the comparison next time kind of a snapshot testing (this is only for regression, to make sure nothing is broken from the new release). Following is an example for searching for specific text,

Map deviceMetrics = new HashMap() {{}};
Map map = driver.executeCdpCommand("Accessibility.getFullAXTree", deviceMetrics);
ObjectMapper objectMapper = new ObjectMapper();
try {
  String json = objectMapper.writeValueAsString(map.get("nodes"));
  Assert.assertTrue(json.contains("8 results available. Use Up and Down to choose options, press Enter to select the currently focused option, press Escape to exit the menu, press Tab to select the option and exit the menu."));
} catch (Exception e) {
  e.printStackTrace();
}

Following is an example on doing snapshot testing by using saved output from a file taken from a previous instance,

List<AXNode> fullAccessibilityTree =
        devTools.send(Accessibility.getFullAXTree(Optional.empty(), Optional.empty()));
List<AccessibilityNode> accessibilityNodes = new ArrayList<>();
fullAccessibilityTree.forEach(at -> accessibilityNodes.add(new AccessibilityNode(at)));
ObjectMapper objectMapper = new ObjectMapper();
try {
  return objectMapper.writeValueAsString(accessibilityNodes);
} catch (IOException e) {
  throw new RuntimeException(e);
}

Then use the output of the above to compare using an Assert with the content saved in the file. At the end of the test it’s always better to disable the accessibility tree due to performance concerns.

devTools.send(Accessibility.disable());

Important

This is a work in progress experimental API which means these methods can be removed, added, updated in the future. Also there are some limitations for the time being, such as,

Only chromeDriver is supported
If you use Accessibility API directly then you need to use a browser which is less than the used seleniumhq version. As an example, in this code I used v115 library at the first line of code when including the library, so, I need to use a chrome browser where the version is less than 115 (<115)
Since this API is still a work in progress, we don’t have a rich documentation yet and only API spec is availble

References

https://chromedevtools.github.io/devtools-protocol/

https://chromedevtools.github.io/devtools-protocol/tot/Accessibility/