Overview of Changes
With Access Engine all of our automated tests have been rewritten from the ground up, and in the process we have made a number of improvements to the tests themselves to:
- Reduce false positives. Some tests were returning violations that were incorrect. These tests have been removed to improve the accuracy of results.
- Streamline testing. Some tests were found to be unnecessary or found violations which were too similar to other tests. These tests have been condensed to reduce the redundancy of results.
- Enhance granularity. For some Best Practices, the tests provided were not detailed enough to accurately describe the violation. The existing tests were rewritten and additional tests were added to improve insight into the violations.
- Provide greater detail. Previous tests did not provide adequate insight into the details of the violation without looking into the code. Test descriptions have been rewritten so that programmers and non-programmers alike can review and understand their testing results.
- Different size screens yield different results. To give more power to Access Assistant users, Access Engine will test based on the specific screen size of the user’s device. This means there may be different results with different screen sizes; please be aware!
- If it’s not visible, it’s not tested. Access Engine has been designed to test only what is visible on the page. This has been done to ensure the accuracy of the automated tests. If a user requires testing of content that must be exposed via a user interaction or responsive breakpoint trigger, they can test this additional content by expanding the content and retesting or by testing the smaller screen size view of a page separately.
- No guided automatics or advisory tests. Guided automatics and advisory tests are now only included in a new "Needs Review" section of the Access Assistant toolbar, where they can be reviewed and passed or failed. Future releases of AMP will include this functionality, as well.
There are a couple areas of testing where we have completed overhauled our testing algorithms to address more complex automated testing scenarios:
- Accessible Name Calculation: Developers must be able to anticipate which tag or attribute assistive technologies identify as the accessible name for a control. We have taken out some of that mystery and enhanced our algorithms to calculate the accessible name for various components and use that in our tests to determine if an accessible name is present and whether the name is useful or not.
- Roles: The accessible role is now taken into account for many of our tests, insuring the semantic meaning of elements is considered when testing. For example, when an ARIA role is applied to an element it’s semantic meaning changes and thus the tests that should or should not be run on the element and it’s descendants needs to change.
Many tests that were previously guided automatic tests -- i.e., they required human evaluation to determine whether or not they were valid violations -- are now fully automated. The following Best Practices were previously guided automatic tests but now have automated coverage:
- Ensure pages do not automatically refresh.
- Ensure implicit row header cells use “th” elements. (some tests are still guided automatic)
Conversely, some Best Practices were previously fully automated, but they were resulting in false positives. These Best Practices have been converted to guided automatic tests to reduce the number of false positives:
- Ensure event handlers that trigger navigation or form submission are avoided.
- Ensure custom controls provide proper textual roles and descriptions.
- Avoid unnecessary use of heading elements.
- Ensure hr elements utilize relative sizing.
- Ensure lists for indentation are not used.
- Ensure the same link text for links with different targets is avoided.
- Ensure rich edit entry fields are directly accessible or provide an alternative entry method.
The original release of our DCQL Engine was designed prior to the release of WCAG 2.0 in 2008! This means we were supporting many WCAG 1.0 tests which were rendered obsolete by the release of WCAG 2.0. We know these tests aren’t saving you any time – so we have taken this opportunity to remove the ones that are not applicable to modern standards.
Increased Test Coverage
Test coverage has been improved for a number of Best Practices through the creation of additional tests that address standards for the latest versions of HTML, SVG and the ARIA specification. The following are areas where we have improved coverage with Access Engine:
- Ensure ARIA roles, states, and properties are valid: The amount of ARIA validation has been tripled to root out difficult to find but all too common ARIA implementation mistakes that affect the ability of assistive technology to work with interactive controls.
- Tables: We are able to spot table markup and categorize and test them according to the developer’s intention with better detection of missing table headers and identification of incorrect header cell association.
- Avoid the sole use of device-dependent event handlers: We have and increased and continue to increase our capabilities to identify events associated with elements to ensure that interactions are device independent.
- Ensure embedded (media) objects are directly accessible: We now have more tests to validate the track elements of media elements are valid.
- Language detection: While we always detected missing language attributes of a document we now verify that the first two characters of the language code of all document level and inline lang attributes are valid according to the specification.
Previous versions of AMP and related toolbars tested all of the rendered source code, even those elements that were not visible on the page. Access Engine only tests displayed content, so the tester can be sure they are testing the page as it looks when they tested it. Change the display—change the results. This means that you will need to test the different states of pages. For example, for a responsive site a test should be performed on each responsive breakpoint,which will give you a more accurate view of the issues related specifically to that view. For pages with content that can be shown or hidden a button is available in the Access Assistant toolbar to re-test the visible page and include the newly detected automatic issues into the current report.
Detailed documentation on the changes made to each test is attached below.
Make sure to check back here as we are looking to add new tests in upcoming releases!