Follow

Performing Manual Testing

Introduction to Manual Testing

Automated testing leverages Access Engine to automatically find accessibility issues on web pages via Access Assistant, AMP spiders, or Access Analytics. However automated testing tools cannot test for everything, and in fact, automated testing can only test for around 25% of all possible accessibility violations. Manual testing should therefore be completed for web pages to ensure complete test coverage and limited risk. 

Manual Testing Tools

There are a couple tools in the Level Access product suite that enable manual testing: 

  1. Access Assistant
  2. AMP Manual Testing Tree

The recommended approach to manual testing is to use Access Assistant's manual testing capabilities, so testing can be done in context.

Manual Testing in Access Assistant

Access Assistant is a browser plugin, available for Chrome, Firefox and Internet Explorer 11, that allows users to automatically test web pages that are active in the browser. Manual testing features are also available in Access Assistant, and these are recommended as a manual testing tool because they allow testers to perform manual testing in a more efficient and expedient manner. Read more about manual testing in Access Assistant.

Manual Testing in AMP

Manual testing can also be done with AMP using the Manual Testing Tree feature. This feature is not as useful to for end users, because it requires users to open the page being tested in a separate browser window, and unlike Access Assistant, it does not allow for elements on the page to be auto-selected for manual creation of a violation instance. 

In AMP users can access the Manual Testing Tree as follows:

  1. Navigate to the report in which the manual testing will occur.
  2. On the Report Dashboard, select the Report Modules link in the Report Information widget.
  3. In the Report Modules view, all modules that are being tested within the report are displayed. Select the Test link in the "Manual Testing" column to access the Manual Testing Tree. 

 

The Manual Testing Tree is built around a series of setup links and two panes. The left pane contains the tree-style list of violations, while the right pane contains information regarding the violation as well as information in how to manual test for the violation as well as how to remediate any found violations.

Setup Links

The setup links allow users to navigate to the AMP Contrast Checker, add an instance of a violation, add a pattern of violations, and add a global violation.

Adding Pattern and Global Violations

To add a Pattern Violation:

  1. Navigate to the Manual Testing Tree per the instructions above.
  2. Select the Add Pattern link to add a pattern of violations.
  3. Enter a Pattern Name and select the Modules that that pattern applies to.
  4. Enter information about the Pattern Members:
    1. Violation: Select from the drop-down list which specific best practice is violated.
    2. Description: Enter a description of the violation.
    3. Note: Add any additional notes that may help remediate the violation.
    4. Instance Severity: Select from the drop-down list a number representing the severity of the violation. Although this severity is usually rated on a 1-10 scale with 10 being the most severe, it is customizable at the installation level for each organization. Therefore, your organization might use a different scale (e.g., 1-5) depending on your individual needs.
  5. Select the Make It Happen button to complete the process.

To add a Global Violation:

  1. Navigate to the Manual Testing Tree per the instructions above.
  2. Select the Add Global Pattern link to add a global violation.
  3. Enter a Global Name for the violation.
  4. Add information about the Global Members:
    1. Violation: Select from the drop-down list which specific best practice is violated.
    2. Description: Enter a description of the violation.
    3. Note: Add any additional notes that may help remediate the violation.
    4. Instance Severity: Select from the drop-down list a number representing the severity of the violation. Although this severity is usually rated on a 1-10 scale with 10 being the most severe, it is customizable at the installation level for each organization. Therefore, your organization might use a different scale (e.g., 1-5) depending on your individual needs.
  5. Select the Make It Happen button to complete the process.

Adding Screenshots

Finally, you can add screenshots to pattern and global violations through these menus to communicate where the problem occurs more easily to others viewing the AMP report. To add screenshots to your Pattern and Global Violations:

  1. Take a screenshot of the violation in question.
  2. Open the menu to create a Pattern Violation or Global Violation as described above.
  3. Activate the Upload button in the Screenshot section.
  4. Select the image file of the screenshot to upload.
  5. Select OK to upload the screenshot and share it with other viewers.

You can also add screenshots to a report instance. To do so, follow these steps:

  1. Navigate to the Dashboard tab of your report, and within the Dashboard, navigate to the Violations sub-tab.
  2. Select the Edit icon in the Actions column to edit the instance modal.
  3. In the Upload Image box, select the Browse button and choose from your own files which screenshot you want to upload.
  4. Enter concise, informative alternative text describing the screenshot for users who cannot see it in the Alt Text box.
  5. Select the Make It Happen button.
  6. You can confirm that you have uploaded the screenshot by clicking on the thumbnail that appears in the Thumbnail column to view your screenshot.

Left (Tree) Pane

The left pane contains a nested list of all the potential violations you might find on a website. The nested structure of this list follows the following order:

  • Violation Type Folder
    • Media Type
      • Best Practice
        • Violation Instance

There are several Violation Type folders:

  • Flagged  - This folder contains violations that were found with automatic testing and have already been flagged.
  • Review Guided Automatics  - This is a situation where automated analysis as identified an applicable situation. For Example, automated analysis has identified that a Table element exists on the page, but requires human testing to determine if table headers are used properly. Guided Automatics still require manual testing.
  • Manual Checklist - Displays all the violations that require manual testing to be found, this list is populated with the best practices relating to technology platform you are testing based on the standards that are active for your organization,
  • Global Checklist - Contains violations that are most likely to be Global violations
  • Report Patterns - Displays any violations marked as a pattern violation
  • Report Globals  - Displays any violations marked as a global violation

Right (Review) Pane

As you manually test for each relevant violation on the website, the right panel of the Manual Testing Pane provides guidance in how to test and remediate accessibility violations. This right pane is built around four tabs:

  • The Review Tab - Provides an overview of the selected violation which is displayed in four fields as well as a number of buttons to mark violations in a module:
    • Review - Displays information about the relevant best practice, this is helpful in understanding what and why something is an accessibility violation
    • Code Preview - For violations found automatically this field provides the source code of the violation from the page that was tested, for violations that require manual testing this field displays the entire page source code.
    • Code Snippet - This field allows you to copy the violating line of code, or make notes about the violation.
    • Note - This field allows you to make any notes you would like about the violation instance.
    • Save - Saves any changes made to the form fields of the review pane.
    • Flag (Not in AMP Toolbar for Firefox) - Flag this violation instance as being a violation that was found on the module being tested.
    • Pass - This button allows the user to mark the violation instance as not being a violation on the module being tested, this can also be used to remove a automatically found violation from the list of violations.
  • The Fix Tab - Provides a guide for how to fix the selected violation. This information contains an explanation of why this is an accessibility violation, what the accessible requirements are and examples of compliant and non-compliant code.
  • The Test Tab - Provides a guide for testing for the selected violation.
  • The Technology Platform Tab - Provides details about the technology platform and media type for the selected violation
Was this article helpful?
1 out of 1 found this helpful
Have more questions? Submit a request

Comments

Level Access Support
www.levelaccess.com | 800.889.9659
© 2005 - 2018 - Level Access All rights reserved.
Privacy | Security | Credits | License