Web Accessibility Testing Techniques

Table of Contents

Abstract

Learn about a wide variety of techniques and tools (including Deque's FireEyes) for testing the accessibility of web content, using a checklist-based approach as a starting point, and going beyond checklists to consider the broader context of how people with disabilities actually use the web. The session will cover some advanced techniques for testing interactive and dynamic content.

Slideshow URL

http://pauljadam.com/weba11ytesting

Presenter

Paul J. Adam

Accessibility Evangelist at Deque Systems, Inc.

@pauljadam on Twitter

Paul@PaulJAdam.com

Slideshow

Press the Left/Right Arrow Keys or Page Up/Down to navigate through slides, the Home key takes you home to the table of contents. Adjust text size in the Info dialog. Share this presentation on Twitter & Facebook.

This slideshow has been developed with jQuery Mobile to create a universally accessible presentation that will work on any accessibility enabled device. PowerPoint & PDF formats are not universally accessible.

Laws & Checklists

Section 508, WCAG 2.0, AODA (WCAG)

WebAIM's WCAG 2.0 checklist

WCAG 2.0 Success Criteria checklist

Paul's WCAG checklist

Section 508 VPAT to get gov. clients

Automated Testing

WorldSpace & FireEyes

Large scale automated accessibility scanning & violation tracking.

FireEyes is totally free, you just need to sign up.

Benefits of an automated scanning & tracking service like WorldSpace

• Track accessibility issues page by page automatically with FireEyes then upload those issues to WorldSpace.
• FireEyes can also record a use case script so you don't have to manually click buttons and login to sites each time you run a check.
• Issues can be assigned to different users on your team so they're responsible to correct and change the status of each issue to fixed.
• Previously uploaded issues can be downloaded at a later time to inspect the problems or given to another developer to download and debug.
• Large scale sites can be spidered from WorldSpace server itself automatically without needing a human to visit each page and manually upload the issues.
• A non technical user can scan and upload issues then give them to a developer to fix and marked as completed.
• The WorldSpace dashboard can help management folks see a status of issues and progress towards accessibility over time.
• FireEyes and WS have help documentation to explain the accessibility problem to developers so they understand how to fix.
• You can set the types of accessibility violations to scan for from the server or FireEyes, e.g. WCAG 2.0 A only or A & AA, or just Section 508.
• WorldSpace is the enterprise solution, if you have hundreds or thousands of pages to scan it's just not possible to do that with a limited accessibility team, you really need an automated solution at this point.

FireEyes in Action

Issues Uploaded to WorldSpace

Demo sites for accessibility testing

Accessible University 2.0

Deque's Mars Commuter

W3C's BAD - Before and After Demo

Remember if you're a small web dev shop or a freelancer, FireEyes is totally free! Make developers run a FireEyes scan and try to fix accessibility problems on their own before escalating a simple problem to the accessibility team who needs to focus on the more complex challenges of accessibility.

User Agent & Assistive Technology Testing

Depending on the browser (UA) and screen reader (AT) combination used in testing you will get different results.

Some UAs & ATs work better together than others, and some simply fail to work together much at all.

• IE + JAWS
• Firefox + NVDA
• Safari + VoiceOver
• Mobile Safari + VoiceOver for iOS
• Firefox + TalkBack for Android

WARNING WARNING... IE + NVDA is a fail waiting to happen! Not a reliable combo for testing. NVDA is open source and they've mentioned all the hacks and workarounds they'd need to do for full IE accessibility.

Think about it, IE and JAWS are closed source, NVDA & Firefox are open source. Keep that in mind!

Of course not all blind people are totally blind, many have low vision and use on screen magnifiers in conjunction with screen readers. Popular combinations are JAWS + ZoomText.

Screen magnifiers will help you find problems in your page layout and UI placement, buttons spread out way far apart, form labels far apart from their inputs.

HTML Code Validation

Code validators only get you so far in identifying accessibility issues because they don't look at the manipulated DOM but they will do a great job of telling you if you've used ARIA and HTML elements properly according to spec.

Validator.nu (X)HTML5 Validator (Living Validator)

Browser Extensions

Firefox

• Jim Thatcher's Favelets Toolbar
• WAVE Toolbar
• Juicy Studio Accessibility Toolbar
• Firebug
• FireEyes
• Headings Map
• Open AJAX Accessibility Extension
• Web Developer Toolbar
• DOM Inspector

Chrome

• Web Accessibility Toolbar
• Chrome Accessibility Developer Tools

Internet Explorer

• Web Accessibility Toolbar (also includes Colour Contrast Analyser & aViewer)

Safari

• Nothing really, only JavaScript bookmarklets work here unless you have any suggestions?

JavaScript Bookmarklets

Work on any JavaScript supported device, including iPhones and iPads!

• Jim Thatcher's Favelets
• HTML_CodeSniffer
• Any other suggestions?

Code Inspection

Inspect Element is built into all modern browsers.

ONLY code inspection will show you the true DOM, you can't veiw the source code that's irrelevant! DOM, da DOM DOM DOM DOM... it's the only thing that matters in the modern web!

Safari Mobile Web Inspector is awesome!

Accessibility Testing Applications

aViewer

?

Contrast & Color Blindness

colour contrast analyser

color oracle - color blindness simulator

web based contrast testers

WebAIM - Color Contrast Checker

fireeyes, chrome a11y tool

Accessibility API Inspectors

OS X Accessibility Inspector

Windows API inspectors included with Microsoft SDK's

iOS Accessibility Inspector built into the iOS Simulator

Name, Role, Value, & State (Custom UI Elements)

1. Set focus to all UI elements.
2. Make sure an accessible name is spoken.
3. Make sure a role is spoken, e.g. role=button|tab|menuitem, etc.
4. Make sure the value is spoken if it's an adjustable element like a slider or checkbox.
5. Make sure the state is spoken, e.g. expanded/collapsed, pressed, selected.
6. Must work with keyboard only and with screen readers to activate the element.

Interactive & Dynamic Widgets

Automated testing tools cannot really test interactive and dynamic widgets or other content that is injected into the DOM or modified after page load. You're going to have to try every use case path with the keyboard only and with a screen reader.

Make sure to use the mouse to hover your mouse over all elements on the page to check if a tooltip or popover appears on hover. If you only TAB through the site you may not catch those issues because they usually don't appear on focus.

TAB through every UI element on the site and make sure you can activate it with the keyboard.

A screen reader may activate an element with an ENTER key giving you a FALSE sense of keyboard accessibility, BUT, if you try pressing ENTER using just the keyboard and NO screen reader you may find it does not actually work. Keyboard-only accessibility is NOT the same as screen reader accessibility.

Modal & Non-Modal Dialogs

Modal dialogs can not be tested with automated tools!

Using the keyboard only!

1. Set focus on the element that opens a modal.
2. Make sure that focus actually goes to the dialog or the first focusable element inside the dialog. THIS USUALLY FAILS!
3. Cycle through all the elements in the dialog and if it's a Modal then focus should be trapped inside the dialog until the user takes action to close it.
4. Close the dialog using the close button and/or the ESC key. ESC key is best practice and standard convention to close dialogs.
5. When a modal is closed, focus MUST be sent back to where it came from. May times focus goes back to the top of the document and the use has now lost their place!
6. Every button in that dialog must have a good accessible name. X is NOT a good accessible name for the close button. Use aria-label="Close".

The Incredible Accessible Modal Window

Accessible jQuery-ui Dialog Widget

Using Aria role=dialog to expose a dialog (pop-up div box)

jQuery UI Dialog

Menubars & Megamenus

Test once with keyboard + screen reader then once with keyboard only.

1. Set focus to first item in menubar/megamenu.
2. Make sure a screen reader is alerted to the presence of a popup menu, e.g. aria-haspopup=true, and/or the expanded/collapsed state should be indicated, e.g. aria-expanded=true.
3. The sub menus should be activated via the DOWN ARROW, SPACEBAR, or ENTER keys. The point here is that the interaction should mimic that native desktop counterpart.
4. RIGHT & LEFT ARROW keys should move you through the main menu items and even the expanded sub menus, this is a SHOULD, but very rare to see.
5. If arrow key navigation does not work then at the least TAB and SHIFT TAB should move your forward and back through the main and sub menu items.
6. If the menu appears on hover then it should also appear on focus. If the menu expands on click then it should also expand on key up. Pressing ENTER on the menu should NOT load a new page unless clicking on the main menu item also does that.

Screen reader users can easily bypass blocks of repeating content like a mega menu because they have quick navigation keys BUT a keyboard only users can only TAB or arrow key so if they are forced to TAB through 100+ sub menu items to get where they want then you have a major keyboard-only usability issue. This is why the user SHOULD be able to cycle through the main menu items ONLY without being forced to TAB through every single sub-menu item.

Adobe.com's Open Source Accessible Mega Menu on GitHub

Mega menu accessibility on adobe.com Blog Post Explaining Design

Menu bar - Mega menu - Web Experience Toolkit (WET)

Accessible University

Dropdown Menu Test: YUI Library MenuNavNode Plugin

Tabs, Accordions, Tooltips, & Expanding Content

1. Set focus to the UI element that opens the expanding content.
2. The screen reader should speak a role if it's a tab and/or indicate the expanded or collapsed state.
3. Activate the element with the keyboard and the focus should either stay where it is or be sent to the newly appeared content.
4. If focus stays where it is then the next focusable element should be the new content. Except for tabs where the next tab items may be focusable before getting into the tab panel.
5. Tooltips may never receive focus but only if their content is spoken automatically to screen readers, e.g., using an aria live region.
6. Focus must be logical and smart, it should never disappear into the top or bottom of the DOM!

Accordion - jQuery UI

Tabs - jQuery UI

Example 34 - Tab Panel - OpenAjax Alliance Accessibility Examples (PROBLEM WITH role=application in NVDA/Firefox) - problem is when you TAB away from the tab and focus on the first heading, NVDA will not read the heading, however, if you remove role=application this problem goes away. Good example of how role=application is DANGEROUS!

Forms & Validation

1. Focus on the first input, make sure the visible label is spoken automatically or another accessible name, e.g. aria-label, title, visually hidden label.
2. For all inputs that are required make sure that it's indicated to the screen reader, e.g. aria-required=true, HTML5 required, or (required) text in the label.
3. For any input that has a required input format, e.g. numbers only, 8-10 characters, ZIP code format, the format instructions should be programmatically associated with the input, i.e. aria-describedby, <label>, or some other method.
4. Leave all inputs blank and attempt to submit the form.
5. Error messages must be spoken automatically, the screen reader should NOT go silent on a bad submit!
6. Focus should be sent to the list of error messages or the first invalid input with an inline error message that's programmatically associated.
7. Focus does not have to move but only if the error messages are spoken automatically, e.g. role=alert or aria-live=assertive.
8. Validation can be client side with JS, HTML5 validation, or server side validation (PHP, JSP, .NET), all methods are fine as long as the user is alerted to all errors. You can even use old school JavaScript alert()'s.

Simple Form Validation Using WAI-ARIA aria-describedby, aria-required, aria-invalid, & jQuery .focus()

Accessible Client-side Form Validation with HTML5, WAI-ARIA, & jQuery Validation Plugin Demo (After Tweaks)

Simple Form Validation Using WAI-ARIA aria-describedby, aria-required, aria-invalid, & jQuery .focus()

Info

This presentation and slideshow template is always evolving, most ALT text should be added and correct. Please contact me if there are any issues.

Press the Left/Right Arrow Keys or Page Up/Down to navigate through slides, the Home key takes you home to the table of contents. If you're using a screen reader you may need to use the pass through key first then hit arrow keys. Page up/down works with screen readers enabled without the need for a pass through key.

JAWS Pass Key Through, INSERT+3

NVDA NVDA+f2: pass Next Key Through

Close

Share

Twitter Facebook Cancel