Trusted Tester

I saw a job opening mentioning this certification done by the US Dept. of Homeland Security. Not thrilled by registering with the DHS for anything, but on the other hand I’ve rarely seen a certification I’d be happy to show off, so…

I will be going through the documentation and making notes here.

More info about the program is at Section 508 Trusted Tester Conformance Test Process Version 5 | Homeland Security.

Let’s get meta real quick, and go through About This Document | Trusted Tester

Who Should Use This Document

This document has been designed for and is intended for use by Trusted Testers.

A Trusted Tester is a person certified to provide accurate and repeatable Revised 508 conformance test results for web content. A Trusted Tester follows the Revised Section 508 Conformance Test Process for Web, uses approved testing tools, and evaluates web applications for conformance with Revised 508 standards. Trusted Testers are those who have passed the Trusted Tester Certification Exam.

For more information on the Revised 508 Trusted Tester Training Course and Exam, contact the Department of Homeland Security Office of Accessible Systems & Technology (OAST) Accessibility Helpdesk at accessibility@dhs.gov.

A “Trusted Tester” is me! Seriously, I could pick this up as a hobby, but I’ve also been fortunate enough to have built and managed platforms on behalf of city governments in the last couple of years that take this very seriously, and I was able to actively apply my knowledge in this regard. :slight_smile:

Harmonized Baseline Alignment

This test process incorporates all tests in the “Harmonized Processes for Revised 508 Testing: Baseline Tests for Web Accessibility” version 3.0. The baseline tests established the minimum steps required to determine compliance with Revised 508 standards and WCAG 2.0 Level A and AA. Test instructions that are specific to Trusted Tester only are identified with TT-specific or “[no baseline].” The outcomes of these tests will be reflected only in Revised 508 test results.

Baseline test results will be reported separately and are not affected by Trusted Tester-specific tests.

Okay, this is a head-scratcher. I can’t find version 3.0 of “Harmonized Processes for Revised 508 Testing: Baseline Tests for Web Accessibility”, though at DHS Section 508 Compliance Test Processes | Homeland Security I may download 1.1 (which has “3” in the filename?!) and 2.0.2…

I’ll presume I’ll figure it out, eventually, and have skimmed those other docs as well; they are ~100 pages long, and include worksheets with checklists, surprisingly similar to a TTRPG book!

Also… this subheading had “alignment” in the name! My theory is that Gygax or wargamers set the most serious tone they could imagine, which also happens to the same mentality and result from a government agency doing the same.

:thinking:

How this Document is Structured

  • Introductory Content:

    • About this document – describes the purpose, audience, and scope of this document Section 1

    • Test Environment – describes the test environments supported by this test process including Operating Systems, browsers, and testing tools

    • Conformance Reporting Requirements – provides guidance

  • Section 508 Conformance Tests – details the test processes. Each test process includes step-by-step instructions on how and what to test, as well as instructions on how to determine test results for each Test Condition.

  • Appendices – Provide cross-references tables to indicate the relationship of the Tests to Revised 508 standards and Baseline tests; definitions; a document change log; and quick reference summaries of the test process.

This is so exciting!

  1. I’m gonna get a thing at the end of this that I will want to actually have: a badge saying accessibility matters so much, I got a government badge that says so!
  2. An entire training document that reads like a classic RPG book…

:star_struck:

Web Content Tests Only

This test process covers web content only. Trusted Tester versions 4.x and older were developed for the original Section 508 standards, which had separate requirements for web and software. With the Revised 508 standards applying WCAG 2.0 Level A and AA to web, software and other electronic content, combining software and web in one test process was the original plan.

However, because the Revised 508 Standards has other requirements for software in addition to WCAG 2.0, and software testing tools were not yet available, the software test process will be separate. While it is not as common due to HTML5 capabilities, software elements are still found with web content in web applications. To test the software elements, use the software test process.

Similarly, any other operating systems, browser or platforms such as mobile tablets, must be evaluated using other testing procedures.

Good guidance on how to proceed using this document in ways that are adaptable to other technologies, and how to proceed with various living protocols.

This references “Trusted Testers versions 4.0 and older” as if humans were Terminators, and makes me wonder if I will have to re-certify later…

Testing Order

The numbering of the tests within the test process do not necessarily indicate the order that tests must be performed. Each tester and each application may determine the optimal testing order for coverage and productivity.

It is recommended, however, that the first test performed is Test 1 Conforming Alternate Version. Identifying conforming alternate versions of content helps define the scope of testing and avoids unnecessary testing. Non-conforming content that has a conforming alternate version is excluded from testing.

Test 2 Auto-Playing and Auto-Updating Content and Test 3 Flashing are next in the test process, followed by Test 4 Keyboard Access and Focus. These test WCAG success criteria that are covered in Conformance Requirement 5 Non-Interference. Failure to meet these success criteria could interfere with any use of the page and may indicate critical accessibility issues.

The test process was designed to streamline the sequence for testers; however testers may choose to do the tests (after Test 1) in their preferred order. Each Test Condition (after Test 1) is independent of the tests that precede, but some will reference tests that follow.

Huh, that’s a lot of instruction for how the tests were designed… now I’m curious how it will play out. . . as a choose your own adventure book!

:dragon:

Very cool link, BTW, I visit it often.

Issues Not Covered in This Test Process

Problems may be found during testing that may affect accessibility but are simply coding errors and often affect general usability for all users. An example might include a link that leads to the wrong target website. Testers may notify a developer of these issues as a comment on a report, but they do not typically result in a compliance failure as they are beyond the scope of the Revised Section 508 Standards.

Testers may notify a developer of these issues as a comment on a report

:rofl:

The Rationale for Each Test

Previous versions of the Trusted Tester process document provided a rationale for each test based on interpretation of the Section 508 standards. With the Section 508 standards refresh and adoption of the WCAG 2.0 Success Criteria, this version of the test process relies principally on the rationale provided in Understanding WCAG 2.0: A guide to understanding and implementing Web Content Accessibility Guidelines 2.0. The test process also relies on accompanying Trusted Tester training to provide additional description and guidance for understanding the logic that drives each test. Each step included in this test process document includes only the information necessary to execute the test. However, the Applicable Standards section of each test references the applicable WCAG 2.0 or Section 508 standard along with a link to the applicable article from the Understanding WCAG 2.0 document.

Oh wow!

First of all, go read that document: Understanding WCAG 2.0

Okay, I just realized that most folks won’t be as into that as I am, so I’ll move onto my other point: this document explicitly acknowledges that document has good enough examples in it to provide an evergreen reference guide, therefore not requiring to include rationale for each testing section.

I know that seems very normal, in the testing industry sense: there are distinct products marketed, and will reference each other as needed.

But this is interesting for two reasons:

  1. This is a standards document being referenced
  2. W3C docs are notoriously poorly written for normies!

But not that one… as much. I mean, it’s gonna be dry throughout, but you don’t need to read most sections more than once to understand what is being said. Here’s an excerpt (from Understanding Success Criterion 1.1.1 | Understanding WCAG 2.0):

Note on CAPTCHA

CAPTCHAs are a controversial topic in the accessibility community. As is described in the paper Inaccessibility of CAPTCHA, CAPTCHAs intrinsically push the edges of human abilities in an attempt to defeat automated processes. Every type of CAPTCHA will be unsolvable by users with certain disabilities. However, they are widely used, and the Web Content Accessibility Guidelines Working Group believes that if CAPTCHAs were forbidden outright, Web sites would choose not to conform to WCAG rather than abandon CAPTCHA. This would create barriers for a great many more users with disabilities. For this reason the Working Group has chosen to structure the requirement about CAPTCHA in a way that meets the needs of most people with disabilities, yet is also considered adoptable by sites. Requiring two different forms of CAPTCHA on a given site ensures that most people with disabilities will find a form they can use.

Because some users with disabilities will still not be able to access sites that meet the minimum requirements, the Working Group provides recommendations for additional steps. Organizations motivated to conform to WCAG should be aware of the importance of this topic and should go as far beyond the minimum requirements of the guidelines as possible.


Okay, that’s the meta on this doc. Next up is “Testing Environment”, where we’ll get into some warez. :sunglasses:

Test Environment

At the initial release of this document, only the operating systems and browsers specified below were validated with the test process and tools to ensure that results were consistent and accurate. The list of supported operating systems and browsers is expected to grow. Please refer to the DHS Section 508 Compliance Testing Tools website at DHS Section 508 Compliance Testing Tools | Homeland Security for the most up to date test environment information and the Trusted Tester Test Tool Installation Guide.

The URL on the actual page is broken and redirects to the DHS front page. I was going to patch it, but it turns out someone already did, 2 years ago…

I’ll expand on DHS Section 508 Compliance Testing Tools | Homeland Security very soon. It will be very interesting to compare their relatively short list of tools with what is overall available (I don’t have great hope there are abundant tools for this :cry:).

Testing Tools

The tools used in the Test Process (and Baseline tests) have been chosen based on several factors including ease of use, ease of teaching, and accuracy of results. They are also free to install and use. The tools assist the tester with verification of accessibility properties. This test process is essentially a code inspection for accessibility properties, but the tools reduce the need for a tester to view source code or have in-depth knowledge of programming languages.

They, that’s great! Even our accessibility tools need be accessible! :slight_smile: Let’s see what they’ve got.

ANDI

ANDI (Accessible Name & Description Inspector) is a free open-source bookmarklet, meaning that the tool does not require installation as a plugin and can be added to multiple browsers as a bookmark. ANDI is designed to help test web content for accessibility. Developed by the US Social Security Administration, ANDI is available at ANDI - Accessibility Testing Tool - Install.

ANDI issues may be reported to the ANDI GitHub page: Issues · SSAgov/ANDI · GitHub.

Whaaaaat? A bookmarklet?! I used a bookmarklet just the other day to solve a funny problem:

Unfortunately the bookmarklet collections I’ve found were disappointing, but now I’ve found a second useful bookmarklet… I think maybe I need to start a collection of bookmarklets!

Color Contrast Analyzer

The Color Contrast Analyzer (CCA) is a free open-source tool that displays the contrast ratio for two selected colors. Developed by Steve Faulkner and the Paciello Group, CCA is available at the following links:

Well… first of all, I do mainline either of those operating systems (which we’ll see if not an issue), so it’s kinda disappointing that recommended software is not available to me.

However, there are caveats, even with those OSes! Those repos are archived. Poking around I get to a page (https://www.tpgi.com/color-contrast-checker/) that explains:

Previous versions

The current version of the Colour Contrast Analyser (CCA) has been rebuilt from the ground up using Electron. For the previous, non-Electron versions (“CCA Classic”), see the CCA-Win and CCA-OSX repositories.

This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.

This means there is an oppurtunity to create a better list of available tools. I always check the color contrast ratio on sites I develop.

Operating Systems

The following operating systems were validated:

  • Windows 10 (desktop mode)
  • macOS (with Safari only)

Although Windows 10 and macOS are the only Operating Systems listed, no foreseeable issues due to using another operating system have been identified. The operating system has little to no impact on web testing results; rather, results are more dependent on the browser.

See, OS doesn’t matter as much. It can, but it hardly comes up now there are only 1.5 browsers in place.

Browsers

The following browsers were validated:

On Windows 10:

  • Google Chrome
  • Mozilla Firefox
  • Microsoft Edge
  • Microsoft Internet Explorer (IE) 11

On macOS:

  • Safari

Use of newer versions of these browsers is acceptable unless otherwise specified on the DHS Section 508 Compliance Testing Tools website at DHS Section 508 Compliance Testing Tools | Homeland Security. As browsers are frequently updated, it may be possible that an update creates critical issues for test procedures or results. Known issues and modifications will be published on the website as quickly as possible.

Not sure what that last line refers to: which website are they talking about?

Anyhow, that is the testing environment, and some tools to look at. Next is “Conformance Reporting Requirements”… great, on top of all these TPS Reports!