ANDI: Automated Accessibility Tool Review

Establishing an accessibility workflow is an important step in ensuring that your site is usable by every member of the internet. With the number of tools that are available out there, each with their own assortment of strengths and weaknesses, it can be difficult to hone in on which tool is best for you and your needs. Sparkbox’s goal is to research a variety of accessibility tools and empower anyone to run accessibility audits. In this article, I will review ANDI from The United States Social Security Agency by using it on a testing site that has planned accessibility issues.

This review will rate the tool based on the following areas:

1. Ease of Setup

2. Ease of Use

3. Reputable Sources

4. Accuracy of Feedback

5. Clarity of Feedback

6. Cost

7. Strengths and Limitations

Watch a walkthrough of the tool and a short review or keep scrolling to read the full review. Jump to the conclusion if you are in a hurry.

In order to fully vet accessibility tools like this one, Sparkbox created a demo site with intentional errors. This site is not related to or connected with any live content or organization.

Ease of Setup: Exceeds Expectations (3/4)

Since ANDI is a bookmarklet, it’s easy to install and run. Just drag and drop the app into your bookmark toolbar. Once it’s in your toolbar, you can visit a website and click the bookmark to start inspecting with ANDI.

Ease of Use: Meets Expectations (2/4)

Overall, this tool takes quite a lot of getting used to and a lot of reading their documentation to understand each mode. Reading through their how-to guide and their module-specific guide will give you the information needed to understand how to get feedback from the tool and how to use their different modes, but be sure to read the whole guide before starting as it can be easy to miss accessibility issues if you are not familiar with ANDI.

To get started, navigate to the site you want to analyze and click on the bookmarklet in your toolbar, then ANDI will open a dashboard at the top of the page showing the issues and feedback it found on its current selected mode. While the dashboard is open, you can then hover over individual elements to see element-specific feedback. But, make sure you also click on the dropdown on the top left to be able to see all the accessibility issue categories.

Reputable Sources: Areas for Improvement (1/4)

While using ANDI, I couldn’t find any information or links to any standardization or WCAG. It’s hard to tell what it uses as its source. Since ANDI is created by a government agency, it’s required to meet Section 508 standards for web accessibility. Considering we don’t know the main source of its alerts and callouts, this is a bit concerning for ANDI to be used as an accessibility tool.

Accuracy of Feedback: Areas for Improvement (1/4)

While it caught 6 of 13 computer-discoverable issues, there were a few relatively common issues that it should have detected: multiple matching IDs on elements and list item elements not being surrounded by a list order element. Some of the errors that ANDI caught we didn’t expect a tool to catch included calling out tab orders and calling out redundant alt text on images such as having the word “image” in the alt text. ANDI also caught duplicate link text, which none of the tools previously tested found. It will also check the amount of headers on the page, but does not seem to check if they’re semantically correct.

To see all the test site issues that ANDI found or missed, skip to Accessibility Issues Found.

Clarity of Feedback: Meets Expectations (2/4)

This tool gives you an accordion-style list of the issues on the right side of the tool and an alert when an issue is clicked on to look up on ANDI’s site. It’s important to note that these issues are based on the current mode and will update when it changes. It also color codes each alert according to severity: red for definitely an issue, orange for likely issues, and yellow for issues that need a human to take a look per their documentation. However it doesn’t link to any guides with a standardization backing or any information on how to correct the issue, so you’ll end up having to do the research yourself. ANDI will also provide a text output called “ANDI Output” via Accessible Name and Description Computation, which can be useful to verify the user is getting the correct context from elements on the page. There’s also a “linearize page” option in the advanced settings which allows for elements repositioned by CSS to be returned to how a screen reader would pick them up, which could be useful.

Cost: Outstanding (4/4)

ANDI is free and open source. This tool does not need an account to use and doesn’t appear to collect any private user data.

Strengths and Limitations: Meets Expectations (2/4)

One of its biggest strengths is how lightweight ANDI is. It’s simply a bookmarklet, no npm install or any required commands to run and it runs on most browsers. This allows for a quick setup in cases where a quick overview is needed. Another strength is it’s an inspector which allows you to get information on the element you specifically have selected, which also leads to limitations.

Since ANDI is only an inspector, you can only see issues on the current selected mode. Another weakness is the lack of references linked to their alerts, so it’s unknown if their alerts are based on internal knowledge or a backing standard like WCAG. Another limitation is that ANDI loads on top of websites rather than loading into a developer tool widget which means it can easily get blocked by a site’s content security policy—this can be a problem if you don’t have access to a staging/developer version of the site without CSP to test.

Conclusion: 👎 Not Recommended for General Use (12/28)

This tool has some cool, convenient features like portability and the ability to detect duplicate text in images and links. But with a lack of reputable sources, it’s hard to recommend ANDI as a primary tool. However, acting as a secondary tool to quickly verify any immediate issues to then research, it does find and give alerts for most computer-discoverable issues and has some handy tools like linearize page and ANDI Output.

Accessibility Issues Found