Measuring the accessibility of our frameworks

What is a framework?

The term “framework” has a specific meaning for engineers. It is a tailored collection of building blocks we use when creating products. Frameworks provide a common language so engineers can work better together. There are many varieties, each with unique benefits. Importantly, they determine key aspects of the project:

1. How quickly can I build?
2. What programming language was used?
3. How do I improve upon what I’ve built?
4. What platforms and phones can I use this to build for?

Facebook uses a variety of frameworks — some of the most popular are React Native, Litho, and ComponentKit. Many frameworks seek to simplify building user interfaces by letting you declare them with simple terms. As an example, let’s say the goal was to create a basic user interface like this:

A framework can provide the language to describe:

1. A square view.
2. A green background.
3. The text “Hello, world!”

The code to describe this may look something like this:

For more information about building apps, read the introduction to React Native or this tutorial on React.

In traditional methods of UI programming, creating this same interface would have required more code, created more complexity, and would have been a worse experience for developers trying to iterate and see their changes.

Measuring a framework’s accessibility

A benefit of using certain frameworks is simplification, but this can also hinder or remove the ability to create accessible features. To evaluate React Native, we used a three-step process: 

1. Create a glossary of capabilities available on the native platform, i.e., iOS or Android
2. Map these capabilities to a universal set of requirements
3. Analyze and score the framework’s ability to accurately build each capability

The list of accessibility capabilities available in iOS and Android is large. Creating a glossary required meticulous evaluation of a variety of platform documents, affiliated websites, and actual code. We left no stone unturned.

Next, we created a universal set of requirements based on a mixture of industry standards and Facebook-specific definitions. We then further organized them into specific guidelines. We used guidelines that could be demonstrated with examples for which we could assign a pass or fail rating. For example:

Category: Content management (grouping elements)

Native platforms (iOS and Android) have ways to group multiple elements into a single focusable unit for a screen reader. For instance, if a box of text and an image need to act as a single button, those elements must be correctly grouped in order to prevent them from being read separately or in the wrong order.

Android — expected behavior (PASS ✅)

When focusing on a grouped element, the announcement should contain the content of all grouped items. The items grouped should not be focusable independently.

Code example { … }

iOS — expected behavior (PASS ✅)

When focusing on a grouped element, the announcement should contain the content of all grouped items. The items grouped should not be focusable independently.

Code example { … }

Organizing the research

We knew that in order to maximize the usefulness of the research, we needed to boil it down into a digestible format. A dashboard could summarize achievements and encourage change by showing progress over time. Then progress would be a score derived from the overall pass rate of the stated requirements. We eventually realized that this method could be used to summarize other frameworks, as well, not just React Native.

When an engineer wanted to dig into details for a particular framework, the dashboard would serve as a home screen for navigation. The requirements, feedback, and score would be categorized under the appropriate framework. Detail pages would exist for every requirement. Examples and related links could be collected together to make extended research easier.

We were surprised by the insights we gained from this work. Applying a consistent set of questions to every framework gave us the opportunity to assess how each framework interacted directly with its native platforms. For example, the concepts of roles and traits often used a different vocabulary. Would a developer understand how to assign a role for text editing on Android if the same trait did not exist on iOS? We ended up reviewing and revising many of our original definitions to make sure that a developer could confidently annotate visual elements to provide a consistent user experience for any assistive technology.

Since taking the pledge last year, we have successfully launched our framework review tool with more than 50 accessibility requirements. The React Native team has discovered 90 issues as a result of our collaboration with the open source community. Several of the teams that manage our internal mobile frameworks have voluntarily joined our internal review ecosystem to improve their framework accessibility. 

We are excited to say that we’re well on our way to making React Native more accessible with the help of both this template and the React Native open source community. We’ll continue to use this tool to address other frameworks in the near future as we scale up our accessibility work. Frameworks are an important success factor for accessibility, providing extraordinary leverage in making apps more accessible, and we encourage others to examine their own frameworks and choose wisely.