AutoVizuA11y

Description

AutoVizuA11y is a React library that automates the process of creating accessible data visualizations for screen reader users. It focuses on features that improve the exploration of charts with a keyboard, without requiring extensive accessibility knowledge from developers.

This tool focuses on three different key components that were identified after consulting with several screen reader users: chart navigability; insightful chart descriptions; and shortcuts that allow one to navigate the charts and get insights more quickly.

Install AutoVizuA11y

Via npm

npm i @feedzai/autovizua11y

Via Github

Clone the repository into a local directory using:

git clone https://github.com/feedzai/autovizua11y.git
cd autovizua11y

AutoVizuA11y for Developers

Inputs

• A set of props, passed in the AutoVizuA11y component, with information regarding the data visualization wrapped.
• The chart that will receive AutoVizuA11y's accessibility features.

Outputs

• Descriptions
  • Labels for every data element;
  • Two descriptions, a longer and a shorter, for every data visualization wrapped by AutoVizuA11y. Both are composed of a title, the visualization type, whether the description was automatically generated, and a human-like summary generated using the gpt-3.5-turbo model from OpenAI. The shorter description is announced by default once the data visualization gets focused. The descriptions can also be manually written.
• Navigation
  • "Horizontally", between different data visualizations and between data elements;
  • "Vertically", between data visualizations and their underlying data elements;
  • Between different data series (in case more than one exist).
• Shortcuts
  • QOL navigation shortcuts that increase speed and efficiency in the exploration of a chart;
  • Shortcuts that offer statistical insights about the data on demand;
  • A Shortcut Guide that can be consulted while focusing an AutoVizuA11y chart.

GitHub Actions

AutoVizuA11y is published via GitHub Actions. With that, zizmor is used to lint workflows and pinact to pin versions of actions like actions/checkout.

If you change the release_package.yml or tests.yml file, install zizmor (if necessary), run the following command, and implement the relevant feedback:

zizmor .

To update action versions, install pinact (if necessary) and run the following command:

pinact run -u --min-age 7

AutoVizuA11y component properties

Property	Required/Optional	Type	Description
data	Required	array of objects	The data present in the chart. The values of each pair are added, in order, to a string and read when the corresponding DOM element is focused. Note: the number of objects need to match the total data points represented in the DOM.
selectorType	Required	object	The HTML type (for example "rect", "circle", or "path") of the data elements or their class name — only one can be chosen. This enables the data elements to be navigable and have an aria-label. AutoVizuA11y assumes that the number of data elements with the specified class or type matches the number of elements passed through the data prop (ensuring that no element is left without a label).
type	Required	string	The type of data visualization. It is announced once a data visualization gets focused, after the title and before the descriptions.
title	Required	string	The title of the visualization, which should be short and concise, showcasing the purpose of the content inside the data visualization. It is announced once a data visualization gets focused, before the type and the longer or shorter descriptions.
insights	Required	string	Expects a string that corresponds to the key in the data object from which values will be used to derive statistical insights. For example, If the insights should be derived from the amount in the data, then that's what should be passed in this property. If an empty string "" is passed, the user will receive an alert stating 'This shortcut does not work in this chart.' This applies to shortcuts related to minimum, average, and maximum values, as well as those involving comparisons to these insights and other data points. Note: the values used for insights need to be of type Number.
descriptor	Optional	string	By receiving a string, this descriptor helps better contextualize what data elements are. It is added at the end of each data element. If no descriptor is provided, blank text ("") is set instead.
multiSeries	Optional	string	When working with multi-series data, provide a string that corresponds to the key in the data object that defines each series, allowing users to navigate between different series/clusters in addition to regular navigation. If an empty string "" is passed, the tool interprets the data as single series.
shortcutGuide	Optional	JSX.Element	AutoVizuA11y has its default NativeShortcutGuide but you may create your own. The ShortcutGuide is wrapped in a <dialog>, and its reference can be obtained trought the property dialogRef, which you can add to your shortcutGuide. The dialogRef is a RefObject<HTMLDialogElement>, which you can use to create, for example, a button that handles the logic of closing this dialog.
internationalization	Optional	object	Internationalization settings for the component. The options for this prop can be checked here.
autoDescriptions	Required (option A)	object	Various options regarding the creation of automatic descriptions with OpenAI models or those compatible with the OpenAI API. AutoVizuA11y does two API calls per wrapped visualization, one for each type of description (longer and shorter). The options for this prop can be checked here. This prop cannot be used at the same time as "manualDescriptions".
manualDescriptions	Required (option B)	object	Two manually written descriptions of the data. By providing this prop, no automatic descriptions are generated, thus not having any costs associated. The options for this prop can be checked here. This prop cannot be used at the same time as "autoDescriptions".

internationalization prop options

Keys	Required/Optional	Type	Description
language	Optional	string	AutoVizuA11y supports both British English and European Portuguese (en-GB and pt-PT), British English being the default. Changing the language has an impact in the messages produced, the NativeShortcutGuide and the prompt to generate the automatic descriptions. To contribute with a new language, please fork the project and modify the ./src/i18n.js file — it's crucial that every string has its own translation with the correct key.
customTranslations	Optional	object	Override the default translations for the specified language. You can override the entire en-GB or pt-PT language set, or just specific keys/messages you want to customize. The object should follow the same structure as the default translation files, with keys matching those found in ./src/i18n.js. Only provided keys will be overridden, while others will use the default translations. Important: Remember to set the language property to match the language you're customizing (e.g., set language: "en-GB" when providing new British English translations, or language: "pt-PT" when providing custom European Portuguese translations).

autoDescriptions prop options

Keys	Required/Optional	Type	Description
apiKey	Required	string	The OpenAI or OpenAI-compatible API key, enabling an LLM to generate human-like descriptions of the data visualization. You can get yours here. It is recommended for the developer to take the necessary precautions in order to hide the API key.
context	Required	string	The context in which the data visualization is present. It is passed in the prompt, when generating automatic the descriptions, resulting in contextualized outputs.
dynamicDescriptions	Optional	boolean	Setting this to false stops the component from generating the two descriptions for that chart after the first render (the descriptions get saved in localstorage). This should be useful for static visualizations.
model	Optional	string	The LLM responsible for generating both descriptions. You can check the models available here. If no model is provided, gpt-3.5-turbo will be chosen by default.
baseUrl	Optional	string	The base URL for the API compatible with the OpenAI Chat Completions API endpoint. If no base URL is provided, https://api.openai.com/v1/ (OpenAI) will be chosen by default.
temperature	Optional	number	A temperature, from 0 to 1, used in the model responsible for generating both descriptions. Descriptions with temperatures closer to 0 should be more deterministic between API calls while those being closer to 1 should be more random between API calls. You can check the models available here. If no model is provided, 0 will be chosen by default.

manualDescriptions prop options

Keys	Required/Optional	Type	Description
longer	Required	string	The longer description associated with the wrapped data visualization.
shorter	Required	string	The shorter description associated with the wrapped data visualization.

Sample Implementation

import { AutoVizuA11y } from "@feedzai/autovizua11y";

// ...

const barData = [
	{ day: "Monday", value: 8 },
	{ day: "Tuesday", value: 6.5 },
	{ day: "Wednesday", value: 7 },
	{ day: "Thursday", value: 9 },
	{ day: "Friday", value: 11 },
	{ day: "Saturday", value: 2 },
	{ day: "Sunday", value: 3 },
];

const multiLineData = [
	{ series: "Croatia", x: 2010, y: 4.37 },
	{ series: "Croatia", x: 2015, y: 4.25 },
	{ series: "Croatia", x: 2020, y: 4.13 },
	{ series: "Croatia", x: 2022, y: 4.03 },
	{ series: "Latvia", x: 2010, y: 4.25 },
	{ series: "Latvia", x: 2015, y: 4.25 },
	{ series: "Latvia", x: 2020, y: 4.25 },
	{ series: "Latvia", x: 2022, y: 4.25 },
	{ series: "Lithuania", x: 2010, y: 4.25 },
	{ series: "Lithuania", x: 2015, y: 4.25 },
	{ series: "Lithuania", x: 2020, y: 4.25 },
	{ series: "Lithuania", x: 2022, y: 4.25 },
];

const longerDesc = "...";
const shorterDesc = "...";

const customFrenchTranslations = {
	"fr-FR": {
		minimumValue: "Valeur maximale",
		maximumValue: "Valeur minimale",
		averageValue: "Valeur moyenne",
		// ...
	},
};

// ...

function App() {
	return (
		<>
			{/* SingleSeries with automatic descriptions */}
			<AutoVizuA11y
				data={barData}
				selectorType={{ element: "rect" }}
				type="bar chart"
				title="Number of hours spent looking at a screen per day of the week."
				insights="value"
				descriptor="hours"
				autoDescriptions={{
					apiKey: API_KEY,
					context="Screen time dashboard"
					dynamicDescriptions: false,
					model: "gpt-3.5-turbo",
					temperature: 0.1,
				}}
			>
				<BarChart></BarChart>
			</AutoVizuA11y>

			{/* MultiSeries with manual descriptions */}
			<AutoVizuA11y
				data={multiLineData}
				selectorType={{ element: "circle" }}
				type="Multi line chart"
				title="Latvia, Lithuania, and Croatia are among the countries where population is decreasing"
				insights="y"
				descriptor="millions"
				multiSeries="series"
				manualDescriptions={{
					longer: longerDesc,
					shorter: shorterDesc,
				}}
			>
				<LineChart></LineChart>
			</AutoVizuA11y>

			{/* Example with custom French translations */}
			<AutoVizuA11y
				data={barData}
				selectorType={{ element: "rect" }}
				type="graphique à barres"
				title="Temps d'écran par jour de la semaine"
				insights="value"
				descriptor="heures"
				internationalization={{
					language: "fr-FR",
					customTranslations: customFrenchTranslations,
				}}
				manualDescriptions={{
					longer: "Ce graphique montre...",
					shorter: "Temps d'écran hebdomadaire...",
				}}
			>
				<BarChart></BarChart>
			</AutoVizuA11y>
		</>
	);
}

AutoVizuA11y for screen reader users

All the following shortcuts only work once focused on a data visualization (or adjacent data elements) covered by AutoVizuA11y.

Besides the navigation between data visualizations, every other shortcut outcome (namely the statistical insights) only regards each individual visualization.

The tool was tested with VoiceOver, JAWS and NVDA, as well as the most commonly used browsers. The key combinations for each shortcut were chosen with the intention to avoid collisions with others from the tested screen readers and browsers. That said, it is suggested for users of JAWS and NVDA turn on "Focus mode" (Insert + Space) so that the navigation with arrow keys can be done between and inside the visualizations.

Examples

You can check a series of examples built using AutoVizuA11y here (some features require an OpenAI or OpenAI-compatible API key).

Tests

All tests have been written using cypress.

To run tests locally:

# root
npm install
npm run build

# /examples
npm install
npm run dev

# root
npx cypress open

Shortcut Guide

The Shortcut Guide can be acessed by the user, using the ? key, while having keyboard focus either on an AutoVizuA11y chart or an underlying data element. As a developer, you may override this component with your own or change its styling.

Shortcut keys

Activation Key(s)	Description
Question Mark	Enter shortcut guide
? or Escape	Leave shortcut guide
Down arrow	Get into a chart
Up arrow	Get out of a chart
Right arrow	Move forward in a page element
Left arrow	Move backward in a page element
Alt (option) + M	Move between series of data inside the chart
Home or Alt (option) + Q	Jump to the beginning of a chart
End or Alt (option) + W	Jump to the end of a chart
Alt (option) + X	Define the number of data points to be jumped at a time
+	Add one number to the data points to be jumped at a time
-	Subtract one number to the data points to be jumped at a time
Alt (option) + J	Minimum value
Alt (option) + K	Average value
Alt (option) + L	Maximum value
Alt (option) + Shift + J	Compare current point to minimum value of the visualization
Alt (option) + Shift + K	Compare current point to average value of the visualization
Alt (option) + Shift + L	Compare current point to maximum value of the visualization
Alt (option) + Z	How a point compares to the rest of the chart
Alt (option) + B	Set longer description of the chart
Alt (option) + S	Set shorter description of the chart (default)

Visual customization

The Shortcut Guide is the only aspect of AutoVizuA11y that is also visual. You may change the styling of the default guide. Below are the classNames of the elements that make up this component:

className	HTML
shortcut-guide	form
shortcut-guide__container	div
shortcut-guide__header	div
shortcut-guide__title	h2
shortcut-guide__button-label	p
shortcut-guide__button	button
shortcut-guide__break	hr
shortcut-guide__body	div
shortcut-guide__section	div
shortcut-guide__list--title	h3
shortcut-guide__list	div
shortcut-guide__row	dl
shortcut-guide__cell--shortcut	dt
shortcut-guide__cell--explanation	dd

Citing AutoVizuA11y

Below is the BibTeX entry for the EuroVis'24 full paper that explains the process of creating AutoVizuA11y.

@article{2024-AutoVizuA11y,
  title = {AutoVizuA11y: A Tool to Automate Screen Reader Accessibility in Charts},
  ISSN = {1467-8659},
  url = {http://dx.doi.org/10.1111/cgf.15099},
  DOI = {10.1111/cgf.15099},
  journal = {Computer Graphics Forum},
  publisher = {Wiley},
  author = {Duarte, Diogo and Costa, Rita and Bizarro, Pedro and Duarte, Carlos},
  year = {2024},
  month = jun
}

---

Other licensing options may be available, please reach out to data-viz@feedzai.com for more information.