In a study done in collaboration with Microsoft, researchers have developed a new voice assistant that allows people with visual impairments to access web content quickly and effortlessly from smart speakers and similar devices.
A new tool called Voice Exploration, Retrieval, and Search (VERSE), was developed to merge the best elements of voice assistants with screen readers to create a tool that makes free-form web searches easier.
“People with visual impairments often rely on screen readers, and increasingly voice-based virtual assistants, when interacting with computer systems,” said Alexandra Vtyurina, a PhD candidate in Waterloo’s David R. Cheriton School of Computer Science who undertook the study during her internship at Microsoft Research.
“Virtual assistants are convenient and accessible but lack the ability to deeply engage with content, such as read beyond the first few sentences of an article, list alternative search results and suggestions. In contrast, screen readers allow for deep engagement with accessible content, and provide fine-grained navigation and control, but at the cost of reduced walk-up-and-use convenience.”
VERSE was designed to add screen-reader like capabilities to virtual assistants, using voice as the primary input method. Users can give commands such as “next”, “previous”, “go back” or “go forward”. VERSE can also be paired with an app, where smartphones and smartwatches provide input. For example, rotating the crown on a smartwatch can advance VERSE to the next search result or paragraph.
In the research study, 53 visually impaired web searchers were surveyed, and more than half reported using voice assistants multiples times a day. The data collected from the survey was then used to design an initial prototype of VERSE, with a user study conducted afterward for feedback.
“VERSE is differentiated by what happens next. If people need more information, they can use VERSE to access other search verticals, for example, news, facts, and related searches, and can visit any article that appears as a search result,” explained Vtyurina.
The study, titled “VERSE: Bridging Screen Readers and Voice Assistants for Enhanced Eyes-Free Web Search”, was authored by Vtyurina, along with Adam Fourney, Meredith Ringel Morris and Ryen W. White from Microsoft Research, and Leah Findlater, an assistant professor at the University of Washington.