Emotionally intuitive AI

Jesse Hoey
Professor, Faculty of Mathematics
> David R. Cheriton School of Computer Science
> Centre for Bioengineering and Biotechnology

People suffering from the early symptoms of Alzheimer’s disease often have difficulty remembering things that recently happened to them. As the disease takes root, a person’s reasoning and behaviour can change. Day-to-day routines — like handwashing— may become challenging for them and they begin to need more assistance from caregivers for simple at-home tasks.

But now there is technology that can help.

Professor Jesse Hoey, a University of Waterloo computer scientist, has combined artificial intelligence with mathematical models of human interaction to create a virtual solution to help older adults living with Alzheimer’s disease and dementia.

The home-based technology, called ACT@Home, acts as a virtual assistant that can automatically monitor a person doing a task and offer verbal prompts to encourage them to start and complete tasks.

ACT@Home is supported by the American Alzheimer’s Association, the Canadian Consortium on Neurodegeneration and Aging and the AGEWELL Network of Centers of Excellence.

The assistive technology helps older adults with Alzheimer’s disease complete activities of daily living more independently, Hoey explains.

“People with Alzheimer’s disease may start to wash their hands, then forget what they’re doing, repeat steps, or just stop partway through,” Hoey says. “Their care partner — often the person they live with — usually steps in to help, but the amount of assistance required can become overwhelming.”

The ACT@Home assistive technology helps by giving automated prompts based on the observed situation. It does not use pre-programmed tasks, but rather, it is uses emotional intelligence to understand how to best convince a person to do something in a given situation.

“One of the failure points in artificial intelligence technology is its inability to align with humans on an emotional level,” explains Hoey.

ACT@Home combines artificial intelligence with social and psychological models by taking cues, like a person’s facial expression and posture, to tailor the prompts appropriately.

This is especially important for patients with Alzheimer’s disease who suffer not only memory and reasoning loss, but also changes in personality and in responses to power dynamics, he says.

“Emotion is what gives humans motivation and drives us to act,” explains Hoey. “Convincing a person with Alzheimer’s disease to do something can be difficult because it depends on the person’s interpretation of the situation, what they think is going on, what cues they are getting and who they think they are in that moment.”

“Our prototype works by building a model of what’s going on emotionally in the mind of someone experiencing cognitive difficulty and then prompting them to complete an activity of daily living in a way that makes sense to them in that moment.”

In a related project, Hoey and his collaborators have been conducting qualitative interviews with older adults living with and without Alzheimer’s disease as well as their caregivers. The project, called Emotional Motivation for Technology that Cares, aims to develop computational maps that identify what sorts of interactions work best for individuals.

Their work is being conducted in collaboration with two long-term care facilities, Schlegel Villages and the Research Institute for Aging in Waterloo as well as Tapestry Retirement Living in Vancouver. Hoey aims to better understand what it means to be a person with the disease and how they interact with others.

“Our goal is to take the findings from these interviews to develop an inexpensive home-based emotionally intelligent cognitive assistant that helps people with Alzheimer’s disease while lightening the burden on their caregivers,” says Hoey.