Creating an emotional understanding in virtual humans

Student demonstrates the COACH artificial intelligence interface
As technology advances to meet more of our needs, humans need to feel comfortable using it. One of the final barriers to the adoption of intelligent cognitive assistant technology, or virtual humans, is the humans place on emotional intelligence and adaptation. Presently, more virtual assistants/humans with stagnant interfaces don’t change according to the user’s affect.

This emotionally stagnant style of reply can deter people from adopting the technology into their daily lives. With the introduction of more useful technologies, researchers believe they can address this problem, especially for the cognitively disabled.

People interact with others based on the social situation they find themselves in – one treats their mother differently than their doctor. This is the base for sociological and psychological theories of human interaction. Professor Jesse Hoey and his team operationalized this theory to create a model of affective awareness and applied it to AI interfaces. The researchers mapped facial expressions to dimensions of an affective space to inform the virtual human of affective changes in users. It then modifies the affect of its response to avoid aversion. 

Recently, a generalization of the model was proposed, called BayesAct. It guides artificially intelligent systems on an emotional level, and allows for complex affective sentiments to be modeled. Hoey applied this mathematical model to the Cognitive Orthosis for Assisting with aCtivities in the Home, or COACH, system – an artificial intelligence interface that assists in hand washing, via camera and audio prompts, for cognitively compromised elderly patients.

This work is a step towards building emotionally aware cognitive assistants, but more importantly, a step towards a smoother transition of artificially intelligent software into our daily lives. This kind of mathematical modeling of changing human affective needs, and subsequent adaptations in response to them, is of high value in development of AI interfaces