Physiological Measures for Games and VR: Novel Tools and Approaches

On August 9, 2023, Dr. Ifi Mavridou and Dr. John E. Muñoz (J&F Alliance, Adjunct) spoke at a panel on the use of physiological monitoring, biofeedback equipment, and tools for VR applications and research. Both researchers are experts in this field, with Mavridou talking about the creation and design of hardware and Muñoz about design and the use of software, they presented their experiences on what these tools can offer for research in games and more. The applications that both Mavridou and Muñoz work with are cutting edge and provide researchers with a toolkit on how they can design and approach their studies. This approach personalizes and tailors the study design not only to make it easier for researchers but also for study participants to jointly design and study immersive experiences.

Mavridou shared her experience in working with EmTeq Labs where she works in the design and creation of extended reality (XR) technologies and wearables. These wearables, much designed like VR headsets, involve biometric equipment that measure how users respond and feel through facial expressions, combining a wide variety of biometric sensors into one device. This would reduce the overall need for excess equipment and wires that often weigh down participants.

Mavridou also noted an overall lack of specialized equipment but a need for it in research. In the past, the types of behaviours and affects of biometric technologies were captured as high and low energy states. The struggle with this equipment is assessing negative and positive states of valance—or how things like approval or disapproval are measured (ie. happy or unhappy). Knowing the valence state can inform researchers on the causation of high or low energy states and what possibly caused that reaction. One of the examples Mavridou brought up was a public speaking training simulation in VR. The participants involved all had a fear of public speaking and through using these types of technologies, researchers could see their physiological reactions and coach the participants through the exercise, helping them build their confidence.

Muñoz opened his remarks by highlighting the importance of physiological signals in research. These signals—such as pupil response, skin temperature, blood pressure, and much more—are used to extract unconscious player states that can’t be accessed through other methodologies like stress management.

Muñoz provided examples on how researchers can gather this data and the different types of methods that are used in studies that require biofeedback as a measurement (questionnaires, interviews, game metrics, and physiological metrics). He noted the positives and drawbacks for each method. For example, questionnaires are quick and easy to analyze but are often dishonest and it’s hard to convey emotions when filling them out. With Muñoz’s focus primarily being on physiological metrics, he finds that interpreting findings is challenging because of the nature of the data being collected.

To Muñoz, what’s most important is to experiment. No one methodology is perfect but rather, these methodologies are complementary and should be used in conjunction with each other to get the most accurate results. He then focused on how physiological metrics can be used specifically to collective player feedback in games-based studies. He noted several types of body signals that can be captured in studies and the data they capture, including: