The Games Institute Seed Symposium - A Retrospective

On April 19^th, the Games Institute (GI) celebrated the conclusion of its 2022 seed grant program with “Disrupting Disciplinary Divides for a Digital Future” – a research symposium. The symposium not only demonstrated the outcomes of 8 separate research projects but also what makes the GI stand out as a unique research ecosystem.

As remarked on by the Executive Director Dr. Neil Randall (English Language and Literature), “If we have learned one thing, it is that interdisciplinary research doesn’t come from putting someone from Humanities and Social Sciences and someone from STEM in a box and asking them to dance. But I can tell you for certain, after doing this for over ten years, that interdisciplinary collaboration does often seem to arise from a shared lunch, a conversation between cubicles (or as we call them pods), a running joke, or a friendly game of Mario Kart.”

Supporting interdisciplinary, and transdisciplinary, research has always been the core purpose of the seed funding as well as our institutional mission. As technology becomes ever more integrated in our lives, GI researchers are asking questions like:

• How can haptic technology be used in VR environments to train healthcare professionals?
• How can VR training simulations help first aid responders to better locate missing persons?
• How can games be used as educational tools for matters of history, climate change, and more?

The GI research symposium offered a snapshot of the daily research GI members conduct that contributes to the future directions outlined in UWaterloo's Global Futures.

We invite you to read this post event recap—project results, researcher insights, and commentary about the nature of interdisciplinary research, its benefits, and the rewarding resulting impact of it.

📣hashtagDementia in hashtagFirst Nations communities is three to five times more prevalent than in the rest of Canada’s population. People with dementia are at higher risk of going missing, and many first responders are not trained to manage dementia-related incidents any differently than other missing persons cases.

Dr. Hector Perez, a former postdoctoral fellow at the GI, and his project team—in partnership with Elders and two hashtagIndigenous organizations: Kahnawà:ke SAR and Peacekeepers in Quebec and Fire and Fisher River Ambulance from Peguis First Nation in Manitoba—collaborated on their seed project, entitled “The Future of Dementia Friendly Edu-action for Indigenous First Responders.”

The purpose of the project was to co-develop training materials in hashtagvirtual reality for Indigenous hashtagfirst responders to better locate missing persons with dementia. A challenge Dr. Perez faced was how does he honor the time and effort of research collaborators that work to fit in with Indigenous protocols and as well as ensuring that the OCAP (Ownership, Control, Access, Possession) Principles are upheld.

💡“How can we acknowledge the time of our participants? Most of them were volunteers and most of them from the Indigenous communities. At the time there were no specific guidelines at Waterloo on how to acknowledge the time of our Elders, the time of our Indigenous participants, so we worked closely with GI administrative staff, Agata Antkiewicz and Pamela Maria Schmidt, to try and come up with a solution.” - Dr. Perez

🌱The solution was to honour the cultural significance of tobacco in Indigenous knowledge protocols, recognize contributions of knowledge with honouraria to Elders and Knowledge Keepers, and respect the community involvement at every stage of the project. Dr. Perez and his team emphasized the impact of research co-creation and provided an understanding of how this can be done in a way that is respectful, reciprocal, and equitable to the Indigenous Peoples that are impacted by this work.

In fact, Dr. Perez has already started! The training videos that were designed with evidence-based search scenarios specific to Indigenous communities have already been disseminated to the first responder groups involved. The videos are available in Mohawk, Ojibwe, English, and French.
 

One of the most important aspects of equity and diversity at the Games Institute is the co-creation of knowledge with community partners representing equity-deserving groups. PhD candidate Veen Wong (School of Public Health Sciences), along with Marillac Place—a community-based organization that provides residential support to mothers and mothers-to-be facing homelessness—presented the seed project “The Future of Community Care” where they worked to improve the services Marillac Place provides.

📣During their stay, Marillac Place residents participate in the “Praise with a Raise” Program (PWAR) which helps them build essential life, parenting, and social skills for independent living. However, the program was more analog based, involving hard copies of program tasks to track participant progress, but Marillac Place wanted to develop an app that made the program more interactive to facilitate empowering self-determination. With these improvement goals, Marillac Place hoped that residents would feel more motivated to complete the program and use the skills learned in day-to-day life.

When reflecting on the research process, Wong reminisced on the importance and impact of this research beyond standard academic deliverables (i.e. conference papers and journal articles, etc.) saying that: 💡“When you’re working in a community-based project, there are real world impacts. And I think it’s just so powerful that the work that Marillac Place is doing and again being able to as, as a research team, support Marillac Place in addressing and disputing homelessness is such a big piece of providing this area of community care.”

🌱Having been deployed at Marillac Place, Wong and the research team intend on release the app as an open-source resource so that other community organization can develop similar programs and apps. In making the research and its findings accessible, "The Future of Community Care" will go on to inspire and encourage others by providing them with the tools in addressing homelessness and helping people at risk of being homeless.
 

As technology continues developing as a permanent fixture in our lives, researchers are rushing to understand it’s impact on how we communicate with each other. The recent explosion of consumer-grade hashtagvirtual reality (hashtagVR) headsets has led to the creation of social virtual reality (SVR) such as hashtagVRChat and Rec Room platforms. SVR is presented as a revolution in online communication offering novel opportunities for creative, physical, and social interactions.

📣 A key component of VR research is understanding embodiment: how do users feel in a virtual body or the “avatar” and can they be fully immersed in the activity or game they are engaging with if the avatar does not feel representative of themselves? Participants’ identities and their position in current power structure can significantly shape their representation choices in virtual environments, affecting how they choose to participate in SVR. This is what Dr. Eugene Kukshinov (Stratford School of Business and Interaction Design) and his team explored in “The Future of Social Virtual Reality” project.

💡“People use SVR to collaborate, to have fun. These platforms were very popular during the COVID-19 pandemic, where people couldn’t go to places or be together, so they would come together in VR to socialize, learn new languages, and watch movies together during a time when this wasn’t possible in person.” – Dr. Kukshinov

The team designed experiments that tested immersive human experiences in SVR, such as the feeling of “being represented”. Participants created avatars and environments, and the research team analyzed how and why users of diverse demographics customized their virtual presence in SVR. They collected over 100 survey responses and in-depth interviews with a diverse group of users to challenge how SVR is used for self-expression and self-representation beyond that of the traditional “white male” culture.

🌱Kukshinov is now joining a growing body of work produced by GI researchers challenging the ethical considerations of VR implementation including gendered language in VR, impacts of gender on motion sickness in VR, and applications of VR in healthcare.

📣Do you know how much data your digital devices collect about you? Are you aware of the emotional and physiological data that they collect? Digital devices like mobile phones, smart watches, and the hashtagInternet of Things (eg. smart thermostats, kitchen appliances, self-driving cars) are all under the umbrella of hashtagExtended Reality (hashtagXR) technology, and it’s important that users understand what sorts of data these devices collect and if that data is secure and private.

These questions formed the basis of PhD candidate Hilda Hadan’s research project “The Future of Privacy in Immersive XR.” The project tested 464 study participants on their awareness of XR technology recording users’ physical data (heart rates, sitting positions, or physical movement) and cognitive data (stress, depression, excitement). In post study debriefs, participants expressed significant concern over the unconsented recording of cognitive data and the privacy surrounding their emotional responses.

Project findings suggest that user agreements should deliver better information on what user data is and isn’t collected.

💡“There have been lots of different proposals that are making those policies a little bit easier to understand and to read, but the part that’s very interesting to me is why are we still stuck with these text-based policies? Are there other types of modalities that we can use? Expanding that design space in XR that students like Hilda investigate from a design perspective has huge potential.” – Dr. Leah Zhang-Kennedy (Stratford School of Business and Interaction Design)

🌱Improving understanding of threats to data privacy in XR is vital. As these technologies become more ubiquitous, such studies become increasingly important regarding data privacy, surveillance, and personal sovereignty.

If you’d like to learn more about this project, check out the paper “Privacy in Immersive Extended Reality: Exploring User Perceptions, Concerns, and Coping Strategies”, presented at this year’s hashtagCHI 2024 conference.
 

📣Providing adequate training to hashtaghealth care professionals is a complex task. hashtagRehabilitation practitioners rarely have the opportunity to develop hands-on experience diagnosing joint injuries. As novice practitioners enter the field, their knowledge is often more theoretical than practical.

PhD candidate Marco Moran-Ledesma's (Systems Design Engineering) project “The Future of Extended Reality Healthcare” involves hashtaginterdisciplinary collaboration with engineers, kinesiologists, and medical practitioners to create a high-fidelity physical prototype of a human leg. Their goal is to design and develop training equipment that is accessible to healthcare professionals.

The artificial leg and knee joint have been constructed to look as realistic as possible. The current prototype is a full-length leg from thigh to foot, that has plastic bones, cartilage, and ligaments which are covered in a silicon rubber skin that mimics the feeling of real skin and muscle tissue. The prototype is also implanted with haptic technology to help mimic the texture, feel, and density of an injured joint to help improve the hands-on training kinesiology students need when making injury assessments.

🌱Marco is currently recruiting physiotherapists, athletic therapists, and other healthcare professionals to test the device. The long-term goal is to have the joint used as a training tool in an extended reality environment where students and educators can practice their skills.

💡“We want to create a hashtagvirtual reality based training scenario in which, for example, we have a virtual clinic with an avatar, and we could perhaps collocate the leg of this avatar with this device. So, whoever is using that environment can experience more realistic clinical interactions, perhaps incorporating hashtagAI to get feedback and questions.” – Marco Moran Ledesma
 

📣hashtagSocial media has long been used as a tool to amplify the voices and concerns of marginalized communities to a broader audience, but over the last two decades, antifeminist sentiments (including racist, imperialist, queerphobic, transphobic, and ableist views) have been on the rise. Despite repeated calls for regulation and protection, few exist.

Dr. Brianna Wiens (English Language and Literature) and her team extensively researched antifeminism on social media for their project “The Future of Digital hashtagFeminism.” The team ran a series of workshops aimed at members of the University of Waterloo community, to create toolkits that would equip workshop participants with an understanding of programming, archiving, and designing from a feminist perspective. One of the biggest challenges the research team encountered was confronting the nature of the internet itself—the ephemeral way social media platforms change, how quickly information spreads, and how incredibly easy it is to lose vast amounts of material with one wrong click.

💡“We need to be thinking on these smaller scale levels that are often overlooked when we only prioritize big data approaches. Analyzing digital phenomena is not an easy task given the nature of the Internet and it's various compounding and sometimes competing dynamics. One of the things we've been trying to do is actually create methods for looking at these smaller scale interactions between text, visuals, and then comments within these larger structural platforms that we know are rife with violence.” – Dr. Brianna Wiens

🌱Examples of the research process are displayed through Dr. Wien’s co-managed Instagram account @aesthetic.resistance, which aims to create different forms of content as forms of counter-resistance.
 

📣There is a mounting slew of ethical concerns regarding how systems operating with hashtagartificial intelligence (hashtagAI) are being trained, and what databases AI is drawing its “intelligence” from—especially with growing outcry about AI violating copyright infringement and stealing from artists. But what if there was a method of creating ethical and sustainable databases for machine learning?

Dr. Randy Harris (English Language and Literature) is exploring this possibility in “The Future of hashtagMachine Learning.” This multi-year project aims to create a database that collects and annotates rhetorical figures, such as metaphors, figures of speech, hyperbole, and more. Currently, Harris and his team are creating a hashtagcitizen science game that would function as a method of producing highly curated and clean data sets for machine learning.

💡“Generative AI and the way it works right now is just to take a huge amount of data and extract any patterns. They can perform fairly impressively in some ways, but they also make quite nonhuman mistakes in other ways. More importantly, these trillions of packets of data involve massive energy consumption, so there is a huge carbon consumption, vast amounts of water to cool the GPUs, so all these models contribute very significantly to environmental degradation. That's one of the main aims of our work is to make machine learning much more efficient so that we so that we can reduce the kind of planetary degradation that that that is causing.” – Dr. Randy Harris

🌱Harris and team are currently developing a citizen science game called GoFigure, which will amass data through game sourcing to build up the training sets that are then pulled into the larger database called Rhetoricon.

📣When using hashtagvoice assistants like hashtagAlexa, hashtagSiri, or ashtagGoogle Assistant, hashtagmultilingual users often run into situations when their devices don’t understand their voice commands. Voice assistants are, unfortunately, often tested on native English speakers, excluding a large part of the world’s population.

PhD candidate Emily Shiu (Psychology) the problem of how voice assistants can be made more equitably in the project “The Future of Voice Assistants.” Emily and her team created LIDLBot, a custom program that simulates interactions with voice and chat assistants, as a psychological experiment. Monolingual and multilingual English speakers believe they are simply testing a new voice assistant technology, but the bot is designed to both understand and misunderstand what the participants say. Emily then analyzed if and how the participant changed their methods and strategies of interaction with the bot for the remainder of the study in an attempt to be understood.

When designing this study, Emily pointed to how people are affected emotionally and psychologically when a voice assistant misunderstands them. The study demonstrated that multilingual participants had lower success rates at guessing how often LIDLBot would understand them, demonstrating lower confidence levels compared to the monolingual participants. The experiment demonstrates the need for more diverse programming teams and backgrounds when building voice assistants to make them more equitable and accessible.

💡Designing a voice assistant requires the combined efforts of multiple disciplines. From the field of hashtaglinguistics, we have to know how people pronounce things to make speech sounds. How do languages differ from one another and what is the language use look like for people who know more than one language? From hashtagpsychology, we need to know the kinds of biases these people are coming in with. How are people perceiving the world around them, and how do these inform people's decisions and actions? hashtagComputer science tells us how to design the assistance program structure to translate user interactions with the machine. And finally, to bridge that all we have hashtaghuman computer interaction. How do we take person-to-person interactions and translate that into a seamless interaction with the computer?” – Emily Shiu

🌱Emily hopes to expand on this project by examining gendered differences and experiences users have with voice assistants. This would further highlight the inherent biases programmed into these technologies and how that impacts peoples experiences with them.