Four computer science students and graduates have received the top honours at the 42nd annual Conference on Human Factors in Computing Systems (CHI). It took place in Honolulu, Hawaii, United States of America, from May 11th to 16th, 2024.
Organized by the Association for Computing Machinery (ACM), CHI is the premier international Human-Computer Interaction (HCI) conference. It is also one of the top-ranked conferences in computer science.
Every year, HCI enthusiasts like product designers, psychologists or software developers can lead or partake in workshops, panels, and presentations to discuss the latest HCI trends and developments. CHI’s ultimate goal is to explore how interactive digital technologies can make our world a better place.
Nikhita and Yen-Ting posing with their award at CHI 2024.
One of CHI’s most significant events is the paper sessions, which invite attendees to submit their latest research papers. If successful, they can present their work to various leading experts, and it will be published in ACM’s Proceedings of the CHI Conference on Human Factors in Computing Systems. However, submitting a paper to CHI is extremely competitive— the average acceptance rate hovers around 25 per cent. Besides, receiving the Best Paper award is only 1 per cent of submissions, while Honourable Mentions are 5 per cent.
This year, CHI accepted 1060 submissions, with several papers from the University of Waterloo’s David R. Cheriton School of Computer Science. Notably, PhD students Nikhita Joshi (MMath ’20) and Xinyu Shi won the Best Paper Awards. Nikhita also earned an Honourable Mention alongside alumni Damien Masson (PhD ’23) and Yen-Ting Yeh (PhD ’23).
Best Paper Awards
Less highlighting means more learning
Nikhita and her supervisor Professor Daniel Vogel co-wrote Constrained Highlighting in a Document Reader Can Improve Reading Comprehension. They explored whether a document reader that capped the number of words a user can highlight could impact reader comprehension. Previous studies have found that excessive highlighting could hamper recollection and plant a false sense of understanding. However, other solutions particularly self-regulation training, can be strenuous and time-consuming.
The team recruited 127 participants to read a short story and complete an open-book reading comprehension test, taken 24 hours later. They were divided into three groups: no highlighting, limited highlighting of 150 words, and unlimited highlighting. Notably, the group with restricted highlighting had the highest reading comprehension scores. This paper was the first to explore user interface constraints for text marking and to demonstrate that restricted highlighting can improve reading comprehension scores. It can also lead to several technological and pedagogical innovations.
The duo presented their paper in the “Supporting Accessibility of Text, Image and Video” paper session.
Scheming colours faster and easier
Xinyu wrote Piet: Facilitating Color Authoring for Motion Graphics Video with her supervisor Professor Jian Zhao, Harvard University alum Yinghou Wang, and Yun Wang, a senior researcher at Microsoft Research. Xinyu and Yinghou conducted this research during their internship at Microsoft Research Asia.
The researchers interviewed six Motion Graphic (MG) designers about their experience with colour authoring: the process of creating a colour scheme to enhance a visual medium’s aesthetic or messaging. Although colouring can benefit MG videos, including maintaining visual continuity and evoking emotions, it is plagued with several challenges. The current workflow is inefficient: designers can only work on the colours while designing individual elements in vector graphics editors like Adobe Illustrator. Onwards, the elements are put together and animated in separate tools such as Adobe After Effects. As a result, the designers can only see the colour impacts during the final production stages. Moreover, designers cannot modify the colours of elements featured in various scenes. They must edit every unit or apply a universal filter to the entire video, making it harder to change or maintain a consistent colour theme. Lastly, current colouring tools are tailored towards an object-driven methodology, whereas most designers prefer a top-down and progressive approach.
Based on these anecdotes, the team launched Piet, an innovative colour authoring tool. It visualizes colour palettes used throughout the video in three levels: overall dominant colours, scene palette in a timeline format, and colours associated with each graphical element. The designer can edit colours at any of these levels and their changes are implemented instantly. This novel approach allows colour adjustments at the post-animation stage. It also helps designers see the relationship between elements and how they are arranged together, both spatially and temporally. As a result, they can easily make colour decisions based on the video's entire arrangement and how it develops over time.
The team conducted a user study, where thirteen expert designers used Piet and felt it was a vast improvement over existing tools. It offered a more fluid, engaging, and intuitive colour authoring experience and allowed designers to explore themes and visual continuity. Many participants stated Piet could be a plugin for other tools like After Effects, leading to a seamless workflow. Overall, this state-of-the-art tool can help editors easily create aesthetically pleasing videos.
The group presented their prototype in the “Colours” session.
Honourable Mentions
Enhancing the LLM experience with a single click
Damien led the DirectGPT: A Direct Manipulation Interface to Interact with Large Language Models, with his supervisors Adjunct Assistant Professor Sylvain Malacria, Adjunct Professor Géry Casiez, and Professor Vogel. Professors Malacria and Casiez are respectively a research scientist and a Full Professor at the University of Lille in France.
The paper showcases the team’s prototype, DirectGPT, which could revolutionize large language models (LLM). A common problem with LLMs particularly chatboxes, is that it doesn’t always capture a user’s request. For example, the user may not know the best vocabulary to convey their needs or forget to include certain details. Even if they specify their request, the LLM may misunderstand such as accidentally changing the other sections of their output. Many users feel frustrated as they must constantly reword their prompt until they are satisfied with the output.
Damien presented his team's research in the “User Studies on Large Language Models” session.
With DirectGPT, users can alter its output by identifying an object of interest. For example, if someone uses DirectGPT to generate content, they can change some words by clicking on them and typing “synonym” in the text box. The words are then replaced and a “synonym tool” is activated for future requests. If a user asks DirectGPT to draw a flower but it is missing a stem, they can type “draw a line from here to there” in the text box and click on certain coordinates to indicate the “here” and “there”. DirectGPT will create a line, thus completing the illustration. For any coding tasks, DirectGPT allows users to click and edit the syntax. Ultimately, DirectGPT can easily execute a user’s request— all within a single click.
The team conducted a user study asking participants to use DirectGPT and ChatGPT for text, code, and vector image editing tasks. When using DirectGPT, participants used “50% fewer and 72% shorter prompts all while being 50% faster and 25% more successful at accomplishing tasks,” compared to ChatGPT. These results highlight that LLMs incorporating direct manipulation can enhance user experience.
Video of the team's paper presentation
Making writing tools more comfortable
Yen-Ting and Nikhita equally contributed to The Effects of Update Interval and Reveal Method on Writer Comfort in Synchronized Shared-Editors, alongside their supervisor Professor Vogel.
Synchronized share-editors like Google Docs and Overleaf are the cornerstone of several workspaces, such as taking notes for a work meeting or writing and editing a group assignment. Although these tools have bolstered collaboration, it does not support the privacy of writing. Research has found that some people feel uncomfortable if someone is constantly watching their activity, particularly if they make a typo. Some find it distracting or frustrating if others edit their work as they write it out. Overall, many people prefer to share their content when they are satisfied with it.
The team developed strategies to ease writers’ discomfort based on two methods: update intervals and reveal method. Under the update interval, content changes were only shown to collaborators based on time, character, and sentence delay. For example, if someone uses the sentence delay feature, their edits are only displayed once terminal punctuation is typed. This strategy allows users to share their thoughts only when it’s complete. Similarly, the other delay features show edits at certain fixed intervals like every 10 seconds or 15 characters.
While the update interval method focuses on when content changes are shown, the reveal method focuses on how they are shown. It employs three strategies: normal typing, perfect typing and pasting. Normal and perfect typing are both simulated typing effects. However perfect typing only shows the keystrokes that produced the final content, whereas normal typing displays all keystrokes since the last update including deleted sentences, typos and edits. Under pasting, all content changes are shown at once. The difference between perfect typing and pasting is that perfect typing will show the typing sequence while pasting doesn’t display any progression.
The researchers conducted two experiments: an image description and a paired essay-writing task. Both experiments focused on the writer’s comfort with slightly uncomfortable scenarios, such as writing the image description while an anonymous collaborator was watching. These experiments led to several unique findings. In the post-experiment questionnaire, 65 per cent of the image description task participants mentioned that some update strategies boosted their confidence, as it helped hide errors and rewording. The sentence delay and paste features were highly rated among participants from the second experiment. Ultimately, this research could transform synchronized writing tools and lead to stronger collaboration.
The trio presented their work at the “Working Practices and Tools” session.
