PhD Defence • Human-Computer Interaction • Understanding Mode and Modality Transfer in Unistroke Gesture Input

Wednesday, September 22, 2021 2:45 pm - 2:45 pm EDT (GMT -04:00)

Please note: This PhD defence will be given online.

Jay Henderson, PhD candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Edward Lank

Unistroke gestures are an attractive input method with an extensive research history, but one challenge with their usage is that the gestures are not always self-revealing. To obtain expertise with these gestures, interaction designers often deploy a guided novice modality — where users can rely on recognizing visual UI elements to perform a gestural command. Once a user knows the gesture and associated command, they can perform it without guidance; thus, relying on recall. The primary aim of my thesis is to obtain a comprehensive understanding of why, when, and how users transfer from guided modes or modalities to potentially more efficient, or novel, methods of interaction — through symbolic-abstract unistroke gestures.

The goal of my work is to not only study user behaviour in novice to more efficient interaction mechanisms, but also to expand upon intermodal transfer to different contexts. We garner this understanding, by empirically evaluated three different use cases of mode and/or modality transitions. Leveraging marking menus, the first piece investigates whether or not designers should force expertise transfer through penalizing the guided mode to recall mode. Second, we investigate how well users can transfer skills between modalities, particularly when it is impractical to present guidance in the target or recall modality. Lastly, we assess how well users’ pre-existing spatial knowledge of an input method (the QWERTY keyboard layout), transfers to performance in a new modality.

Applying lessons from these three assessments, we segment intermodal transfer into three possible characterizations — beyond the traditional novice to expert contextualization. This is followed by a series of implications and potential areas of future exploration spawning from our work.

To join this PhD defence on Zoom, please go to