Haptics is an integral part of Virtual Reality (VR), but creating stimuli is a difficult task for beginner designers and experienced hapticians alike. A set of examples can help: a study of Macaron, a example-based haptic editor, showed that examples helped designers learn idioms of haptic design and create designs more quickly. As libraries inevitably grow larger, navigation and search techniques will need to scale to work with them.
With this project, we sought to answer a central question: how can we navigate large sets of design examples and use them in design?
Creating 500 hand-crafted designs
To study how we navigate a very large library of examples, we first had to create a library of examples. Unfortunately, we couldn't scrape an existing data set as there simply aren't any - existing libraries are at most ~100-150 items. We had to create our own, setting the target of 10,000 examples (two orders of magnitude larger than existing libraries). However, this was harder than we expected.
Multiple designers
We involved fourteen designers internal to our team, including internal undergraduate research assistants and members of the University of Waterloo’s Games Institute, to ensure a wide variety of perspectives and variety of sensations.
Need for concrete, varied design prompts
Two design tools
The result of this design activity was 500 bespoke haptic designs.
RecHap System
Data augmentation: 500 to 10,000 designs
Once we had 500 bespoke design, we used data augmentation techniques to create a larger data set to similate a very large library of examples in our interface. We varied original designs through random scaling, rotation, duration, and brush selection.
Autoencoder training
We then trained an autoencoder model on the resulting 10,000 designs to encode the designs into a latent space. This could let users navigate similar and dissimilar examples through the latent space. Although similar methods could have yielded similar results (e.g., nearest neighbour sampling), this architecture might be more extensible into generative models like GANs.
RecHap Interface
The autoencoder was then embedded into a design tool that featured a browser with search, a history feature, a live preview displaying the sensation visually and with haptics, and a recommender panel.The recommender panel provides 10 recommended designs, a mix of similar (nearby) designs and dissimilar designers.
User study
12 participants explored examples and were given the option to design their own. We compared with and without suggestions as a within-participant design.
We found that the version with suggestions improved most measures of creativity support, including results worth effort, expressiveness, exploration, enjoyment, and collaboration.
Materials
- Citation:
- K. Theivendran, A. Wu, W. Frier and O. Schneider, "RecHap: An Interactive Recommender System for Navigating a Large Number of Mid-Air Haptic Designs," in IEEE Transactions on Haptics, vol. 17, no. 2, pp. 165-176, April-June 2024, doi: 10.1109/TOH.2023.3276812.
- Paper (IEEE Link)