Background
Keyboard “shortcut keys” are often used to issue specific commands, such as a copy or paste text function. In text-heavy applications, all shortcut keys need to include one or more special modifier keys (e.g. command, control). Shortcut keys may be difficult to remember and studies show keyboard shortcuts are underused by most computer users. Researchers attribute this to a gap between graphical input and pressing keys, poor visibility and mnemonics, and the uncomfortable and error-prone act of pressing multiple keys simultaneously.
Description of the invention
Using computer vision and a camera, University of Waterloo researchers developed a virtual reality method that recognizes finger positioning on the keyboard and maps desired shortcut actions in response. The developed system detects which hand and finger is used to press a key, and whether the hand posture is open or closed for instance, so that pressing the same key can have multiple command mappings. The detection method is packaged into a background service to enable system-wide finger-aware shortcut keys in any application.
This enables a larger input space for traditional keyboards with increased expressivity by pressing keys in different ways to access more shortcuts, and increased availability by differentiating between normal keyboard input and shortcut input.
Advantages
- More natural human gesture experience that enables better ease of use
- Increased input device (eg. keyboard) access to system functionality and thus increased productivity
- Enables broader assistive device options to those with physical disabilities
Potential applications
- Replacing or augmenting standard shortcut keys in text-heavy or video editing applications
- Increased functionality for people with disability
- Customizable command mappings
- Keyboard-less computer input device
- Virtual Reality (VR)/Augmented Reality (AR) input device
- Vehicle onboard infotainment
Finger-Aware Shortcuts trigger different commands by detecting the finger, hand, and posture used to press the key.
Reference
8810-7419
Inventor(s)
Jingie Zheng
Daniel Vogel
Patent status
US Patent Pending
Stage of development
Working Prototype
Ongoing research