Some chairs may look futuristic, but a particular chair at the Cheriton School of Computer Science is futuristic. It has been custom fitted with sensors, servos, computers and a small projector by a team of human-computer interaction researchers to create an office chair as a platform for personal spatial augmented reality.
“Much research has shown that the movements we make in office chairs have implicit meaning, and because these movements have meaning they can be enhanced by spatial augmented reality,” explains project lead Nikhita Joshi, a PhD student advised by Professor Daniel Vogel. “For example, leaning back in an office chair suggests a person is relaxing. We wondered if we could enhance and support such movements using spatial augmented reality.”
L
to
R:
PhD
candidates
Nikhita
Joshi
and
Antony
Albert
Raj
Irudayaraj,
and
Professor
Daniel
Vogel
from
the
Cheriton
School
of
Computer
Science’s
Human-Computer
Interaction
Lab.
PhD
graduate
Jeremy
Hartmann
was
unavailable
for
the
photo.
Nikhita’s
research
focuses
on
understanding
how
significant
interface
constraints,
such
as
time
or
character
limits,
can
be
leveraged
to
enable
users.
She
is
also
interested
in
augmented
reality,
novel
interaction
techniques,
and
software
learning.
Antony’s
research
interests
span
interactive
displays,
wearable
systems,
and
haptics.
Professor
Vogel’s
research
focuses
on
fundamental
characteristics
of
human
input
and
novel
forms
of
interaction
for
current
and
future
computing
form
factors
like
touch,
tangibles,
mid-air
gestures,
and
whole-body
input,
for
everything
from
on-body
wearable
devices
and
mobile
phones
to
large
displays
and
virtual
reality.
Jeremy
graduated
from
the
Cheriton
School
of
Computer
Science
with
a
PhD
in
2022.
His
research
explores
merging
the
real
with
the
virtual,
which
his
company
MTION
is
putting
into
practice
as
a
virtual
reality
streaming
service
and
digital
clubhouse.
Spatial augmented reality is a technology that merges the real and virtual worlds by super-imposing computer-generated content onto surfaces using one or more digital projectors.
“Say you’re leaning forward in your office chair,” Nikhita said. “Leaning forward suggests you’re engaged in a task or meeting with a co-worker. A spatial augmented reality system could detect your sitting posture and the location of the chair in the office, then use these inputs to project digital content you’re discussing with a co-worker on the wall, perhaps in a zoomed in way.”
But before a standard office chair can be used to project computer-generated images on surfaces, the chair itself needs to be able to sense its environment. It needs to be instrumented.
Chair
sensors
and
projector
actuation
(a)
Force-sensitive
resistors
on
the
back
and
seat
of
the
chair
sense
the
user’s
posture
(b)
Capacitive
copper
strips
on
the
armrests
sense
touch
input
(c)
A
circuit
board
and
Arduino
computer
under
the
seat
processes
the
chair
inputs
(d)
A
Raspberry
Pi
computer
under
the
seat
collects
the
data
and
images
and
streams
them
to
a
local
server
(e)
A
custom
mount
under
the
right
armrest
holds
a
depth
camera
to
sense
chair
location
and
a
servo
pan/tilt
mechanism
to
steer
the
projector
(f)
An
accelerometer
on
the
chair’s
back
tracks
tilt
and
rotation
(g)
A
battery
pack
powers
the
devices
“For touch input we placed a series of capacitive sensors on the chair’s armrests,” explains Antony Albert Raj Irudayaraj, a PhD student advised by Professor Vogel who collaborated with Nikhita on the project. “The back of the chair has an inertial measurement unit — an accelerometer — to detect tilt and rotation, and the seat and backrest have force-sensitive resistors to detect seated postures and leg position. A depth camera tracks the chair’s position and surfaces in the office where content can be projected. Underneath the chair are small battery-powered computers to process these inputs and determine the projector’s output. On the right side is a servo-actuated pan/tilt head that determines where the projector displays digital content in the office.”
It’s important to note that an instrumented office chair does not use the various inputs from a user to explicitly control the projector’s output, Professor Vogel notes. “The system we developed processes implicit input to support some action we want to do. The system then finds the best way to support that action. When you move the chair, the projector has to move to compensate for that movement. It’s a complex relationship that combines what the user is doing, how they’re sitting, what the contextual cues mean, and where the chair moves. The system processes these inputs and brings them together using a projector to place digital content on surfaces in the office that’s useful to the user.”
With the chair now instrumented, the team explored 11 demonstration applications in a typical office to evaluate their proof-of-concept system.
“We made videos of these demonstration applications across three categories — notifications, supporting work-related tasks, and encouraging breaks and relaxation,” Nikhita said. “We then conducted an online survey to evaluate how the chair’s technical components work together, by asking questions that examined the participants’ understanding of the system, and its perceived usefulness and comfort. We wanted feedback on our proof-of-concept because an instrumented chair is futuristic.”
Demonstration applications of an instrumented office chair as a platform for spatial augmented reality in the workplace
| Notifications | |
|---|---|
| Day at a glance | A summary of the user’s day is projected on a wall above the computer |
| Ambient notifications | Notifications are projected on nearby objects to serve as reminders |
| Be-back-soon message | When a user stands up from the chair, the person’s name, photo, and a be-back-soon message are projected on a nearby wall |
| Notification tray | Notifications of the user’s emails, social media feed, calendar events, and weather alerts, are projected on a nearby door |
| Supporting work-related tasks | |
| Enhancing a meeting | During a meeting with a co-worker, the projector rotates to display a meeting agenda or presentation slideshow on a nearby cabinet |
| Augmenting physical drawing surfaces | Brainstorming sessions with co-workers often involve drawing on a whiteboard; to augment illustration, a grid, geometric pattern, or circuit diagram, is projected on the whiteboard |
| Supporting work-related tasks | If the chair is moved toward a specialized work area, the projector displays related content such as a video tutorial |
| Encouraging work breaks and relaxation | |
| Reducing eye strain | The chair applies the 20-20-20 guideline — i.e., every 20 minutes, look away from your monitor and focus on an item about 20 feet away for 20 seconds to reduce eye strain; the projector periodically moves content from a desktop to surfaces further away |
| Deep-breathing exercises | If the user leans back in the chair, the projector displays a deep-breathing exercise to follow on the ceiling |
| Ambient lighting to augment video games | Ambient lights are projected above the user’s monitor as a video game is played to enhance the game |
| Games | Computer games are expanded to larger surfaces, such as the floor or a nearby wall, for individual and multiplayer participation |
The survey found that respondents understood the purpose of the 11 applications clearly, but were split on their perceived usefulness. Spatial augmented reality that supported tasks, reduced eye strain, served as reminders, and prompted deep-breathing exercises were seen as particularly useful.
“The survey also revealed some interesting differences,” Nikhita said. “For example, within notifications we looked at different types — abstract vs detailed notifications. With a projector everyone can see the content, not just the user, so preserving privacy becomes important. Respondents were not as comfortable showing the content of an email but liked an ambient notification that a message had been received. This is an important finding, and future work might be on ways to occlude parts of a notification to preserve privacy.”
The survey also provided insight on certain applications, such as working with tools at a desk, where the instrumented chair projects a video tutorial on the wall to aid the user, Nikhita said. “These findings could be used to dive deeper into specific uses and augment them further to make the applications more compelling.”
“We focused on tasks people do in a typical day in an office, but an instrumented chair could have many other applications,” said Professor Vogel when asked about possible future directions. “A medical exam room also has chairs that could be instrumented. A doctor’s chair could be instrumented to project a patient’s medical records or the results from diagnostic tests on the wall when the physician makes certain movements in the chair. Similarly, a teacher’s chair could be instrumented to support teaching activities in a classroom.”
This research received the Best Paper Award at SUI 2022, the ACM Spatial User Interaction Symposium, where it was presented originally in December 2022.
To learn more about the research on which this feature is based, please see Nikhita Joshi, Antony Albert Raj Irudayaraj, Jeremy Hartmann, Daniel Vogel. An Instrumented Office Chair with a Steerable Projector for Personal Spatial Augmented Reality. SUI 2022: Proceedings of the 2022 ACM Symposium on Spatial User Interaction, December 2022, article no.: 1, pp. 1–12.
Please also see the demonstration video for example of the applications in use.