MASc Seminar: Evaluating a Social Robot as a Public Speaking Coach

Tuesday, November 21, 2023 10:00 am - 11:00 am EST (GMT -05:00)

Candidate: Delara Forghani

Date: November 21, 2023

Time: 10:00 AM - 11:00 AM

Remote Attendance for those who cannot attend in person: https://uwaterloo.zoom.us/j/97657712377?pwd=aTRVdml1dWlVMHJ4YThreXhhQzZidz09

Location: E5 5047

Supervisor(s): Kerstin Dautenhahn and Chrystopher Nehaniv

Abstract:

We all know that skills are developed through practice and rehearsal. Mastery and leadership often arise from dedicated effort. Among the skills that demand ongoing attention and practice, public speaking holds a crucial place. It is a skill by which individuals are assessed in various facets of life, be it in academia or as they progress in their professional careers. Nonetheless, when it comes to advancing in this skill, it is notably more time and energy-efficient to receive feedback from an individual capable of providing a reliable analysis of our performance. Ideally, engaging a human public speaking coach would be the optimal choice. However, accessing a human coach may not always be feasible due to a variety of reasons.

In this master's thesis, we delve into our journey of exploring the potential of using social robots as coaches for public speaking rehearsals, with the aim of enhancing individuals' presentation skills.

My MASc project begins by conducting a comparison between two interactive technologies: a voice assistant agent and a social humanoid robot, both considered as potential artificial systems for enhancing presentation skills. This comparison explores the embodiment of these technologies and underscores the significance of incorporating social non-verbal behaviours in their embodiment capabilities. To carry out this comparison, we organized an online study through Amazon Mechanical Turk, involving a diverse sample of participants across different age groups. Participants were asked to share their perspectives on various aspects of the agents depicted as coaches for public speaking rehearsals. They were presented with a scenario involving a video showing a student's brief presentation rehearsal in front of both agents, followed by feedback provided by these agents. Our analysis encompassed both quantitative and qualitative methods. While participants generally displayed a preference for using social interactive agents as coaches for public speaking rehearsals, the social robot outperformed the voice assistant agent on several metrics. Notably, it received higher rankings in terms of human nature attributes, perceived likeability, and perceived warmth.  

Building upon the findings of the first study, and considering the more favourable evaluation of the social robot, we selected a social robot as the coaching agent for public speaking rehearsals involving university students in a single in-person presentation rehearsal session. The social robot's role involved analyzing participants' speech quality and audience orientation and providing verbal feedback based on the calculated performance analysis.

In this study, we sought to gauge participants' interactions with the robot, specifically their level of acceptance and their willingness to use the system in the future. We also administered questionnaires to assess participants' perceptions of the robot's behaviour during the session and the accuracy of its feedback. The results of this study highlighted strong agreement among participants regarding the use of the social robot as their public speaking rehearsal coach within the university context, and their expressed intention to utilize it in the future. Furthermore, participants' perception of the robot's competence and usefulness was closely linked to the accuracy of its feedback and the perceived quality of its listening behaviour. Also, we asked a human coach to evaluate the robot coach, and presented the results. Future work can improve on this work and provide an accessible public speaking coach that University students can benefit from.