Please note: This master’s thesis presentation will take place in DC 3317 and online.
Kris Frasheri, Master’s candidate
David R. Cheriton School of Computer Science
Supervisor: Professor Edith Law
The domain of machine learning (ML) has grappled with the challenge of curating subjective datasets, where there can be many equally valid labels due to differences in perspectives and a significant technical gap remains in how we can effectively incorporate multiple subjective viewpoints into the labelling process.
We contribute PERSONA, a dataset labelling tool that presents LLM-generated personas with diverse labelling perspectives to encourage annotators to consider different human values during the dataset labelling process. We studied how interactions with these personas affect the annotator’s decision-making patterns. Based on a two-part user study, our evaluation shows that PERSONA enriches the labelling process by prompting the annotators to reflect on different viewpoints, showing the potential value of integrating LLMs in machine learning data generation pipelines.
To join this master’s thesis presentation in person, please go to DC 3317. You can also attend virtually on MS Teams.