PhD Seminar • Artificial Intelligence • Multi-Modal Discussion Transformer: Integrating Text, Images and Graph Transformers to Detect Hate Speech on Social Media

Monday, December 2, 2024 12:00 pm - 1:00 pm EST (GMT -05:00)

Please note: This PhD seminar will take place in DC 3317 and online.

Liam Hebert, PhD candidate
David R. Cheriton School of Computer Science

Supervisors: Professors Robin Cohen, Lukasz Golab

We present the Multi-Modal Discussion Transformer (mDT), a novel method for detecting hate speech on online social networks such as Reddit discussions. In contrast to traditional comment-only methods, our approach to labelling a comment as hate speech involves a holistic analysis of text and images grounded in the discussion context. This is done by leveraging graph transformers to capture the contextual relationships in the discussion surrounding a comment and grounding the interwoven fusion layers that combine text and image embeddings instead of processing modalities separately. To evaluate our work, we present a new dataset, HatefulDiscussions, comprising complete multi-modal discussions from multiple online communities on Reddit. We compare the performance of our model to baselines that only process individual comments and conduct extensive ablation studies. We then conclude with a reflection on how this work can be applied broadly to holistically understand any social media conversation.


To attend this PhD seminar in person, please go to DC 3317. You can also attend virtually on Zoom.