Speaker: Prof. Graham Taylor
Abstract: Conditional Generative Adversarial Networks (cGANs) are finding increasingly widespread use in many application domains, most notably text-to-image synthesis. In this talk, I will address two outstanding limitations of this paradigm.
First, existing research has primarily focused on generating a single image from available conditioning information in one step. A practical extension of one-step generation is generating an image iteratively, conditioned on ongoing linguistic input or feedback. I will present a recurrent image generation model which takes into account both the generated output up to the current step as well as all past instructions for generation. This model is able to generate the background, add new objects, and apply simple transformations to existing objects.
Second, quantitative evaluation of such models often involves multiple distinct metrics to assess different desirable properties, such as image quality, conditional consistency, and intra-conditioning diversity. In this setting, model benchmarking becomes a challenge, as each metric may indicate a different "best" model. I will present the Fréchet Joint Distance (FJD), which is defined as the Fréchet distance between joint distributions of images and conditioning, allowing it to implicitly capture the aforementioned properties in a single metric. We use FJD to compare existing cGAN-based models for a variety of conditioning modalities (e.g. class labels, object masks, bounding boxes, images, and text captions.
Speaker Bio: Graham Taylor is a Canada Research Chair and Associate Professor of Engineering at the University of Guelph. He directs the University of Guelph Centre for Advancing Responsible and Ethical AI and is a member of the Vector Institute for AI. He has co-organized the annual CIFAR Deep Learning Summer School and trained more than 60 students and researchers on AI-related projects. In 2016 he was named as one of 18 inaugural CIFAR Azrieli Global Scholars. In 2018 he was honoured as one of Canada's Top 40 under 40. In 2019 he was named a Canada CIFAR AI Chair. He spent 2018-2019 as a Visiting Faculty member at Google Brain, Montreal.
Graham co-founded Kindred, which was featured at number 29 on MIT Technology Review's 2017 list of smartest companies in the world. He is the Academic Director of NextAI, a non-profit accelerator for AI-focused entrepreneurs.
Seminar Recording: https://youtu.be/R9BiNsuwTjM