Please note: This seminar will take place in DC 1304.
Jun
Gao,
PhD
candidate
Department
of
Computer
Science,
University
of
Toronto
3D content is key in several domains, such as VR/AR, architecture, film, gaming, and robotics, and lies in the heart of the metaverse applications. While generative AI has achieved significant success in language, image, and video, its application in 3D content encounters fundamental challenges in the scarcity of 3D training data and increased complexities inherent in 3D.
In this talk, I will present my research on developing 3D generative AI models to create realistic, high-quality, and diverse 3D content. First, I will discuss how bringing the 3D modeling techniques from computer graphics into 3D generative AI could not only enhance efficiency but also unlock new capabilities in generating relightable and simulable 3D content. Such a modeling approach further regularizes the generation behavior, enforcing the model to focus on the 3D geometry. Second, I will show how leveraging 2D foundation models could facilitate high-quality and diverse 3D generation for geometry, texture, and semantics by combining our 3D modeling techniques with differentiable rendering. Finally, I will discuss the future direction of generating realistic 3D virtual worlds for immersive interactions with humans.
Bio: Jun Gao is a PhD candidate at the University of Toronto, advised by Prof. Sanja Fidler. He is also a Research Scientist at NVIDIA.
His research lies in the interaction of computer vision, computer graphics, and machine learning, focusing on developing generative AI models to create 3D content for reconstructing, generating, and simulating lifelike 3D worlds. His research received the SIGGRAPH Asia Best Paper award. Previously, Jun received his Bachelor’s degree from Peking University.