You are here

Colloquia and Joint Seminars

Upcoming colloquiums

Dr. Mauro Maggioni
Bloomberg Distinguished Professor in Mathematics and Applied Mathematics and Statistics at Johns Hopkins University,

March 9th 2017
3:30pm in MC5501
Refreshments at 3:15pm

Geometric Methods for the Approximation of High-dimensional Dynamical Systems

We discuss a geometry-based statistical learning framework for performing model reduction and modeling of stochastic high-dimensional dynamical systems. We consider two complementary settings. In the first one, we are given long trajectories of a system, e.g. from molecular dynamics, and we discuss new techniques for estimating, in a robust fashion, an effective number of degrees of freedom of the system, which may vary in the state space of then system, and a local scale where the dynamics is well-approximated by a reduced dynamics with a small number of degrees of freedom. We then use these ideas to produce an approximation to the generator of the system and obtain, via eigenfunctions of an empirical Fokker-Planck question, reaction coordinates for the system that capture the large time behavior of the dynamics. We present various examples from molecular dynamics illustrating these ideas. In the second setting we only have access to a (large number of expensive) simulators that can return short simulations of high-dimensional stochastic system, and introduce a novel statistical learning framework for learning automatically a family of local approximations to the system, that can be (automatically) pieced together to form a fast global reduced model for the system, called ATLAS. ATLAS is guaranteed to be accurate (in the sense of producing stochastic paths whose distribution is close to that of paths generated by the original system) not only at small time scales, but also at large time scales, under suitable assumptions on the dynamics. We discuss applications to homogenization of rough diffusions in low and high dimensions, as well as relatively simple systems with separations of time scales, and deterministic chaotic systems in high-dimensions, that are well-approximated by stochastic differential equations.

Past colloquiums

Yaoliang Yu
University of Waterloo

February 8th 2017
3:30pm in MC5501
Refreshments at 3:15pm

Fast gradient algorithms for structured sparsity

Structured sparsity is an important modeling tool that expands the applicability of convex formulations for data analysis, however it also creates significant challenges for efficient algorithm design. In this talk I will discuss how gradient algorithms can be adapted to meet the modern computational needs in large-scale machine learning, thanks to their cheap per-iteration costs. I will give results on how to efficiently compute the proximal map, the key component in gradient algorithms, through (a) identifying sufficient conditions that reduce the proximal map of a sum of functions to the composition of the proximal maps of each individual summand; (b) exploiting the proximal average as a provably sound approximation and yielding strictly better convergence than the usual smoothing strategy, without incurring any overhead; (c) completely bypassing the proximal map by proposing a generalized conditional gradient algorithm that requires sometimes a significantly cheaper polar operation. Throughout the talk I will demonstrate the application of these results in matrix completion, dictionary learning, isotonic regression, event detection, etc. I will conclude my talk by mentioning some progress and challenges in the nonconvex and distributed setting.

David Duvenaud
University of Toronto

January 20th 2017
3:30pm in MC5479
Refreshments at 3:15pm

Title: 
Composing graphical models with neural networks for structured representations and fast inference

Abstract:

 How can we build structured, but flexible models? We propose a general modeling and inference framework that combines the complementary strengths of probabilistic graphical models and deep learning methods. Our model family combines latent graphical models with neural network observation models. For inference, we use recognition networks restricted to output evidence potentials that are conjugate to the latent model. These local potentials are then combined using efficient graphical model inference algorithms. All components are trained simultaneously with a single scalable stochastic variational objective. We illustrate this framework with several example models, and by showing how to automatically segment and categorize mouse behavior from raw video.

Bio: 

David Duvenaud is an Assistant Professor in Computer Science and Statistics at the University of Toronto. He did his postdoc at Harvard University with Prof Ryan P. Adams, working on hyperparameter optimization, variational inference, deep learning methods and  automatic chemical design. He did his Ph.d. at the University of Cambridge, studying Bayesian nonparametrics with Zoubin Ghahramani and Carl Rasmussen.  David also spent two summers in the machine vision team at Google Research, and co-founded Invenia, an energy forecasting and trading company.

Past CM Colloquiums