Seminar by Samory Kpotufe

Statistics and Biostatistics seminar series

Samory Kpotufe
Columbia University

Room: M3 3127

A more Unified Theory of Transfer Learning

Transfer Learning aims to leverage samples from different but related distributions to improve performance on a target task. In its simplest form, one aims to optimally aggregate data from one source distribution with data from the target task.

Multiple procedures have been proposed over the last decade to address this problem, each driven by one of many possible divergence measures between source and target distributions. We ask whether there exist unified algorithmic approaches that automatically adapt to many of these divergence measures simultaneously.

We show that this is indeed the case for a large family of divergences proposed across classification and regression problems: these divergences all happen to upper-bound the same measures of continuity between source and target risks, which we refer to as "moduli of transfer", hence reducing the algorithmic question to that of adapting to these moduli. This more unified view allows us, first, to identify algorithmic approaches that are simultaneously adaptive to these various divergence measures—via a reduction to certain types of confidence set. Second, it allows for a more refined understanding of the statistical limits of transfer under such divergences, and in particular, reveals regimes with faster rates than might be expected under coarser lenses.

The talk is based on joint work with collaborators over the last few years, namely, S. Hanneke, and also M. Kalan, N. Galbraith, Y. Mahdaviyeh, G. Martinet, , J. Suk, Y. Deng.