Please note: This PhD seminar will be given in person.
Zhongwen
(Rex)
Zhang,
PhD
candidate
David
R.
Cheriton
School
of
Computer
Science
Supervisor: Professor Yuri Boykov
Maximizing information between network input and output is a basic principle for unsupervised training of discriminative posterior models, where it might be as fundamental as the likelihood maximization principle for the generative models, e.g., GMM. However, only a handful of unsupervised information-based techniques exist in computer vision, in part due to limited understanding of the corresponding losses.
We provide a systematic summary of existing related loss functions and motivate our self-supervised loss where pseudo-labels naturally appear as auxiliary variables when splitting mutual information. We also derive an efficient EM algorithm with closed-form solutions for E and M steps. Empirically, our approach is more robust compared to earlier methods for information maximization. Also, we conceptually illustrate that the classifier norm regularization for margin maximization must be explicitly present in information maximizing losses, which otherwise give degenerate clustering.