Master’s Thesis Presentation • Machine Learning • Bayesian Federated Learning in Predictive Space

Please note: This master’s thesis presentation will take place online.

Mohsin Hasan, Master’s candidate
David R. Cheriton School of Computer Science

Supervisor: Professor Pascal Poupart

Federated Learning (FL) involves training a model over a dataset distributed among clients, with the constraint that each client’s data is private. Within this setting, small and noisy datasets are common, which highlights the need for well-calibrated models which are able to represent the uncertainty in their predictions. Alongside this, two other important goals for a practical FL algorithm are 1) that it has low communication costs, operating over only a few rounds of communication, and 2) that it achieves good performance when client datasets are heterogeneous. Among existing FL techniques, the closest to achieving such goals include Bayesian FL methods which collect parameter samples from local posteriors, and aggregate them to approximate the global posterior (in the parameter or predictive space). These provide uncertainty estimates, more naturally handle data heterogeneity owing to their Bayesian nature, and can operate in a single round of communication.

We demonstrate theoretical issues with existing predictive space techniques which, due to approximation error, give systematically overconfident predictions. We remedy this by proposing β-Predictive Bayes, a Bayesian FL algorithm which performs a modified aggregation of the local predictive posteriors, using a tunable parameter β. β is tuned to improve the global model’s calibration, before it is distilled. We empirically evaluate this method on a number of regression and classification datasets to demonstrate that it is generally better calibrated than other baselines, over a range of heterogeneous data partitions.

To attend this master’s thesis presentation on Zoom, please go to https://vectorinstitute.zoom.us/j/85031194169.