Confidence-aware Personalized Federated Learning via Variational Expectation Maximization

by   Junyi Zhu, et al.

Federated Learning (FL) is a distributed learning scheme to train a shared model across clients. One common and fundamental challenge in FL is that the sets of data across clients could be non-identically distributed and have different sizes. Personalized Federated Learning (PFL) attempts to solve this challenge via locally adapted models. In this work, we present a novel framework for PFL based on hierarchical Bayesian modeling and variational inference. A global model is introduced as a latent variable to augment the joint distribution of clients' parameters and capture the common trends of different clients, optimization is derived based on the principle of maximizing the marginal likelihood and conducted using variational expectation maximization. Our algorithm gives rise to a closed-form estimation of a confidence value which comprises the uncertainty of clients' parameters and local model deviations from the global model. The confidence value is used to weigh clients' parameters in the aggregation stage and adjust the regularization effect of the global model. We evaluate our method through extensive empirical studies on multiple datasets. Experimental results show that our approach obtains competitive results under mild heterogeneous circumstances while significantly outperforming state-of-the-art PFL frameworks in highly heterogeneous settings. Our code is available at


CRFL: Certifiably Robust Federated Learning against Backdoor Attacks

Federated Learning (FL) as a distributed learning paradigm that aggregat...

SPIDER: Searching Personalized Neural Architecture for Federated Learning

Federated learning (FL) is an efficient learning framework that assists ...

FLIS: Clustered Federated Learning via Inference Similarity for Non-IID Data Distribution

Classical federated learning approaches yield significant performance de...

FedGroup: Ternary Cosine Similarity-based Clustered Federated Learning Framework toward High Accuracy in Heterogeneous Data

Federated Learning (FL) enables the multiple participating devices to co...

EM for Mixture of Linear Regression with Clustered Data

Modern data-driven and distributed learning frameworks deal with diverse...

Personalized Federated Learning under Mixture of Distributions

The recent trend towards Personalized Federated Learning (PFL) has garne...

Please sign up or login with your details

Forgot password? Click here to reset