PeFLL: A Lifelong Learning Approach to Personalized Federated Learning

by   Jonathan Scott, et al.

Personalized federated learning (pFL) has emerged as a popular approach to dealing with the challenge of statistical heterogeneity between the data distributions of the participating clients. Instead of learning a single global model, pFL aims to learn an individual model for each client while still making use of the data available at other clients. In this work, we present PeFLL, a new pFL approach rooted in lifelong learning that performs well not only on clients present during its training phase, but also on any that may emerge in the future. PeFLL learns to output client specific models by jointly training an embedding network and a hypernetwork. The embedding network learns to represent clients in a latent descriptor space in a way that reflects their similarity to each other. The hypernetwork learns a mapping from this latent space to the space of possible client models. We demonstrate experimentally that PeFLL produces models of superior accuracy compared to previous methods, especially for clients not seen during training, and that it scales well to large numbers of clients. Moreover, generating a personalized model for a new client is efficient as no additional fine-tuning or optimization is required by either the client or the server. We also present theoretical results supporting PeFLL in the form of a new PAC-Bayesian generalization bound for lifelong learning and we prove the convergence of our proposed optimization procedure.


page 1

page 2

page 3

page 4


Personalized Federated Learning using Hypernetworks

Personalized federated learning is tasked with training machine learning...

Personalized Federated Learning with Gaussian Processes

Federated learning aims to learn a global model that performs well on cl...

Intent Detection at Scale: Tuning a Generic Model using Relevant Intents

Accurately predicting the intent of customer support requests is vital f...

NLP Service APIs and Models for Efficient Registration of New Clients

State-of-the-art NLP inference uses enormous neural architectures and mo...

Federated Learning of Shareable Bases for Personalization-Friendly Image Classification

Personalized federated learning (PFL) aims to harness the collective wis...

Analyzing Client Behavior in a Syringe Exchange Program

Multiple syringe exchange programs serve the Chicago metropolitan area, ...

FedTP: Federated Learning by Transformer Personalization

Federated learning is an emerging learning paradigm where multiple clien...

Please sign up or login with your details

Forgot password? Click here to reset