Privacy-preserving Decentralized Deep Learning with Multiparty Homomorphic Encryption

by   Guowen Xu, et al.

Decentralized deep learning plays a key role in collaborative model training due to its attractive properties, including tolerating high network latency and less prone to single-point failures. Unfortunately, such a training mode is more vulnerable to data privacy leaks compared to other distributed training frameworks. Existing efforts exclusively use differential privacy as the cornerstone to alleviate the data privacy threat. However, it is still not clear whether differential privacy can provide a satisfactory utility-privacy trade-off for model training, due to its inherent contradictions. To address this problem, we propose D-MHE, the first secure and efficient decentralized training framework with lossless precision. Inspired by the latest developments in the homomorphic encryption technology, we design a multiparty version of Brakerski-Fan-Vercauteren (BFV), one of the most advanced cryptosystems, and use it to implement private gradient updates of users'local models. D-MHE can reduce the communication complexity of general Secure Multiparty Computation (MPC) tasks from quadratic to linear in the number of users, making it very suitable and scalable for large-scale decentralized learning systems. Moreover, D-MHE provides strict semantic security protection even if the majority of users are dishonest with collusion. We conduct extensive experiments on MNIST and CIFAR-10 datasets to demonstrate the superiority of D-MHE in terms of model accuracy, computation and communication cost compared with existing schemes.


page 1

page 13


Scotch: An Efficient Secure Computation Framework for Secure Aggregation

Federated learning enables multiple data owners to jointly train a machi...

LEASGD: an Efficient and Privacy-Preserving Decentralized Algorithm for Distributed Learning

Distributed learning systems have enabled training large-scale models ov...

Training Differentially Private Models with Secure Multiparty Computation

We address the problem of learning a machine learning model from trainin...

Efficient Logistic Regression with Local Differential Privacy

Internet of Things devices are expanding rapidly and generating huge amo...

Decentralized Graph Neural Network for Privacy-Preserving Recommendation

Building a graph neural network (GNN)-based recommender system without v...

Linear Model with Local Differential Privacy

Scientific collaborations benefit from collaborative learning of distrib...

Collusion Attacks on Decentralized Attributed-Based Encryption: Analyses and a Solution

Attribute-based Encryption (ABE) is an information centric security solu...

Please sign up or login with your details

Forgot password? Click here to reset