lxuechen / Differentially-Private-Deep-Learning

This repo implements several algorithms for learning with differential privacy.

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Differentially-Private-Deep-Learning

This repo provides some example code to help you get started with differentially private deep learning.

Our implementation uses Pytorch. We cover several algorithms including Differentially Private SGD [1], Gradient Embedding Perturbation [2], and Reparametrized Gradient Perturbation [3].

In the vision folder, we train deep ResNets from scratch on benchmark vision datasets such as the CIFAR-10 dataset.

In the language folder, we fine-tune BERT models on four tasks from the GLUE benchrmark.

References

[1]: Deep learning with differential privacy. Martin Abadi, Andy Chu, Ian Goodfellow, H Brendan McMahan, Ilya Mironov, Kunal Talwar, and Li Zhang. In ACM SIGSAC Conference on Computer and Communications Security, 2016.

[2]: Do Not Let Privacy Overbill Utility: Gradient Embedding Perturbation for Private Learning. Da Yu, Huishuai Zhang, Wei Chen, and Tie-Yan Liu. In International Conference on Learning Representations (ICLR), 2021.

[3]: Large Scale Private Learning via Low-rank Reparametrization. Da Yu, Huishuai Zhang, Wei Chen, Jian Yin, and Tie-Yan Liu. In International Conference on Machine Learning (ICML), 2021.

About

This repo implements several algorithms for learning with differential privacy.


Languages

Language:Python 95.1%Language:C++ 2.2%Language:Cuda 1.6%Language:Cython 0.6%Language:Shell 0.4%Language:Batchfile 0.0%Language:Makefile 0.0%Language:Gnuplot 0.0%