RQP-SGD: Differential Private Machine Learning through Noisy SGD and Randomized Quantization

  • 2024-02-09 18:34:08
  • Ce Feng, Parv Venkitasubramaniam
  • 0

Abstract

The rise of IoT devices has prompted the demand for deploying machinelearning at-the-edge with real-time, efficient, and secure data processing. Inthis context, implementing machine learning (ML) models with real-valued weightparameters can prove to be impractical particularly for large models, and thereis a need to train models with quantized discrete weights. At the same time,these low-dimensional models also need to preserve privacy of the underlyingdataset. In this work, we present RQP-SGD, a new approach forprivacy-preserving quantization to train machine learning models for low-memoryML-at-the-edge. This approach combines differentially private stochasticgradient descent (DP-SGD) with randomized quantization, providing a measurableprivacy guarantee in machine learning. In particular, we study the utilityconvergence of implementing RQP-SGD on ML tasks with convex objectives andquantization constraints and demonstrate its efficacy over deterministicquantization. Through experiments conducted on two datasets, we show thepractical effectiveness of RQP-SGD.

 

Quick Read (beta)

loading the full paper ...