Abstract
In this work, we formally study the membership privacy risk of generativemodels and propose a membership privacy estimation framework. We formulate themembership privacy risk as a statistical divergence between training samplesand hold-out samples, and propose sample-based methods to estimate thisdivergence. Unlike previous works, our proposed metric and estimators makerealistic and flexible assumptions. First, we offer a generalizable metric asan alternative to accuracy for imbalanced datasets. Second, our estimators arecapable of estimating the membership privacy risk given any scalar or vectorvalued attributes from the learned model, while prior work require access tospecific attributes. This allows our framework to provide data-drivencertificates for trained generative models in terms of membership privacy risk.Finally, we show a connection to differential privacy, which allows ourproposed estimators to be used to understand the privacy budget 'epsilon'needed for differentially private generative models. We demonstrate the utilityof our framework through experimental demonstrations on different generativemodels using various model attributes yielding some new insights aboutmembership leakage and vulnerabilities of models.