Abstract
With the increasing applications of language models, it has become crucial toprotect these models from leaking private information. Previous work hasattempted to tackle this challenge by training RNN-based language models withdifferential privacy guarantees. However, applying classical differentialprivacy to language models leads to poor model performance as the underlyingprivacy notion is over-pessimistic and provides undifferentiated protection forall tokens in the data. Given that the private information in natural languageis sparse (for example, the bulk of an email might not carry personallyidentifiable information), we propose a new privacy notion, selectivedifferential privacy, to provide rigorous privacy guarantees on the sensitiveportion of the data to improve model utility. To realize such a new notion, wedevelop a corresponding privacy mechanism, Selective-DPSGD, for RNN-basedlanguage models. Besides language modeling, we also apply the method to a moreconcrete application--dialog systems. Experiments on both language modeling anddialog system building show that the proposed privacy-preserving mechanismachieves better utilities while remaining safe under various privacy attackscompared to the baselines. The data and code are released athttps://github.com/wyshi/lm_privacy to facilitate future research .