Abstract
Recommender systems (RecSys) have become an essential component of many webapplications. The core of the system is a recommendation model trained onhighly sensitive user-item interaction data. While privacy-enhancing techniquesare actively studied in the research community, the real-world modeldevelopment still depends on minimal privacy protection, e.g., via controlledaccess. Users of such systems should have the right to choose \emph{not} toshare highly sensitive interactions. However, there is no method allowing theuser to know which interactions are more sensitive than others. Thus,quantifying the privacy risk of RecSys training data is a critical step toenabling privacy-aware RecSys model development and deployment. We propose amembership-inference attack (MIA)- based privacy scoring method, RecPS, tomeasure privacy risks at both the interaction and user levels. The RecPSinteraction-level score definition is motivated and derived from differentialprivacy, which is then extended to the user-level scoring method. A criticalcomponent is the interaction-level MIA method RecLiRA, which gives high-qualitymembership estimation. We have conducted extensive experiments on well-knownbenchmark datasets and RecSys models to show the unique features and benefitsof RecPS scoring in risk assessment and RecSys model unlearning.