Abstract
Modern privacy regulations have spurred the evolution of machine unlearning,a technique that enables the removal of data from an already trained ML modelwithout requiring retraining from scratch. Previous unlearning methods tend toinduce the model to achieve lowest classification accuracy on the removal data.Nonetheless, the authentic objective of machine unlearning is to align theunlearned model with the gold model, i.e., achieving the same classificationaccuracy as the gold model. For this purpose, we present a Twin MachineUnlearning (TMU) approach, where a twin unlearning problem is definedcorresponding to the original unlearning problem. As a results, thegeneralization-label predictor trained on the twin problem can be transferredto the original problem, facilitating aligned data removal. Comprehensiveempirical experiments illustrate that our approach significantly enhances thealignment between the unlearned model and the gold model. Meanwhile, our methodallows data removal without compromising the model accuracy.