Abstract
Modern privacy regulations have spurred the evolution of machine unlearning,a technique enabling a trained model to efficiently forget specific trainingdata. In prior unlearning methods, the concept of "data forgetting" is ofteninterpreted and implemented as achieving zero classification accuracy on suchdata. Nevertheless, the authentic aim of machine unlearning is to achievealignment between the unlearned model and the gold model, i.e., encouragingthem to have identical classification accuracy. On the other hand, the goldmodel often exhibits non-zero classification accuracy due to its generalizationability. To achieve aligned data forgetting, we propose a Twin MachineUnlearning (TMU) approach, where a twin unlearning problem is definedcorresponding to the original unlearning problem. Consequently, thegeneralization-label predictor trained on the twin problem can be transferredto the original problem, facilitating aligned data forgetting. Comprehensiveempirical experiments illustrate that our approach significantly enhances thealignment between the unlearned model and the gold model.