Abstract
Deep neural networks often learn and rely on spurious correlations, i.e.,superficial associations between non-causal features and the targets. Forinstance, an image classifier may identify camels based on the desertbackgrounds. While it can yield high overall accuracy during training, itdegrades generalization on more diverse scenarios where such correlations donot hold. This problem poses significant challenges for out-of-distributionrobustness and trustworthiness. Existing methods typically mitigate this issueby using external group annotations or auxiliary deterministic models to learnunbiased representations. However, such information is costly to obtain, anddeterministic models may fail to capture the full spectrum of biases learned bythe models. To address these limitations, we propose Evidential Alignment, anovel framework that leverages uncertainty quantification to understand thebehavior of the biased models without requiring group annotations. Byquantifying the evidence of model prediction with second-order riskminimization and calibrating the biased models with the proposed evidentialcalibration technique, Evidential Alignment identifies and suppresses spuriouscorrelations while preserving core features. We theoretically justify theeffectiveness of our method as capable of learning the patterns of biasedmodels and debiasing the model without requiring any spurious correlationannotations. Empirical results demonstrate that our method significantlyimproves group robustness across diverse architectures and data modalities,providing a scalable and principled solution to spurious correlations.