Abstract
To mitigate unfair and unethical discrimination over sensitive features(e.g., gender, age, or race), fairness testing plays an integral role inengineering systems that leverage AI models to handle tabular data. A keychallenge therein is how to effectively reveal fairness bugs under anintractable sample size using perturbation. Much current work has been focusingon designing the test sample generators, ignoring the valuable knowledge aboutdata characteristics that can help guide the perturbation and hence limitingtheir full potential. In this paper, we seek to bridge such a gap by proposinga generic framework of causally perturbed fairness testing, dubbed CausalFT.Through causal inference, the key idea of CausalFT is to extract the mostdirectly and causally relevant non-sensitive feature to its sensitivecounterpart, which can jointly influence the prediction of the label. Such acausal relationship is then seamlessly injected into the perturbation to guidea test sample generator. Unlike existing generator-level work, CausalFT servesas a higher-level framework that can be paired with diverse base generators.Extensive experiments on 1296 cases confirm that CausalFT can considerablyimprove arbitrary base generators in revealing fairness bugs over 93% of thecases with acceptable extra runtime overhead. Compared with a state-of-the-artapproach that ranks the non-sensitive features solely based on correlation,CausalFT performs significantly better on 64% cases while being much moreefficient. Further, CausalFT can better improve bias resilience in nearly allcases.