Abstract
Many applications of AI involve scoring individuals using a learned functionof their attributes. These predictive risk scores are then used to takedecisions based on whether the score exceeds a certain threshold, which mayvary depending on the context. The level of delegation granted to such systemsin critical applications like credit lending and medical diagnosis will heavilydepend on how questions of fairness can be answered. In this paper, we studyfairness for the problem of learning scoring functions from binary labeleddata, a classic learning task known as bipartite ranking. We argue that thefunctional nature of the ROC curve, the gold standard measure of rankingaccuracy in this context, leads to several ways of formulating fairnessconstraints. We introduce general families of fairness definitions based on theAUC and on ROC curves, and show that our ROC-based constraints can beinstantiated such that classifiers obtained by thresholding the scoringfunction satisfy classification fairness for a desired range of thresholds. Weestablish generalization bounds for scoring functions learned under suchconstraints, design practical learning algorithms and show the relevance ourapproach with numerical experiments on real and synthetic data.