Abstract
Discussions of algorithmic bias tend to focus on examples where either thedata or the people building the algorithms are biased. This gives theimpression that clean data and good intentions could eliminate bias. Theneutrality of the algorithms themselves is defended by prominent ArtificialIntelligence researchers. However, algorithms are not neutral. In addition tobiased data and biased algorithm makers, AI algorithms themselves can bebiased. This is illustrated with the example of collaborative filtering, whichis known to suffer from popularity, and homogenizing biases. Iterativeinformation filtering algorithms in general create a selection bias in thecourse of learning from user responses to documents that the algorithmrecommended. These are not merely biases in the statistical sense; thesestatistical biases can cause discriminatory outcomes. Data points on themargins of distributions of human data tend to correspond to marginalizedpeople. Popularity and homogenizing biases have the effect of furthermarginalizing the already marginal. This source of bias warrants seriousattention given the ubiquity of algorithmic decision-making.