A Unified Statistical Learning Model for Rankings and Scores with Application to Grant Panel Review

  • 2022-06-24 17:49:27
  • Michael Pearce, Elena A. Erosheva
  • 0

Abstract

Rankings and scores are two common data types used by judges to expresspreferences and/or perceptions of quality in a collection of objects. Numerousmodels exist to study data of each type separately, but no unified statisticalmodel captures both data types simultaneously without first performing dataconversion. We propose the Mallows-Binomial model to close this gap, whichcombines a Mallows' $\phi$ ranking model with Binomial score models throughshared parameters that quantify object quality, a consensus ranking, and thelevel of consensus between judges. We propose an efficient tree-searchalgorithm to calculate the exact MLE of model parameters, study statisticalproperties of the model both analytically and through simulation, and apply ourmodel to real data from an instance of grant panel review that collected bothscores and partial rankings. Furthermore, we demonstrate how model outputs canbe used to rank objects with confidence. The proposed model is shown tosensibly combine information from both scores and rankings to quantify objectquality and measure consensus with appropriate levels of statisticaluncertainty.

 

Quick Read (beta)

loading the full paper ...