Abstract
When an experimenter has the option of running an adaptive trial, is itadmissible to ignore this option and run a non-adaptive trial instead? Weprovide a negative answer to this question in the best-arm identificationproblem, where the experimenter aims to allocate measurement effortsjudiciously to confidently deploy the most effective treatment arm. We findthat, whenever there are at least three treatment arms, there exist simpleadaptive designs that universally and strictly dominate non-adaptive completelyrandomized trials. This dominance is characterized by a notion calledefficiency exponent, which quantifies a design's statistical efficiency whenthe experimental sample is large. Our analysis focuses on the class of batchedarm elimination designs, which progressively eliminate underperforming arms atpre-specified batch intervals. We characterize simple sufficient conditionsunder which these designs universally and strictly dominate completelyrandomized trials. These results resolve the second open problem posed in Qin[2022].