Life After Bootstrap: Residual Randomization Inference in Regression Models

  • 2019-08-12 16:09:15
  • Panos Toulis
  • 8

Abstract

We develop a randomization-based method for inference in regression models.The basis of inference is an invariance assumption on the regression errors,such as invariance to permutations or random signs. To test significance, therandomization method repeatedly calculates a suitable test statistic overtransformations of the regression residuals according to the invariant.Inversion of the test can produce confidence intervals. We prove generalconditions for asymptotic validity of this residual randomization test andillustrate in many models, including clustered errors with one-way or two-wayclustering structure. We also show that finite-sample validity is possibleunder a suitable construction, and illustrate with an exact test for a case ofthe Behrens-Fisher problem. The proposed method offers four main advantagesover the bootstrap: (1) it addresses the inference problem in a unified way,while bootstrap typically needs to be adapted to the task; (2) it can be morepowerful by exploiting a richer and more flexible set of invariances thanexchangeability; (3) it does not rely on asymptotic normality; and (4) it canbe valid in finite samples. In extensive empirical evaluations, including highdimensional regression and autocorrelated errors, the proposed method performsfavorably against many alternatives, including bootstrap variants andasymptotic robust error methods.

 

Quick Read (beta)

loading the full paper ...