Abstract
This study introduces a debiasing method for regression estimators, includinghigh-dimensional and nonparametric regression estimators. For example,nonparametric regression methods allow for the estimation of regressionfunctions in a data-driven manner with minimal assumptions; however, thesemethods typically fail to achieve $\sqrt{n}$-consistency in their convergencerates, and many, including those in machine learning, lack guarantees thattheir estimators asymptotically follow a normal distribution. To address thesechallenges, we propose a debiasing technique for nonparametric estimators byadding a bias-correction term to the original estimators, extending theconventional one-step estimator used in semiparametric analysis. Specifically,for each data point, we estimate the conditional expected residual of theoriginal nonparametric estimator, which can, for instance, be computed usingkernel (Nadaraya-Watson) regression, and incorporate it as a bias-reductionterm. Our theoretical analysis demonstrates that the proposed estimatorachieves $\sqrt{n}$-consistency and asymptotic normality under a mildconvergence rate condition for both the original nonparametric estimator andthe conditional expected residual estimator. Notably, this approach remainsmodel-free as long as the original estimator and the conditional expectedresidual estimator satisfy the convergence rate condition. The proposed methodoffers several advantages, including improved estimation accuracy andsimplified construction of confidence intervals.