Abstract
Machine learning methods often assume that input features are available at nocost. However, in domains like healthcare, where acquiring features could beexpensive or harmful, it is necessary to balance a feature's acquisition costagainst its predictive value. The task of training an AI agent to decide whichfeatures to acquire is called active feature acquisition (AFA). By deploying anAFA agent, we effectively alter the acquisition strategy and trigger adistribution shift. To safely deploy AFA agents under this distribution shift,we present the problem of active feature acquisition performance evaluation(AFAPE). We examine AFAPE under i) a no direct effect (NDE) assumption, statingthat acquisitions do not affect the underlying feature values; and ii) a nounobserved confounding (NUC) assumption, stating that retrospective featureacquisition decisions were only based on observed features. We show that onecan apply missing data methods under the NDE assumption and offlinereinforcement learning under the NUC assumption. When NUC and NDE hold, wepropose a novel semi-offline reinforcement learning framework. This frameworkrequires a weaker positivity assumption and introduces three new estimators: Adirect method (DM), an inverse probability weighting (IPW), and a doublereinforcement learning (DRL) estimator.