Abstract
In many real-world scenarios, acquiring all features of a data instance canbe expensive or impractical due to monetary cost, latency, or privacy concerns.Active Feature Acquisition (AFA) addresses this challenge by dynamicallyselecting a subset of informative features for each data instance, tradingpredictive performance against acquisition cost. While numerous methods havebeen proposed for AFA, ranging from greedy information-theoretic strategies tonon-myopic reinforcement learning approaches, fair and systematic evaluation ofthese methods has been hindered by the lack of standardized benchmarks. In thispaper, we introduce AFABench, the first benchmark framework for AFA. Ourbenchmark includes a diverse set of synthetic and real-world datasets, supportsa wide range of acquisition policies, and provides a modular design thatenables easy integration of new methods and tasks. We implement and evaluaterepresentative algorithms from all major categories, including static, greedy,and reinforcement learning-based approaches. To test the lookahead capabilitiesof AFA policies, we introduce a novel synthetic dataset, AFAContext, designedto expose the limitations of greedy selection. Our results highlight keytrade-offs between different AFA strategies and provide actionable insights forfuture research. The benchmark code is available at:https://github.com/Linusaronsson/AFA-Benchmark.