Abstract
Tabular foundational models have exhibited strong in-context learning (ICL)capabilities on structured data, allowing them to make accurate predictions ontest sets without parameter updates, using training examples as context. Thisemerging approach positions itself as a competitive alternative to traditionalgradient-boosted tree methods. However, while biases in conventional machinelearning models are well documented, it remains unclear how these biasesmanifest in tabular ICL. The paper investigates the fairness implications oftabular ICL and explores three preprocessing strategies--correlation removal,group-balanced demonstration selection, and uncertainty-based demonstrationselection--to address bias. Comprehensive experiments indicate thatuncertainty-based demonstration selection consistently enhances group fairnessof in-context predictions. The source code for reproducing the results of thiswork can be found at https://github.com/patrikken/Fair-TabICL.