Abstract
We study the application of large language models to zero-shot and few-shotclassification of tabular data. We prompt the large language model with aserialization of the tabular data to a natural-language string, together with ashort description of the classification problem. In the few-shot setting, wefine-tune the large language model using some labeled examples. We evaluateseveral serialization methods including templates, table-to-text models, andlarge language models. Despite its simplicity, we find that this techniqueoutperforms prior deep-learning-based tabular classification methods on severalbenchmark datasets. In most cases, even zero-shot classification obtainsnon-trivial performance, illustrating the method's ability to exploit priorknowledge encoded in large language models. Unlike many deep learning methodsfor tabular datasets, this approach is also competitive with strong traditionalbaselines like gradient-boosted trees, especially in the very-few-shot setting.