Abstract
This paper illustrates the central role of loss functions in data-drivendecision making, providing a comprehensive survey on their influence incost-sensitive classification (CSC) and reinforcement learning (RL). Wedemonstrate how different regression loss functions affect the sampleefficiency and adaptivity of value-based decision making algorithms. Acrossmultiple settings, we prove that algorithms using the binary cross-entropy lossachieve first-order bounds scaling with the optimal policy's cost and are muchmore efficient than the commonly used squared loss. Moreover, we prove thatdistributional algorithms using the maximum likelihood loss achievesecond-order bounds scaling with the policy variance and are even sharper thanfirst-order bounds. This in particular proves the benefits of distributionalRL. We hope that this paper serves as a guide analyzing decision makingalgorithms with varying loss functions, and can inspire the reader to seek outbetter loss functions to improve any decision making algorithm.