Abstract
Motivated by the growing demand for low-precision arithmetic in computationalscience, we exploit lower-precision emulation in Python -- widely regarded asthe dominant programming language for numerical analysis and machine learning.Low-precision training has revolutionized deep learning by enabling moreefficient computation and reduced memory and energy consumption whilemaintaining model fidelity. To better enable numerical experimentation with andexploration of low precision computation, we developed the Pychop library,which supports customizable floating-point formats and a comprehensive set ofrounding modes in Python, allowing users to benefit from fast, low-precisionemulation in numerous applications. Pychop also introduces interfaces for bothPyTorch and JAX, enabling efficient low-precision emulation on GPUs for neuralnetwork training and inference with unparalleled flexibility. In this paper, we offer a comprehensive exposition of the design,implementation, validation, and practical application of Pychop, establishingit as a foundational tool for advancing efficient mixed-precision algorithms.Furthermore, we present empirical results on low-precision emulation for imageclassification and object detection using published datasets, illustrating thesensitivity of the use of low precision and offering valuable insights into itsimpact. Pychop enables in-depth investigations into the effects of numericalprecision, facilitates the development of novel hardware accelerators, andintegrates seamlessly into existing deep learning workflows. Software andexperimental code are publicly available athttps://github.com/inEXASCALE/pychop.