Abstract
The input data pipeline is an essential component of each machine learning(ML) training job. It is responsible for reading massive amounts of trainingdata, processing batches of samples using complex transformations, and loadingthem onto training nodes at low latency and high throughput. Performant inputdata systems are becoming increasingly critical, driven by skyrocketing datavolumes and training throughput demands. Unfortunately, current input datasystems cannot fully leverage key performance optimizations, resulting inhugely inefficient infrastructures that require significant resources - orworse - underutilize expensive accelerators. To address these demands, we present cedar, an optimized and unifiedprogramming framework for ML input data pipelines. cedar allows users to defineinput data pipelines using composable operators that support arbitrary MLframeworks and libraries. cedar introduces an extensible optimizer thatsystematically applies a complex combination of optimizations (e.g.,offloading, caching, prefetching, fusion, and reordering). It orchestratesprocessing across a customizable set of local and distributed compute resourcesin order to improve processing performance and efficiency, all without userinput. Across eight pipelines, cedar improves performance by up to 1.87x to10.65x compared to state-of-the-art input data systems.