Abstract
Modern approach to artificial intelligence (AI) aims to design algorithmsthat learn directly from data. This approach has achieved impressive resultsand has contributed significantly to the progress of AI, particularly in thesphere of supervised deep learning. It has also simplified the design ofmachine learning systems as the learning process is highly automated. However,not all data processing tasks in conventional deep learning pipelines have beenautomated. In most cases data has to be manually collected, preprocessed andfurther extended through data augmentation before they can be effective fortraining. Recently, special techniques for automating these tasks have emerged.The automation of data processing tasks is driven by the need to utilize largevolumes of complex, heterogeneous data for machine learning and big dataapplications. Today, end-to-end automated data processing systems based onautomated machine learning (AutoML) techniques are capable of taking raw dataand transforming them into useful features for Big Data tasks by automating allintermediate processing stages. In this work, we present a thorough review ofapproaches for automating data processing tasks in deep learning pipelines,including automated data preprocessing--e.g., data cleaning, labeling, missingdata imputation, and categorical data encoding--as well as data augmentation(including synthetic data generation using generative AI methods) and featureengineering--specifically, automated feature extraction, feature constructionand feature selection. In addition to automating specific data processingtasks, we discuss the use of AutoML methods and tools to simultaneouslyoptimize all stages of the machine learning pipeline.