Abstract
Deep neural networks (DNNs) have demonstrated superior performance overclassical machine learning to support many features in safety-critical systems.Although DNNs are now widely used in such systems (e.g., self driving cars),there is limited progress regarding automated support for functional safetyanalysis in DNN-based systems. For example, the identification of root causesof errors, to enable both risk analysis and DNN retraining, remains an openproblem. In this paper, we propose SAFE, a black-box approach to automaticallycharacterize the root causes of DNN errors. SAFE relies on a transfer learningmodel pre-trained on ImageNet to extract the features from error-inducingimages. It then applies a density-based clustering algorithm to detectarbitrary shaped clusters of images modeling plausible causes of error. Last,clusters are used to effectively retrain and improve the DNN. The black-boxnature of SAFE is motivated by our objective not to require changes or evenaccess to the DNN internals to facilitate adoption. Experimental results show the superior ability of SAFE in identifyingdifferent root causes of DNN errors based on case studies in the automotivedomain. It also yields significant improvements in DNN accuracy afterretraining, while saving significant execution time and memory when compared toalternatives.