Abstract
Despite the tremendous advances in machine learning (ML), training withimbalanced data still poses challenges in many real-world applications. Among aseries of diverse techniques to solve this problem, sampling algorithms areregarded as an efficient solution. However, the problem is more fundamental,with many works emphasizing the importance of instance hardness. This issuerefers to the significance of managing unsafe or potentially noisy instancesthat are more likely to be misclassified and serve as the root cause of poorclassification performance. This paper introduces HardVis, a visual analyticssystem designed to handle instance hardness mainly in imbalanced classificationscenarios. Our proposed system assists users in visually comparing differentdistributions of data types, selecting types of instances based on localcharacteristics that will later be affected by the active sampling method, andvalidating which suggestions from undersampling or oversampling techniques arebeneficial for the ML model. Additionally, rather than uniformlyundersampling/oversampling a specific class, we allow users to find and sampleeasy and difficult to classify training instances from all classes. Users canexplore subsets of data from different perspectives to decide all thoseparameters, while HardVis keeps track of their steps and evaluates the model'spredictive performance in a test set separately. The end result is awell-balanced data set that boosts the predictive power of the ML model. Theefficacy and effectiveness of HardVis are demonstrated with a hypotheticalusage scenario and a use case. Finally, we also look at how useful our systemis based on feedback we received from ML experts.