Abstract
Data augmentation that introduces diversity into the input data has long beenused in training deep learning models. It has demonstrated benefits inimproving robustness and generalization, practically aligning well with otherregularization strategies such as dropout and weight decay. However, theunderlying mechanism of how diverse training data contributes to modelimprovements remains unknown. In this paper, we investigate the impact of datadiversity on the weight space of deep neural networks using Random MatrixTheory. Through spectral analysis and comparing models trained with dataaugmentation, dropout, and weight decay, we reveal that increasing datadiversity alters the weight spectral distribution similarly to otherregularization techniques, while displaying a pattern more closely aligned withdropout than with weight decay. Building on these insights, we propose a metricto explain and compare the benefits of diversity introduced by traditional dataaugmentations and those achieved through synthetic data.