Abstract
In supervised learning, smoothing label or prediction distribution in neuralnetwork training has been proven useful in preventing the model from beingover-confident, and is crucial for learning more robust visual representations.This observation motivates us to explore ways to make predictions flattened inunsupervised learning. Considering that human-annotated labels are not adoptedin unsupervised learning, we introduce a straightforward approach to perturbinput image space in order to soften the output prediction space indirectly,meanwhile, assigning new label values in the unsupervised frameworksaccordingly. Despite its conceptual simplicity, we show empirically that withthe simple solution -- Unsupervised image mixtures (Un-Mix), we can learn morerobust visual representations from the transformed input. Extensive experimentsare conducted on CIFAR-10, CIFAR-100, STL-10, Tiny ImageNet and standardImageNet with popular unsupervised methods SimCLR, BYOL, MoCo V1&V2, etc. Ourproposed image mixture and label assignment strategy can obtain consistentimprovement by 1~3% following exactly the same hyperparameters and trainingprocedures of the base methods.