Abstract
Modern neural networks (NNs) often do not generalize well in the presence ofa "covariate shift"; that is, in situations where the training and test datadistributions differ, but the conditional distribution of classification labelsremains unchanged. In such cases, NN generalization can be reduced to a problemof learning more domain-invariant features. Domain adaptation (DA) methodsinclude a range of techniques aimed at achieving this; however, these methodshave struggled with the need for extensive hyperparameter tuning, which thenincurs significant computational costs. In this work, we introduce SIDDA, anout-of-the-box DA training algorithm built upon the Sinkhorn divergence, thatcan achieve effective domain alignment with minimal hyperparameter tuning andcomputational overhead. We demonstrate the efficacy of our method on multiplesimulated and real datasets of varying complexity, including simple shapes,handwritten digits, and real astronomical observations. SIDDA is compatiblewith a variety of NN architectures, and it works particularly well in improvingclassification accuracy and model calibration when paired with equivariantneural networks (ENNs). We find that SIDDA enhances the generalizationcapabilities of NNs, achieving up to a $\approx40\%$ improvement inclassification accuracy on unlabeled target data. We also study the efficacy ofDA on ENNs with respect to the varying group orders of the dihedral group$D_N$, and find that the model performance improves as the degree ofequivariance increases. Finally, we find that SIDDA enhances model calibrationon both source and target data--achieving over an order of magnitudeimprovement in the ECE and Brier score. SIDDA's versatility, combined with itsautomated approach to domain alignment, has the potential to advancemulti-dataset studies by enabling the development of highly generalizablemodels.