Domain adaptation performance of a learning algorithm on a target domain is afunction of its source domain error and a divergence measure between the datadistribution of these two domains. We present a study of various distance-basedmeasures in the context of NLP tasks, that characterize the dissimilaritybetween domains based on sample estimates. We first conduct analysisexperiments to show which of these distance measures can best differentiatesamples from same versus different domains, and are correlated with empiricalresults. Next, we develop a DistanceNet model which uses these distancemeasures, or a mixture of these distance measures, as an additional lossfunction to be minimized jointly with the task's loss function, so as toachieve better unsupervised domain adaptation. Finally, we extend this model toa novel DistanceNet-Bandit model, which employs a multi-armed bandit controllerto dynamically switch between multiple source domains and allow the model tolearn an optimal trajectory and mixture of domains for transfer to thelow-resource target domain. We conduct experiments on popular sentimentanalysis datasets with several diverse domains and show that our DistanceNetmodel, as well as its dynamic bandit variant, can outperform competitivebaselines in the context of unsupervised domain adaptation.