In many computer vision applications, obtaining images of high resolution inboth the spatial and spectral domains are equally important. However, due tohardware limitations, one can only expect to acquire images of high resolutionin either the spatial or spectral domains. This paper focuses on hyperspectralimage super-resolution (HSI-SR), where a hyperspectral image (HSI) with lowspatial resolution (LR) but high spectral resolution is fused with amultispectral image (MSI) with high spatial resolution (HR) but low spectralresolution to obtain HR HSI. Existing deep learning-based solutions are allsupervised that would need a large training set and the availability of HR HSI,which is unrealistic. Here, we make the first attempt to solving the HSI-SRproblem using an unsupervised encoder-decoder architecture that carries thefollowing uniquenesses. First, it is composed of two encoder-decoder networks,coupled through a shared decoder, in order to preserve the rich spectralinformation from the HSI network. Second, the network encourages therepresentations from both modalities to follow a sparse Dirichlet distributionwhich naturally incorporates the two physical constraints of HSI and MSI.Third, the angular difference between representations are minimized in order toreduce the spectral distortion. We refer to the proposed architecture asunsupervised Sparse Dirichlet-Net, or uSDN. Extensive experimental resultsdemonstrate the superior performance of uSDN as compared to thestate-of-the-art.