Abstract
The statistical supervised learning framework assumes an input-output setwith a joint probability distribution that is reliably represented by thetraining dataset. The learner is then required to output a prediction rulelearned from the training dataset's input-output pairs. In this work, weprovide meaningful insights into the asymptotic equipartition property (AEP)\citep{Shannon:1948} in the context of machine learning, and illuminate some ofits potential ramifications for few-shot learning. We provide theoreticalguarantees for reliable learning under the information-theoretic AEP, and forthe generalization error with respect to the sample size. We then focus on ahighly efficient recurrent neural net (RNN) framework and propose areduced-entropy algorithm for few-shot learning. We also propose a mathematicalintuition for the RNN as an approximation of a sparse coding solver. We verifythe applicability, robustness, and computational efficiency of the proposedapproach with image deblurring and optical coherence tomography (OCT) specklesuppression. Our experimental results demonstrate significant potential forimproving learning models' sample efficiency, generalization, and timecomplexity, that can therefore be leveraged for practical real-timeapplications.