Abstract
Accurate uncertainty estimation is crucial for deploying neural networks inrisk-sensitive applications such as medical diagnosis. Monte Carlo Dropout is awidely used technique for approximating predictive uncertainty by performingstochastic forward passes with dropout during inference. However, using staticdropout rates across all layers and inputs can lead to suboptimal uncertaintyestimates, as it fails to adapt to the varying characteristics of individualinputs and network layers. Existing approaches optimize dropout rates duringtraining using labeled data, resulting in fixed inference-time parameters thatcannot adjust to new data distributions, compromising uncertainty estimates inMonte Carlo simulations. In this paper, we propose Rate-In, an algorithm that dynamically adjustsdropout rates during inference by quantifying the information loss induced bydropout in each layer's feature maps. By treating dropout as controlled noiseinjection and leveraging information-theoretic principles, Rate-In adaptsdropout rates per layer and per input instance without requiring ground truthlabels. By quantifying the functional information loss in feature maps, weadaptively tune dropout rates to maintain perceptual quality across diversemedical imaging tasks and architectural configurations. Our extensive empiricalstudy on synthetic data and real-world medical imaging tasks demonstrates thatRate-In improves calibration and sharpens uncertainty estimates compared tofixed or heuristic dropout rates without compromising predictive performance.Rate-In offers a practical, unsupervised, inference-time approach to optimizingdropout for more reliable predictive uncertainty estimation in criticalapplications.