Abstract
Early-exit neural networks reduce inference cost by enabling confidentpredictions at intermediate layers. However, joint training often leads togradient interference, with deeper classifiers dominating optimization. Wepropose Confidence-Gated Training (CGT), a paradigm that conditionallypropagates gradients from deeper exits only when preceding exits fail. Thisencourages shallow classifiers to act as primary decision points whilereserving deeper layers for harder inputs. By aligning training with theinference-time policy, CGT mitigates overthinking, improves early-exitaccuracy, and preserves efficiency. Experiments on the Indian Pines andFashion-MNIST benchmarks show that CGT lowers average inference cost whileimproving overall accuracy, offering a practical solution for deploying deepmodels in resource-constrained environments.