Abstract
Local learning offers an alternative to traditional end-to-endback-propagation in deep neural networks, significantly reducing GPU memoryusage. While local learning has shown promise in image classification tasks,its application to other visual tasks remains limited. This limitation arisesprimarily from two factors: 1) architectures tailored for classification areoften not transferable to other tasks, leading to a lack of reusability oftask-specific knowledge; 2) the absence of cross-scale feature communicationresults in degraded performance in tasks such as object detection andsuper-resolution. To address these challenges, we propose the Memory-augmentedAuxiliary Network (MAN), which introduces a simplified design principle andincorporates a feature bank to enhance cross-task adaptability andcommunication. This work represents the first successful application of locallearning methods beyond classification, demonstrating that MAN not onlyconserves GPU memory but also achieves performance on par with end-to-endapproaches across multiple datasets for various visual tasks.