Abstract

The backpropagation (BP) algorithm is often thought to be biologicallyimplausible in the brain. One of the main reasons is that BP requires symmetricweight matrices in the feedforward and feedback pathways. To address this"weight transport problem" (Grossberg, 1987), two more biologically plausiblealgorithms, proposed by Liao et al. (2016) and Lillicrap et al. (2016), relaxBP's weight symmetry requirements and demonstrate comparable learningcapabilities to that of BP on small datasets. However, a recent study byBartunov et al. (2018) evaluate variants of target-propagation (TP) andfeedback alignment (FA) on MINIST, CIFAR, and ImageNet datasets, and find thatalthough many of the proposed algorithms perform well on MNIST and CIFAR, theyperform significantly worse than BP on ImageNet. Here, we additionally evaluatethe sign-symmetry algorithm (Liao et al., 2016), which differs from both BP andFA in that the feedback and feedforward weights share signs but not magnitudes.We examine the performance of sign-symmetry and feedback alignment on ImageNetand MS COCO datasets using different network architectures (ResNet-18 andAlexNet for ImageNet, RetinaNet for MS COCO). Surprisingly, networks trainedwith sign-symmetry can attain classification performance approaching that ofBP-trained networks. These results complement the study by Bartunov et al.(2018), and establish a new benchmark for future biologically plausiblelearning algorithms on more difficult datasets and more complex architectures.

 

Introduction (beta)

None

 

Conclusion (beta)

None