Abstract
Stochastic gradient descent requires that training samples be drawn from auniformly random distribution of the data. For a deployed system that mustlearn online from an uncontrolled and unknown environment, the ordering ofinput samples often fails to meet this criterion, making lifelong learning adifficult challenge. We exploit the locality of the unsupervised Spike TimingDependent Plasticity (STDP) learning rule to target local representations in aSpiking Neural Network (SNN) to adapt to novel information while protectingessential information in the remainder of the SNN from catastrophic forgetting.In our Controlled Forgetting Networks (CFNs), novel information triggersstimulated firing and heterogeneously modulated plasticity, inspired bybiological dopamine signals, to cause rapid and isolated adaptation in thesynapses of neurons associated with outlier information. This targetingcontrols the forgetting process in a way that reduces the degradation ofaccuracy for older tasks while learning new tasks. Our experimental results onthe MNIST dataset validate the capability of CFNs to learn successfully overtime from an unknown, changing environment, achieving 95.36% accuracy, which webelieve is the best unsupervised accuracy ever achieved by a fixed-size,single-layer SNN on a completely disjoint MNIST dataset.