Abstract
Unlike humans, who are capable of continual learning over their lifetimes,artificial neural networks have long been known to suffer from a phenomenonknown as catastrophic forgetting, whereby new learning can lead to abrupterasure of previously acquired knowledge. Whereas in a neural network theparameters are typically modelled as scalar values, an individual synapse inthe brain comprises a complex network of interacting biochemical componentsthat evolve at different timescales. In this paper, we show that by equippingtabular and deep reinforcement learning agents with a synaptic model thatincorporates this biological complexity (Benna & Fusi, 2016), catastrophicforgetting can be mitigated at multiple timescales. In particular, we find thatas well as enabling continual learning across sequential training of two simpletasks, it can also be used to overcome within-task forgetting by reducing theneed for an experience replay database.