Abstract
The Forward-Forward algorithm is an alternative learning method whichconsists of two forward passes rather than a forward and backward pass employedby backpropagation. Forward-Forward networks employ layer local loss functionswhich are optimized based on the layer activation for each forward pass ratherthan a single global objective function. This work explores the dynamics ofmodel and layer accuracy changes in Forward-Forward networks as trainingprogresses in pursuit of a mechanistic understanding of their internalbehavior. Treatments to various system characteristics are applied toinvestigate changes in layer and overall model accuracy as training progresses,how accuracy is impacted by layer depth, and how strongly individual layeraccuracy is correlated with overall model accuracy. The empirical resultspresented suggest that layers deeper within Forward-Forward networks experiencea delay in accuracy improvement relative to shallower layers and that shallowerlayer accuracy is strongly correlated with overall model accuracy.