Abstract
Deep convolutional artificial neural networks (ANNs) are the leading class ofcandidate models of the mechanisms of visual processing in the primate ventralstream. While initially inspired by brain anatomy, over the past years, theseANNs have evolved from a simple eight-layer architecture in AlexNet toextremely deep and branching architectures, demonstrating increasingly betterobject categorization performance, yet bringing into question how brain-likethey still are. In particular, typical deep models from the machine learningcommunity are often hard to map onto the brain's anatomy due to their vastnumber of layers and missing biologically-important connections, such asrecurrence. Here we demonstrate that better anatomical alignment to the brainand high performance on machine learning as well as neuroscience measures donot have to be in contradiction. We developed CORnet-S, a shallow ANN with fouranatomically mapped areas and recurrent connectivity, guided by Brain-Score, anew large-scale composite of neural and behavioral benchmarks for quantifyingthe functional fidelity of models of the primate ventral visual stream. Despitebeing significantly shallower than most models, CORnet-S is the top model onBrain-Score and outperforms similarly compact models on ImageNet. Moreover, ourextensive analyses of CORnet-S circuitry variants reveal that recurrence is themain predictive factor of both Brain-Score and ImageNet top-1 performance.Finally, we report that the temporal evolution of the CORnet-S "IT" neuralpopulation resembles the actual monkey IT population dynamics. Taken together,these results establish CORnet-S, a compact, recurrent ANN, as the current bestmodel of the primate ventral visual stream.