Abstract
The cornerstone of neural algorithmic reasoning is the ability to solvealgorithmic tasks, especially in a way that generalises out of distribution.While recent years have seen a surge in methodological improvements in thisarea, they mostly focused on building specialist models. Specialist models arecapable of learning to neurally execute either only one algorithm or acollection of algorithms with identical control-flow backbone. Here, instead,we focus on constructing a generalist neural algorithmic learner -- a singlegraph neural network processor capable of learning to execute a wide range ofalgorithms, such as sorting, searching, dynamic programming, path-finding andgeometry. We leverage the CLRS benchmark to empirically show that, much likerecent successes in the domain of perception, generalist algorithmic learnerscan be built by "incorporating" knowledge. That is, it is possible toeffectively learn algorithms in a multi-task manner, so long as we can learn toexecute them well in a single-task regime. Motivated by this, we present aseries of improvements to the input representation, training regime andprocessor architecture over CLRS, improving average single-task performance byover 20% from prior art. We then conduct a thorough ablation of multi-tasklearners leveraging these improvements. Our results demonstrate a generalistlearner that effectively incorporates knowledge captured by specialist models.