Abstract
AI has long pursued the goal of having systems reason over *explicitlyprovided* knowledge, but building suitable representations has provedchallenging. Here we explore whether transformers can similarly learn to reason(or emulate reasoning), but using rules expressed in language, thus bypassing aformal representation. We provide the first demonstration that this ispossible, and characterize the extent of this capability. To do this, we use acollection of synthetic datasets that test increasing levels of reasoningcomplexity (number of rules, presence of negation, and depth of chaining). Wefind transformers appear to learn rule-based reasoning with high (99%) accuracyon these datasets, and in a way that generalizes to test data requiringsubstantially deeper chaining than in the training data (95%+ scores). We alsodemonstrate that the models transfer well to two hand-authored rulebases, andto rulebases paraphrased into more natural language. These findings aresignificant as it suggests a new role for transformers, namely as a limited"soft theorem prover" operating over explicit theories in language. This inturn suggests new possibilities for explainability, correctability, andcounterfactual reasoning in question-answering. All datasets and a live demoare available at http://rule-reasoning.apps.allenai.org/