Abstract
Rule-based reasoning, a fundamental type of legal reasoning, enables us todraw conclusions by accurately applying a rule to a set of facts. We explorecausal language models as rule-based reasoners, specifically with respect tocompositional rules - rules consisting of multiple elements which form acomplex logical expression. Reasoning about compositional rules is challengingbecause it requires multiple reasoning steps, and attending to the logicalrelationships between elements. We introduce a new prompting method, Chain ofLogic, which elicits rule-based reasoning through decomposition (solvingelements as independent threads of logic), and recomposition (recombining thesesub-answers to resolve the underlying logical expression). This method wasinspired by the IRAC (Issue, Rule, Application, Conclusion) framework, asequential reasoning approach used by lawyers. We evaluate chain of logicacross eight rule-based reasoning tasks involving three distinct compositionalrules from the LegalBench benchmark and demonstrate it consistently outperformsother prompting methods, including chain of thought and self-ask, usingopen-source and commercial language models.