Abstract

Instead of querying LLMs in a one-shot manner and hoping to get the rightanswer for a reasoning task, we propose a paradigm we call \emph{verbalizedalgorithms} (VAs), which leverage classical algorithms with establishedtheoretical understanding. VAs decompose a task into simple elementaryoperations on natural language strings that they should be able to answerreliably, and limit the scope of LLMs to only those simple tasks. For example,for sorting a series of natural language strings, \emph{verbalized sorting}uses an LLM as a binary comparison oracle in a known and well-analyzed sortingalgorithm (e.g., bitonic sorting network). We demonstrate the effectiveness ofthis approach on sorting and clustering tasks.

 

Quick Read (beta)