Abstract
Enabling large language models to utilize real-world tools effectively iscrucial for achieving embodied intelligence. Existing approaches to toollearning have either primarily relied on extremely large language models, suchas GPT-4, to attain generalized tool-use abilities in a zero-shot manner, orutilized supervised learning to train limited scopes of tools on compactmodels. However, it remains uncertain whether smaller language models canachieve generalized tool-use abilities without tool-specific training. Toaddress this question, this paper introduces ToolAlpaca, a novel frameworkdesigned to automatically generate a diverse tool-use corpus and learngeneralized tool-use abilities on compact language models with minimal humanintervention. Specifically, ToolAlpaca first automatically creates a highlydiversified tool-use corpus by building a multi-agent simulation environment.The corpus contains 3938 tool-use instances from more than 400 real-world toolAPIs spanning 50 distinct categories. Subsequently, the constructed corpus isemployed to fine-tune compact language models, resulting in two models, namelyToolAlpaca-7B and ToolAlpaca-13B, respectively. Finally, we evaluate theability of these models to utilize previously unseen tools without specifictraining. Experimental results demonstrate that ToolAlpaca achieves effectivegeneralized tool-use capabilities comparable to those of extremely largelanguage models like GPT-3.5, demonstrating that learning generalized tool-useability is feasible for compact language models.