Abstract
Large language models (LLMs) exhibit impressive capabilities across a widerange of tasks, yet the choice of which model to use often involves a trade-offbetween performance and cost. More powerful models, though effective, come withhigher expenses, while less capable models are more cost-effective. To addressthis dilemma, we propose several efficient router models that dynamicallyselect between a stronger and a weaker LLM during inference, aiming to optimizethe balance between cost and response quality. We develop a training frameworkfor these routers leveraging human preference data and data augmentationtechniques to enhance performance. Our evaluation on widely-recognizedbenchmarks shows that our approach significantly reduces costs-by over 2 timesin certain cases-without compromising the quality of responses. Interestingly,our router models also demonstrate significant transfer learning capabilities,maintaining their performance even when the strong and weak models are changedat test time. This highlights the potential of these routers to provide acost-effective yet high-performance solution for deploying LLMs.