Abstract
The recent development of language models has shown promising results byachieving state-of-the-art performance on various natural language tasks byfine-tuning pretrained models. In task-oriented dialogue (ToD) systems,language models can be used for end-to-end training without relying on dialoguestate tracking to track the dialogue history but allowing the language modelsto generate responses according to the context given as input. This paperconducts a comparative study to show the effectiveness and strength of usingrecent pretrained models for fine-tuning, such as BART and T5, on endto-end ToDsystems. The experimental results show substantial performance improvementsafter language model fine-tuning. The models produce more fluent responsesafter adding knowledge to the context that guides the model to avoidhallucination and generate accurate entities in the generated responses.Furthermore, we found that BART and T5 outperform GPT-based models in BLEU andF1 scores and achieve state-of-the-art performance in a ToD system.