Abstract
Training large language models (LLMs) to follow instructions hassignificantly enhanced their ability to tackle unseen tasks. However, despitetheir strong generalization capabilities, instruction-following LLMs encounterdifficulties when dealing with tasks that require domain knowledge. This workintroduces a specialized instruction fine-tuning for the domain ofcomputational argumentation (CA). The goal is to enable an LLM to effectivelytackle any unseen CA tasks while preserving its generalization capabilities.Reviewing existing CA research, we crafted natural language instructions for105 CA tasks to this end. On this basis, we developed a CA-specific benchmarkfor LLMs that allows for a comprehensive evaluation of LLMs' capabilities insolving various CA tasks. We synthesized 52k CA-related instructions, adaptingthe self-instruct process to train a CA-specialized instruction-following LLM.Our experiments suggest that CA-specialized instruction fine-tuningsignificantly enhances the LLM on both seen and unseen CA tasks. At the sametime, performance on the general NLP tasks of the SuperNI benchmark remainsstable.