Abstract
The integration of language models in the legal domain holds considerablepromise for streamlining processes and improving efficiency in managingextensive workloads. However, the specialized terminology, nuanced language,and formal style of legal texts can present substantial challenges. This studyexamines whether preference-based training techniques, specificallyReinforcement Learning from Human Feedback and Direct Preference Optimization,can enhance models' performance in generating Icelandic legal summaries thatalign with domain-specific language standards and user preferences. We comparemodels fine-tuned with preference training to those using conventionalsupervised learning. Results indicate that preference training improves thelegal accuracy of generated summaries over standard fine-tuning but does notsignificantly enhance the overall quality of Icelandic language usage.Discrepancies between automated metrics and human evaluations furtherunderscore the importance of qualitative assessment in developing languagemodels for the legal domain.