Abstract
Recent releases of pre-trained Large Language Models (LLMs) have gainedconsiderable traction, yet research on fine-tuning and employingdomain-specific LLMs remains scarce. This study investigates approaches forfine-tuning and leveraging domain-specific LLMs, highlighting trends in LLMs,foundational models, and methods for domain-specific pre-training. Focusing onthe financial sector, it details dataset selection, preprocessing, modelchoice, and considerations crucial for LLM fine-tuning in finance. Addressingthe unique characteristics of financial data, the study explores theconstruction of domain-specific vocabularies and considerations for securityand regulatory compliance. In the practical application of LLM fine-tuning, thestudy outlines the procedure and implementation for generating domain-specificLLMs in finance. Various financial cases, including stock price prediction,sentiment analysis of financial news, automated document processing, research,information extraction, and customer service enhancement, are exemplified. Thestudy explores the potential of LLMs in the financial domain, identifieslimitations, and proposes directions for improvement, contributing valuableinsights for future research. Ultimately, it advances natural languageprocessing technology in business, suggesting proactive LLM utilization infinancial services across industries.