Abstract
Hallucinations in large language models pose a critical challenge forapplications requiring factual reliability, particularly in high-stakes domainssuch as finance. This work presents an effective approach for detecting andediting factually incorrect content in model-generated responses based on theprovided context. Given a user-defined domain-specific error taxonomy, weconstruct a synthetic dataset by inserting tagged errors into financialquestion-answering corpora and then fine-tune four language models, Phi-4,Phi-4-mini, Qwen3-4B, and Qwen3-14B, to detect and edit these factualinaccuracies. Our best-performing model, fine-tuned Phi-4, achieves an 8%improvement in binary F1 score and a 30% gain in overall detection performancecompared to OpenAI-o3. Notably, our fine-tuned Phi-4-mini model, despite havingonly 4 billion parameters, maintains competitive performance with just a 2%drop in binary detection and a 0.1% decline in overall detection compared toOpenAI-o3. Our work provides a practical solution for detecting and editingfactual inconsistencies in financial text generation while introducing ageneralizable framework that can enhance the trustworthiness and alignment oflarge language models across diverse applications beyond finance. Our code anddata are available at https://github.com/pegasi-ai/shield.