Abstract
Mild Traumatic Brain Injury (TBI) detection presents significant challengesdue to the subtle and often ambiguous presentation of symptoms in medicalimaging, making accurate diagnosis a complex task. To address these challenges,we propose Proof-of-TBI, a medical diagnosis support system that integratesmultiple fine-tuned vision-language models with the OpenAI-o3 reasoning largelanguage model (LLM). Our approach fine-tunes multiple vision-language modelsusing a labeled dataset of TBI MRI scans, training them to diagnose TBIsymptoms effectively. The predictions from these models are aggregated througha consensus-based decision-making process. The system evaluates the predictionsfrom all fine-tuned vision language models using the OpenAI-o3 reasoning LLM, amodel that has demonstrated remarkable reasoning performance, to produce themost accurate final diagnosis. The LLM Agents orchestrates interactions betweenthe vision-language models and the reasoning LLM, managing the finaldecision-making process with transparency, reliability, and automation. Thisend-to-end decision-making workflow combines the vision-language modelconsortium with the OpenAI-o3 reasoning LLM, enabled by custom promptengineering by the LLM agents. The prototype for the proposed platform wasdeveloped in collaboration with the U.S. Army Medical Research team in NewportNews, Virginia, incorporating five fine-tuned vision-language models. Theresults demonstrate the transformative potential of combining fine-tunedvision-language model inputs with the OpenAI-o3 reasoning LLM to create arobust, secure, and highly accurate diagnostic system for mild TBI prediction.To the best of our knowledge, this research represents the first application offine-tuned vision-language models integrated with a reasoning LLM for TBIprediction tasks.