Abstract
Medical image analysis is essential in modern healthcare. Deep learning hasredirected research focus toward complex medical multimodal tasks, includingreport generation and visual question answering. Traditional task-specificmodels often fall short in handling these challenges. Large vision-languagemodels (LVLMs) offer new solutions for solving such tasks. In this study, webuild on the popular LLaVA framework to systematically explore modelarchitectures and training strategies for both 2D and 3D medical LVLMs. Wepresent extensive empirical findings and practical guidance. To supportreproducibility and future research, we release a modular codebase, MedM-VL,and two pre-trained models: MedM-VL-2D for 2D medical image analysis andMedM-VL-CT-Chest for 3D CT-based applications. The code is available at:https://github.com/MSIIP/MedM-VL