Abstract
Multimodal large language models (MLLMs) have advanced perception acrosstext, vision, and audio, yet they often struggle with structured cross-modalreasoning, particularly when integrating audio and visual signals. We introduceEchoInk-R1, a reinforcement learning framework that enhances such reasoning inMLLMs. Built upon the Qwen2.5-Omni-7B foundation and optimized with GroupRelative Policy Optimization (GRPO), EchoInk-R1 tackles multiple-choicequestion answering over synchronized audio-image pairs. To enable this, wecurate AVQA-R1-6K, a dataset pairing such audio-image inputs withmultiple-choice questions derived from OmniInstruct-v1. EchoInk-R1-7B achieves85.77% accuracy on the validation set, outperforming the base model, whichscores 80.53%, using only 562 reinforcement learning steps. Beyond accuracy,EchoInk-R1 demonstrates reflective reasoning by revisiting initialinterpretations and refining responses when facing ambiguous multimodal inputs.These results suggest that lightweight reinforcement learning fine-tuningenhances cross-modal reasoning in MLLMs. EchoInk-R1 is the first framework tounify audio, visual, and textual modalities for general open-world reasoningvia reinforcement learning. Code and data are publicly released to facilitatefurther research.