Demystifying the Visual Quality Paradox in Multimodal Large Language Models

  • 2025-06-18 18:14:07
  • Shuo Xing, Lanqing Guo, Hongyuan Hua, Seoyoung Lee, Peiran Li, Yufei Wang, Zhangyang Wang, Zhengzhong Tu
  • 0

Abstract

Recent Multimodal Large Language Models (MLLMs) excel on benchmarkvision-language tasks, yet little is known about how input visual qualityshapes their responses. Does higher perceptual quality of images alreadytranslate to better MLLM understanding? We conduct the first systematic studyspanning leading MLLMs and a suite of vision-language benchmarks, applyingcontrolled degradations and stylistic shifts to each image. Surprisingly, weuncover a visual-quality paradox: model, task, and even individual-instanceperformance can improve when images deviate from human-perceived fidelity.Off-the-shelf restoration pipelines fail to reconcile these idiosyncraticpreferences. To close the gap, we introduce Visual-Quality Test-Time Tuning(VQ-TTT)-a lightweight adaptation module that: (1) inserts a learnable,low-rank kernel before the frozen vision encoder to modulate frequency content;and (2) fine-tunes only shallow vision-encoder layers via LoRA. VQ-TTTdynamically adjusts each input image in a single forward pass, aligning it withtask-specific model preferences. Across the evaluated MLLMs and all datasets,VQ-TTT lifts significant average accuracy, with no external models, cachedfeatures, or extra training data. These findings redefine ``better'' visualinputs for MLLMs and highlight the need for adaptive, rather than universally``clean'', imagery, in the new era of AI being the main data customer.

 

Quick Read (beta)

loading the full paper ...