Factuality Matters: When Image Generation and Editing Meet Structured Visuals

  • 2025-10-06 17:56:55
  • Le Zhuo, Songhao Han, Yuandong Pu, Boxiang Qiu, Sayak Paul, Yue Liao, Yihao Liu, Jie Shao, Xi Chen, Si Liu, Hongsheng Li
  • 0

Abstract

While modern visual generation models excel at creating aestheticallypleasing natural images, they struggle with producing or editing structuredvisuals like charts, diagrams, and mathematical figures, which demandcomposition planning, text rendering, and multimodal reasoning for factualfidelity. To address this, we present the first comprehensive, systematicinvestigation of this domain, encompassing data construction, model training,and an evaluation benchmark. First, we construct a large-scale dataset of 1.3million high-quality structured image pairs derived from executable drawingprograms and augmented with chain-of-thought reasoning annotations. Building onit, we train a unified model that integrates a VLM with FLUX.1 Kontext via alightweight connector for enhanced multimodal understanding. A three-stagetraining curriculum enables progressive feature alignment, knowledge infusion,and reasoning-augmented generation, further boosted by an external reasoner atinference time. Finally, we introduce StructBench, a novel benchmark forgeneration and editing with over 1,700 challenging instances, and anaccompanying evaluation metric, StructScore, which employs a multi-round Q\&Aprotocol to assess fine-grained factual accuracy. Evaluations of 15 modelsreveal that even leading closed-source systems remain far from satisfactory.Our model attains strong editing performance, and inference-time reasoningyields consistent gains across diverse architectures. By releasing the dataset,model, and benchmark, we aim to advance unified multimodal foundations forstructured visuals.

 

Quick Read (beta)

loading the full paper ...