ReWorld: Multi-Dimensional Reward Modeling for Embodied Video Generation

  • 2026-11-02 07:00:44
  • Baorui Peng, Wenyao Zhang, Liang Xu, Bohan Li, Hu Zhu, Zekun Qi, Hongsi Liu, Jiazhao Zhang, Wenjun Zeng, Xin Jin
  • 0

Abstract

Current embodied video generation models primarily emphasize visual generative quality while overlooking physical fidelity, dynamic consistency, and task logic, especially for contact-rich manipulation tasks, which limits their applicability to downstream tasks. In this paper, we introduce ReWorld, a framework aimed to employ reinforcement learning to align the video generation with physical realism, task completion capability, embodiment plausibility and visual quality. Specifically, we first construct a large-scale (~235K) video preference dataset and employ it to train a hierarchical reward model designed to capture multi-dimensional reward consistent with human preferences. We further propose a practical alignment algorithm that post-trains flow-based video generators using this reward through a computationally efficient PPO-style algorithm. Comprehensive experiments and theoretical analysis demonstrate that ReWorld significantly improves the physical fidelity, logical coherence, embodiment and visual quality of generated rollouts, outperforming previous methods.

 

Quick Read (beta)

loading the full paper ...