EnerVerse: Envisioning Embodied Future Space for Robotics Manipulation

  • 2025-01-03 17:00:33
  • Siyuan Huang, Liliang Chen, Pengfei Zhou, Shengcong Chen, Zhengkai Jiang, Yue Hu, Peng Gao, Hongsheng Li, Maoqing Yao, Guanghui Ren
  • 0

Abstract

We introduce EnerVerse, a comprehensive framework for embodied future spacegeneration specifically designed for robotic manipulation tasks. EnerVerseseamlessly integrates convolutional and bidirectional attention mechanisms forinner-chunk space modeling, ensuring low-level consistency and continuity.Recognizing the inherent redundancy in video data, we propose a sparse memorycontext combined with a chunkwise unidirectional generative paradigm to enablethe generation of infinitely long sequences. To further augment roboticcapabilities, we introduce the Free Anchor View (FAV) space, which providesflexible perspectives to enhance observation and analysis. The FAV spacemitigates motion modeling ambiguity, removes physical constraints in confinedenvironments, and significantly improves the robot's generalization andadaptability across various tasks and settings. To address the prohibitivecosts and labor intensity of acquiring multi-camera observations, we present adata engine pipeline that integrates a generative model with 4D GaussianSplatting (4DGS). This pipeline leverages the generative model's robustgeneralization capabilities and the spatial constraints provided by 4DGS,enabling an iterative enhancement of data quality and diversity, thus creatinga data flywheel effect that effectively narrows the sim-to-real gap. Finally,our experiments demonstrate that the embodied future space generation priorsubstantially enhances policy predictive capabilities, resulting in improvedoverall performance, particularly in long-range robotic manipulation tasks.

 

Quick Read (beta)

loading the full paper ...