Abstract
Text-to-video generation models have shown significant progress in the recentyears. However, they still struggle with generating complex dynamic scenesbased on compositional text prompts, such as attribute binding for multipleobjects, temporal dynamics associated with different objects, and interactionsbetween objects. Our key motivation is that complex tasks can be decomposedinto simpler ones, each handled by a role-specialized MLLM agent. Multipleagents can collaborate together to achieve collective intelligence for complexgoals. We propose GenMAC, an iterative, multi-agent framework that enablescompositional text-to-video generation. The collaborative workflow includesthree stages: Design, Generation, and Redesign, with an iterative loop betweenthe Generation and Redesign stages to progressively verify and refine thegenerated videos. The Redesign stage is the most challenging stage that aims toverify the generated videos, suggest corrections, and redesign the textprompts, frame-wise layouts, and guidance scales for the next iteration ofgeneration. To avoid hallucination of a single MLLM agent, we decompose thisstage to four sequentially-executed MLLM-based agents: verification agent,suggestion agent, correction agent, and output structuring agent. Furthermore,to tackle diverse scenarios of compositional text-to-video generation, wedesign a self-routing mechanism to adaptively select the proper correctionagent from a collection of correction agents each specialized for one scenario.Extensive experiments demonstrate the effectiveness of GenMAC, achievingstate-of-the art performance in compositional text-to-video generation.