Abstract
The advent of AI-Generated Content (AIGC) has spurred research into automatedvideo generation to streamline conventional processes. However, automatingstorytelling video production, particularly for customized narratives, remainschallenging due to the complexity of maintaining subject consistency acrossshots. While existing approaches like Mora and AesopAgent integrate multipleagents for Story-to-Video (S2V) generation, they fall short in preservingprotagonist consistency and supporting Customized Storytelling Video Generation(CSVG). To address these limitations, we propose StoryAgent, a multi-agentframework designed for CSVG. StoryAgent decomposes CSVG into distinct subtasksassigned to specialized agents, mirroring the professional production process.Notably, our framework includes agents for story design, storyboard generation,video creation, agent coordination, and result evaluation. Leveraging thestrengths of different models, StoryAgent enhances control over the generationprocess, significantly improving character consistency. Specifically, weintroduce a customized Image-to-Video (I2V) method, LoRA-BE, to enhanceintra-shot temporal consistency, while a novel storyboard generation pipelineis proposed to maintain subject consistency across shots. Extensive experimentsdemonstrate the effectiveness of our approach in synthesizing highly consistentstorytelling videos, outperforming state-of-the-art methods. Our contributionsinclude the introduction of StoryAgent, a versatile framework for videogeneration tasks, and novel techniques for preserving protagonist consistency.