FilmAgent：用于虚拟3D空间中端到端电影自动化的多智能体框架

FilmAgent: A Multi-Agent Framework for End-to-End Film Automation in Virtual 3D Spaces

January 22, 2025

作者: Zhenran Xu, Longyue Wang, Jifang Wang, Zhouyi Li, Senbao Shi, Xue Yang, Yiyu Wang, Baotian Hu, Jun Yu, Min Zhang

cs.AI

摘要

虚拟电影制作需要复杂的决策过程，包括剧本写作、虚拟摄影术以及精确的演员定位和动作。受最近自然语言处理代理社会中自动决策的进展启发，本文介绍了FilmAgent，这是一个基于LLM的多代理协作框架，用于在我们构建的3D虚拟空间中实现电影自动化的端到端流程。FilmAgent模拟了各种工作人员角色，包括导演、编剧、演员和摄影师，并涵盖了电影制作工作流程的关键阶段：（1）创意开发将头脑风暴的想法转化为结构化的故事大纲；（2）剧本写作详细描述了每个场景的对话和角色动作；（3）摄影术确定了每个镜头的摄像机设置。一组代理通过迭代反馈和修订进行协作，从而验证中间剧本并减少幻觉。我们对15个想法和4个关键方面的生成视频进行评估。人类评估显示，FilmAgent在所有方面都优于所有基准线，并平均得分为3.98（满分5分），表明了电影制作中多代理协作的可行性。进一步分析显示，尽管使用较不先进的GPT-4o模型，FilmAgent超越了单一代理o1，显示了良好协调的多代理系统的优势。最后，我们讨论了OpenAI的文本到视频模型Sora和我们的FilmAgent在电影制作中的互补优势和劣势。

English

Virtual film production requires intricate decision-making processes, including scriptwriting, virtual cinematography, and precise actor positioning and actions. Motivated by recent advances in automated decision-making with language agent-based societies, this paper introduces FilmAgent, a novel LLM-based multi-agent collaborative framework for end-to-end film automation in our constructed 3D virtual spaces. FilmAgent simulates various crew roles, including directors, screenwriters, actors, and cinematographers, and covers key stages of a film production workflow: (1) idea development transforms brainstormed ideas into structured story outlines; (2) scriptwriting elaborates on dialogue and character actions for each scene; (3) cinematography determines the camera setups for each shot. A team of agents collaborates through iterative feedback and revisions, thereby verifying intermediate scripts and reducing hallucinations. We evaluate the generated videos on 15 ideas and 4 key aspects. Human evaluation shows that FilmAgent outperforms all baselines across all aspects and scores 3.98 out of 5 on average, showing the feasibility of multi-agent collaboration in filmmaking. Further analysis reveals that FilmAgent, despite using the less advanced GPT-4o model, surpasses the single-agent o1, showing the advantage of a well-coordinated multi-agent system. Lastly, we discuss the complementary strengths and weaknesses of OpenAI's text-to-video model Sora and our FilmAgent in filmmaking.