游戏工厂：利用生成交互式视频创造新游戏

GameFactory: Creating New Games with Generative Interactive Videos

January 14, 2025

作者: Jiwen Yu, Yiran Qin, Xintao Wang, Pengfei Wan, Di Zhang, Xihui Liu

cs.AI

摘要

生成式游戏引擎有潜力彻底改变游戏开发，通过自主创建新内容和减少手动工作量。然而，现有基于视频的游戏生成方法未能解决场景泛化的关键挑战，限制了它们适用于具有固定风格和场景的现有游戏。在本文中，我们提出了GameFactory，一个专注于探索游戏视频生成中场景泛化的框架。为了实现全新和多样化游戏的创作，我们利用在开放域视频数据上训练的预训练视频扩散模型。为了弥合开放域先验和小规模游戏数据集之间的领域差距，我们提出了一个多阶段训练策略，将游戏风格学习与动作控制解耦，保持开放域泛化同时实现动作可控性。利用Minecraft作为我们的数据来源，我们发布了GF-Minecraft，一个高质量且多样化的动作注释视频数据集供研究使用。此外，我们扩展了我们的框架，实现自回归动作可控的游戏视频生成，实现无限长度的互动游戏视频制作。实验结果表明，GameFactory有效生成开放域、多样化和动作可控的游戏视频，代表了人工智能驱动游戏生成的重要进展。我们的数据集和项目页面可在https://vvictoryuki.github.io/gamefactory/公开获取。

English

Generative game engines have the potential to revolutionize game development by autonomously creating new content and reducing manual workload. However, existing video-based game generation methods fail to address the critical challenge of scene generalization, limiting their applicability to existing games with fixed styles and scenes. In this paper, we present GameFactory, a framework focused on exploring scene generalization in game video generation. To enable the creation of entirely new and diverse games, we leverage pre-trained video diffusion models trained on open-domain video data. To bridge the domain gap between open-domain priors and small-scale game dataset, we propose a multi-phase training strategy that decouples game style learning from action control, preserving open-domain generalization while achieving action controllability. Using Minecraft as our data source, we release GF-Minecraft, a high-quality and diversity action-annotated video dataset for research. Furthermore, we extend our framework to enable autoregressive action-controllable game video generation, allowing the production of unlimited-length interactive game videos. Experimental results demonstrate that GameFactory effectively generates open-domain, diverse, and action-controllable game videos, representing a significant step forward in AI-driven game generation. Our dataset and project page are publicly available at https://vvictoryuki.github.io/gamefactory/.