Seaweed-7B：高效成本訓練的視頻生成基礎模型

Seaweed-7B: Cost-Effective Training of Video Generation Foundation Model

April 11, 2025

作者: Team Seawead, Ceyuan Yang, Zhijie Lin, Yang Zhao, Shanchuan Lin, Zhibei Ma, Haoyuan Guo, Hao Chen, Lu Qi, Sen Wang, Feng Cheng, Feilong Zuo Xuejiao Zeng, Ziyan Yang, Fangyuan Kong, Zhiwu Qing, Fei Xiao, Meng Wei, Tuyen Hoang, Siyu Zhang, Peihao Zhu, Qi Zhao, Jiangqiao Yan, Liangke Gui, Sheng Bi, Jiashi Li, Yuxi Ren, Rui Wang, Huixia Li, Xuefeng Xiao, Shu Liu, Feng Ling, Heng Zhang, Houmin Wei, Huafeng Kuang, Jerry Duncan, Junda Zhang, Junru Zheng, Li Sun, Manlin Zhang, Renfei Sun, Xiaobin Zhuang, Xiaojie Li, Xin Xia, Xuyan Chi, Yanghua Peng, Yuping Wang, Yuxuan Wang, Zhongkai Zhao, Zhuo Chen, Zuquan Song, Zhenheng Yang, Jiashi Feng, Jianchao Yang, Lu Jiang

cs.AI

摘要

本技術報告提出了一種成本效益高的策略，用於訓練視頻生成基礎模型。我們介紹了一個中等規模的研究模型，名為Seaweed-7B，該模型約有70億參數（7B），並從零開始訓練，使用了665,000個H100 GPU小時。儘管訓練時使用了適中的計算資源，Seaweed-7B在與當代更大規模的視頻生成模型相比時，展現出了極具競爭力的性能。在資源受限的環境中，設計選擇尤為關鍵。本技術報告強調了提升中等規模擴散模型性能的關鍵設計決策。根據實證研究，我們得出兩個觀察結果：（1）Seaweed-7B的表現可與使用更多GPU資源訓練的更大模型相媲美，甚至超越；（2）我們的模型展現出強大的泛化能力，能夠通過輕量級微調或繼續訓練，有效地適應廣泛的下游應用。詳見項目頁面：https://seaweed.video/。

English

This technical report presents a cost-efficient strategy for training a video generation foundation model. We present a mid-sized research model with approximately 7 billion parameters (7B) called Seaweed-7B trained from scratch using 665,000 H100 GPU hours. Despite being trained with moderate computational resources, Seaweed-7B demonstrates highly competitive performance compared to contemporary video generation models of much larger size. Design choices are especially crucial in a resource-constrained setting. This technical report highlights the key design decisions that enhance the performance of the medium-sized diffusion model. Empirically, we make two observations: (1) Seaweed-7B achieves performance comparable to, or even surpasses, larger models trained on substantially greater GPU resources, and (2) our model, which exhibits strong generalization ability, can be effectively adapted across a wide range of downstream applications either by lightweight fine-tuning or continue training. See the project page at https://seaweed.video/