黑雁韵律者：利用RWKV-7作为大规模时间序列建模中Transformer的简洁而卓越替代方案

BlackGoose Rimer: Harnessing RWKV-7 as a Simple yet Superior Replacement for Transformers in Large-Scale Time Series Modeling

March 8, 2025

作者: Li weile, Liu Xiao

cs.AI

摘要

时间序列模型在扩展以处理大规模复杂数据集方面面临重大挑战，这与大型语言模型（LLMs）所实现的扩展能力相似。时间序列数据的独特特性以及模型扩展的计算需求，要求我们采取创新方法。尽管研究人员已探索了多种架构，如Transformer、LSTM和GRU，以应对这些挑战，但我们提出了一种基于RWKV-7的新颖解决方案，该方案将元学习融入其状态更新机制中。通过将RWKV-7的时间混合与通道混合组件整合到基于Transformer的时间序列模型Timer中，我们实现了性能的显著提升，大约在1.13至43.3倍之间，同时训练时间减少了4.5倍，且仅使用了1/23的参数。我们的代码和模型权重已公开发布，供进一步研究与开发，访问地址为https://github.com/Alic-Li/BlackGoose_Rimer。

English

Time series models face significant challenges in scaling to handle large and complex datasets, akin to the scaling achieved by large language models (LLMs). The unique characteristics of time series data and the computational demands of model scaling necessitate innovative approaches. While researchers have explored various architectures such as Transformers, LSTMs, and GRUs to address these challenges, we propose a novel solution using RWKV-7, which incorporates meta-learning into its state update mechanism. By integrating RWKV-7's time mix and channel mix components into the transformer-based time series model Timer, we achieve a substantial performance improvement of approximately 1.13 to 43.3x and a 4.5x reduction in training time with 1/23 parameters, all while utilizing fewer parameters. Our code and model weights are publicly available for further research and development at https://github.com/Alic-Li/BlackGoose_Rimer.