將數據置於離線多智能體強化學習的核心

摘要

離線多智能體強化學習（MARL）是一個令人振奮的研究方向，利用靜態數據集為多智能體系統找到最佳控制策略。儘管該領域在定義上是數據驅動的，但迄今為止的努力忽略了數據，以追求最先進的結果。我們首先通過對文獻進行調查來證實這一點，展示大多數作品如何生成自己的數據集，缺乏一致的方法論，並提供有關這些數據集特徵的稀缺信息。然後，我們展示忽略數據性質為何是有問題的，通過突出示例說明算法性能與使用的數據集密切相關，需要在該領域中進行實驗的共同基礎。為此，我們朝著改進離線MARL中數據使用和數據意識邁出了一大步，提出了三個關鍵貢獻：（1）生成新數據集的清晰指南；（2）對80多個現有數據集進行標準化，存儲在一個公開可用的存儲庫中，使用一致的存儲格式和易於使用的API；以及（3）一套分析工具，讓我們更好地了解這些數據集，幫助進一步發展。

English

Offline multi-agent reinforcement learning (MARL) is an exciting direction of research that uses static datasets to find optimal control policies for multi-agent systems. Though the field is by definition data-driven, efforts have thus far neglected data in their drive to achieve state-of-the-art results. We first substantiate this claim by surveying the literature, showing how the majority of works generate their own datasets without consistent methodology and provide sparse information about the characteristics of these datasets. We then show why neglecting the nature of the data is problematic, through salient examples of how tightly algorithmic performance is coupled to the dataset used, necessitating a common foundation for experiments in the field. In response, we take a big step towards improving data usage and data awareness in offline MARL, with three key contributions: (1) a clear guideline for generating novel datasets; (2) a standardisation of over 80 existing datasets, hosted in a publicly available repository, using a consistent storage format and easy-to-use API; and (3) a suite of analysis tools that allow us to understand these datasets better, aiding further development.

將數據置於離線多智能體強化學習的核心

Putting Data at the Centre of Offline Multi-Agent Reinforcement Learning

摘要

Summary

Support

Support