Coffee-Gym：一個用於評估和改進錯誤程式碼上的自然語言反饋的環境。

Coffee-Gym: An Environment for Evaluating and Improving Natural Language Feedback on Erroneous Code

September 29, 2024

作者: Hyungjoo Chae, Taeyoon Kwon, Seungjun Moon, Yongho Song, Dongjin Kang, Kai Tzu-iunn Ong, Beong-woo Kwak, Seonghyeon Bae, Seung-won Hwang, Jinyoung Yeo

cs.AI

摘要

本文介紹了Coffee-Gym，一個用於訓練能夠提供程式碼編輯反饋的模型的全面RL環境。Coffee-Gym包括兩個主要組件：(1) Coffee，一個包含人類編碼問題的程式碼編輯軌跡和機器生成的錯誤程式碼編輯反饋的數據集；(2) CoffeeEval，一個獎勵函數，通過評估修改後程式碼在單元測試中的表現，忠實地反映反饋的幫助性。通過這兩者，Coffee-Gym解決了缺乏高質量數據集來訓練RL反饋模型的問題，並提供比SOTA獎勵模型（即GPT-4）更準確的獎勵。應用Coffee-Gym，我們引出了優於基準線的反饋模型，能夠增強開源代碼LLMs的程式碼編輯，使其與封閉源LLMs相媲美。我們將數據集和模型檢查點公開提供。

English

This paper presents Coffee-Gym, a comprehensive RL environment for training models that provide feedback on code editing. Coffee-Gym includes two major components: (1) Coffee, a dataset containing humans' code edit traces for coding questions and machine-written feedback for editing erroneous code; (2) CoffeeEval, a reward function that faithfully reflects the helpfulness of feedback by assessing the performance of the revised code in unit tests. With them, Coffee-Gym addresses the unavailability of high-quality datasets for training feedback models with RL, and provides more accurate rewards than the SOTA reward model (i.e., GPT-4). By applying Coffee-Gym, we elicit feedback models that outperform baselines in enhancing open-source code LLMs' code editing, making them comparable with closed-source LLMs. We make the dataset and the model checkpoint publicly available.