智能體實驗室：以LLM智能體作為研究助理

Agent Laboratory: Using LLM Agents as Research Assistants

January 8, 2025

作者: Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, Emad Barsoum

cs.AI

摘要

從歷史來看，科學發現一直是一個漫長且昂貴的過程，從最初的構想到最終結果都需要大量的時間和資源。為了加速科學發現、降低研究成本並提高研究質量，我們引入了Agent Laboratory，這是一個基於自主LLM的框架，能夠完成整個研究過程。該框架接受人類提供的研究想法，並通過三個階段——文獻回顧、實驗和報告撰寫，生成包括代碼庫和研究報告在內的全面研究輸出，同時讓用戶在每個階段提供反饋和指導。我們使用各種最先進的LLM部署Agent Laboratory，並邀請多位研究人員通過參與調查來評估其質量，提供人類反饋以指導研究過程，然後評估最終論文。我們發現：(1)由o1-preview驅動的Agent Laboratory產生了最佳的研究結果；(2)生成的機器學習代碼能夠達到與現有方法相比的最先進性能；(3)人類參與，在每個階段提供反饋，顯著提高了研究的整體質量；(4)Agent Laboratory大幅降低了研究費用，與以往的自主研究方法相比，實現了84%的減少。我們希望Agent Laboratory能讓研究人員將更多精力投入於創意構想，而不是低級編碼和撰寫，從而加速科學發現。

English

Historically, scientific discovery has been a lengthy and costly process, demanding substantial time and resources from initial conception to final results. To accelerate scientific discovery, reduce research costs, and improve research quality, we introduce Agent Laboratory, an autonomous LLM-based framework capable of completing the entire research process. This framework accepts a human-provided research idea and progresses through three stages--literature review, experimentation, and report writing to produce comprehensive research outputs, including a code repository and a research report, while enabling users to provide feedback and guidance at each stage. We deploy Agent Laboratory with various state-of-the-art LLMs and invite multiple researchers to assess its quality by participating in a survey, providing human feedback to guide the research process, and then evaluate the final paper. We found that: (1) Agent Laboratory driven by o1-preview generates the best research outcomes; (2) The generated machine learning code is able to achieve state-of-the-art performance compared to existing methods; (3) Human involvement, providing feedback at each stage, significantly improves the overall quality of research; (4) Agent Laboratory significantly reduces research expenses, achieving an 84% decrease compared to previous autonomous research methods. We hope Agent Laboratory enables researchers to allocate more effort toward creative ideation rather than low-level coding and writing, ultimately accelerating scientific discovery.