智能体实验室：将LLM智能体用作研究助手

Agent Laboratory: Using LLM Agents as Research Assistants

January 8, 2025

作者: Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, Emad Barsoum

cs.AI

摘要

在历史上，科学发现一直是一个漫长且昂贵的过程，从最初构思到最终结果需要大量的时间和资源。为了加速科学发现，降低研究成本，提高研究质量，我们引入了Agent Laboratory，这是一个基于自主LLM的框架，能够完成整个研究过程。该框架接受人类提供的研究想法，并通过文献综述、实验和撰写报告三个阶段来生成全面的研究成果，包括代码库和研究报告，同时允许用户在每个阶段提供反馈和指导。我们使用各种最先进的LLM部署Agent Laboratory，并邀请多位研究人员通过参与调查来评估其质量，提供人类反馈以指导研究过程，然后评估最终论文。我们发现：（1）由o1-preview驱动的Agent Laboratory生成了最佳的研究成果；（2）生成的机器学习代码能够与现有方法相比实现最先进的性能；（3）人类参与，在每个阶段提供反馈，显著提高了研究的整体质量；（4）Agent Laboratory显著降低了研究费用，与以往的自主研究方法相比，实现了84%的降低。我们希望Agent Laboratory能够让研究人员将更多精力投入创意构思而不是低层次的编码和撰写，从而加速科学发现。

English

Historically, scientific discovery has been a lengthy and costly process, demanding substantial time and resources from initial conception to final results. To accelerate scientific discovery, reduce research costs, and improve research quality, we introduce Agent Laboratory, an autonomous LLM-based framework capable of completing the entire research process. This framework accepts a human-provided research idea and progresses through three stages--literature review, experimentation, and report writing to produce comprehensive research outputs, including a code repository and a research report, while enabling users to provide feedback and guidance at each stage. We deploy Agent Laboratory with various state-of-the-art LLMs and invite multiple researchers to assess its quality by participating in a survey, providing human feedback to guide the research process, and then evaluate the final paper. We found that: (1) Agent Laboratory driven by o1-preview generates the best research outcomes; (2) The generated machine learning code is able to achieve state-of-the-art performance compared to existing methods; (3) Human involvement, providing feedback at each stage, significantly improves the overall quality of research; (4) Agent Laboratory significantly reduces research expenses, achieving an 84% decrease compared to previous autonomous research methods. We hope Agent Laboratory enables researchers to allocate more effort toward creative ideation rather than low-level coding and writing, ultimately accelerating scientific discovery.