導師 CoPilot：一種用於擴展即時專業知識的人工智慧方法

Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise

October 3, 2024

作者: Rose E. Wang, Ana T. Ribeiro, Carly D. Robinson, Susanna Loeb, Dora Demszky

cs.AI

摘要

生成式人工智慧，特別是語言模型（LMs），具有潛力改變具有社會影響力的現實領域，尤其是在專家資源有限的情況下。例如，在教育領域，訓練新手教育工作者需要專家指導以提高效率，但成本高昂，這在大規模提升教育質量方面造成了重大障礙。這個挑戰對來自弱勢社區的學生造成不成比例的傷害，而這些學生最有可能從高質量教育中受益。我們介紹了Tutor CoPilot，一種新穎的人工智慧方法，利用專家思維模型為導師提供類似專家的指導。這項研究是在現場輔導中對人工智慧系統進行的第一項隨機對照試驗，涉及來自歷史上受到輕視社區的900名導師和1,800名K-12學生。根據預先註冊的分析計劃，我們發現與使用Tutor CoPilot的導師合作的學生更有可能掌握主題（p<0.01），提高了4個百分點。值得注意的是，評分較低導師的學生獲益最大，掌握程度提高了9個百分點。我們發現Tutor CoPilot每年每位導師的成本僅為20美元。我們使用分類器分析了超過550,000條消息，以識別教學策略，發現使用Tutor CoPilot的導師更有可能使用高質量策略來促進學生理解（例如，提問引導），並且更不太可能直接給出答案。導師訪談突顯了Tutor CoPilot的指導如何幫助導師應對學生需求，但也指出了Tutor CoPilot存在的問題，例如生成的建議不適合年級水平。總的來說，我們對Tutor CoPilot的研究展示了人工智慧系統如何在現實領域中擴展專業知識，彌合技能差距，並創造一個未來，讓高質量教育對所有學生都可及。

English

Generative AI, particularly Language Models (LMs), has the potential to transform real-world domains with societal impact, particularly where access to experts is limited. For example, in education, training novice educators with expert guidance is important for effectiveness but expensive, creating significant barriers to improving education quality at scale. This challenge disproportionately harms students from under-served communities, who stand to gain the most from high-quality education. We introduce Tutor CoPilot, a novel Human-AI approach that leverages a model of expert thinking to provide expert-like guidance to tutors as they tutor. This study is the first randomized controlled trial of a Human-AI system in live tutoring, involving 900 tutors and 1,800 K-12 students from historically under-served communities. Following a preregistered analysis plan, we find that students working with tutors that have access to Tutor CoPilot are 4 percentage points (p.p.) more likely to master topics (p<0.01). Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by 9 p.p. We find that Tutor CoPilot costs only $20 per-tutor annually. We analyze 550,000+ messages using classifiers to identify pedagogical strategies, and find that tutors with access to Tutor CoPilot are more likely to use high-quality strategies to foster student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student. Tutor interviews highlight how Tutor CoPilot's guidance helps tutors to respond to student needs, though they flag issues in Tutor CoPilot, such as generating suggestions that are not grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates how Human-AI systems can scale expertise in real-world domains, bridge gaps in skills and create a future where high-quality education is accessible to all students.