MedMobile：具有專家級臨床能力的手機尺寸語言模型

MedMobile: A mobile-sized language model with expert-level clinical capabilities

October 11, 2024

作者: Krithik Vishwanath, Jaden Stryker, Anton Alaykin, Daniel Alexander Alber, Eric Karl Oermann

cs.AI

摘要

語言模型（LMs）在醫學領域展示了專家級的推理和回憶能力。然而，計算成本和隱私問題正在成為廣泛應用的障礙。我們介紹了一種簡潔的改編phi-3-mini，MedMobile，這是一個擁有38億參數的LM，可以在移動設備上運行，用於醫學應用。我們展示了MedMobile在MedQA（USMLE）上取得了75.7%的分數，超過了醫生的及格分數（約60%），並接近其100倍大小模型的分數。我們隨後進行了一系列仔細的消融實驗，並展示了思維鏈、集成和微調對性能提升的最大影響，而意外的檢索增強生成未能顯示出顯著改進。

English

Language models (LMs) have demonstrated expert-level reasoning and recall abilities in medicine. However, computational costs and privacy concerns are mounting barriers to wide-scale implementation. We introduce a parsimonious adaptation of phi-3-mini, MedMobile, a 3.8 billion parameter LM capable of running on a mobile device, for medical applications. We demonstrate that MedMobile scores 75.7% on the MedQA (USMLE), surpassing the passing mark for physicians (~60%), and approaching the scores of models 100 times its size. We subsequently perform a careful set of ablations, and demonstrate that chain of thought, ensembling, and fine-tuning lead to the greatest performance gains, while unexpectedly retrieval augmented generation fails to demonstrate significant improvements