智能結腸鏡的前沿

摘要

結腸鏡檢查目前是大腸癌最敏感的篩查方法之一。本研究探討智能結腸鏡技術的前沿及其對多模態醫學應用的潛在影響。為實現此目標，我們首先通過四個結腸鏡場景感知任務，包括分類、檢測、分割和視覺語言理解，評估當前以數據為中心和以模型為中心的景觀。這一評估使我們能夠識別特定領域的挑戰，並顯示結腸鏡的多模態研究仍然有待進一步探索。為迎接即將到來的多模態時代，我們建立了三個基礎性倡議：一個大規模多模態指導調整數據集 ColonINST、一個針對結腸鏡設計的多模態語言模型 ColonGPT，以及一個多模態基準測試。為了促進對這一快速發展領域的持續監控，我們提供了一個用於最新更新的公共網站：https://github.com/ai4colonoscopy/IntelliScope。

English

Colonoscopy is currently one of the most sensitive screening methods for colorectal cancer. This study investigates the frontiers of intelligent colonoscopy techniques and their prospective implications for multimodal medical applications. With this goal, we begin by assessing the current data-centric and model-centric landscapes through four tasks for colonoscopic scene perception, including classification, detection, segmentation, and vision-language understanding. This assessment enables us to identify domain-specific challenges and reveals that multimodal research in colonoscopy remains open for further exploration. To embrace the coming multimodal era, we establish three foundational initiatives: a large-scale multimodal instruction tuning dataset ColonINST, a colonoscopy-designed multimodal language model ColonGPT, and a multimodal benchmark. To facilitate ongoing monitoring of this rapidly evolving field, we provide a public website for the latest updates: https://github.com/ai4colonoscopy/IntelliScope.

智能結腸鏡的前沿

Frontiers in Intelligent Colonoscopy

摘要

Summary

Support

Support