LLaMo：基於大型語言模型的分子圖助手

LLaMo: Large Language Model-based Molecular Graph Assistant

October 31, 2024

作者: Jinyoung Park, Minseong Bae, Dohwan Ko, Hyunwoo J. Kim

cs.AI

摘要

大型語言模型（LLMs）展示了出色的泛化和遵循指示能力，並通過指示調整。LLMs和指示調整的進步導致了大型視覺語言模型（LVLMs）的發展。然而，在分子領域中，LLMs和指示調整的能力尚未得到充分探索。因此，我們提出了LLaMo：基於大型語言模型的分子圖助手，這是一個端到端訓練的大型分子圖語言模型。為了彌合語言和圖形模態之間的差異，我們提出了多級圖形投影器，通過摘要每個GNN層的輸出表示和通過交叉注意機制提取圖形代表和主題表示，將圖形表示轉換為圖形令牌。我們還引入了機器生成的分子圖指示數據，以指示調整大型分子圖語言模型，以實現通用分子和語言理解。我們的廣泛實驗表明，LLaMo在多樣任務上表現出最佳性能，例如分子描述生成、性質預測和IUPAC命名預測。LLaMo的代碼可在https://github.com/mlvlab/LLaMo找到。

English

Large Language Models (LLMs) have demonstrated remarkable generalization and instruction-following capabilities with instruction tuning. The advancements in LLMs and instruction tuning have led to the development of Large Vision-Language Models (LVLMs). However, the competency of the LLMs and instruction tuning have been less explored in the molecular domain. Thus, we propose LLaMo: Large Language Model-based Molecular graph assistant, which is an end-to-end trained large molecular graph-language model. To bridge the discrepancy between the language and graph modalities, we present the multi-level graph projector that transforms graph representations into graph tokens by abstracting the output representations of each GNN layer and motif representations with the cross-attention mechanism. We also introduce machine-generated molecular graph instruction data to instruction-tune the large molecular graph-language model for general-purpose molecule and language understanding. Our extensive experiments demonstrate that LLaMo shows the best performance on diverse tasks, such as molecular description generation, property prediction, and IUPAC name prediction. The code of LLaMo is available at https://github.com/mlvlab/LLaMo.