探索视觉Transformer中的关键神经元路径

Discovering Influential Neuron Path in Vision Transformers

March 12, 2025

作者: Yifan Wang, Yifei Liu, Yingdong Shi, Changming Li, Anqi Pang, Sibei Yang, Jingyi Yu, Kan Ren

cs.AI

摘要

视觉Transformer模型展现出强大的能力，但其内部机制对人类而言仍不透明，这为实际应用带来了挑战与风险。尽管先前的研究已尝试通过输入归因和神经元角色分析来揭示这些模型的神秘面纱，但在考虑层级信息及跨层信息流动的整体路径方面仍存在显著空白。本文中，我们探究了视觉Transformer中关键神经元路径的重要性，即从模型输入到输出、对模型推理影响最为显著的神经元序列。我们首先提出了一种联合影响力度量方法，用于评估一组神经元对模型结果的贡献。进而，我们提供了一种逐层递进的神经元定位策略，旨在高效地筛选出每一层中最具影响力的神经元，从而在目标模型内发现从输入到输出的关键神经元路径。实验证明，相较于现有基线方法，我们的方法在寻找信息流动的最具影响力神经元路径方面表现更优。此外，这些神经元路径揭示了视觉Transformer在处理同一图像类别内的视觉信息时，展现出特定的内部工作机制。我们进一步分析了这些神经元在图像分类任务中的关键作用，表明所发现的神经元路径已保留了模型在下游任务上的能力，这或许也为模型剪枝等实际应用提供了启示。项目网站及实现代码可在https://foundation-model-research.github.io/NeuronPath/获取。

English

Vision Transformer models exhibit immense power yet remain opaque to human understanding, posing challenges and risks for practical applications. While prior research has attempted to demystify these models through input attribution and neuron role analysis, there's been a notable gap in considering layer-level information and the holistic path of information flow across layers. In this paper, we investigate the significance of influential neuron paths within vision Transformers, which is a path of neurons from the model input to output that impacts the model inference most significantly. We first propose a joint influence measure to assess the contribution of a set of neurons to the model outcome. And we further provide a layer-progressive neuron locating approach that efficiently selects the most influential neuron at each layer trying to discover the crucial neuron path from input to output within the target model. Our experiments demonstrate the superiority of our method finding the most influential neuron path along which the information flows, over the existing baseline solutions. Additionally, the neuron paths have illustrated that vision Transformers exhibit some specific inner working mechanism for processing the visual information within the same image category. We further analyze the key effects of these neurons on the image classification task, showcasing that the found neuron paths have already preserved the model capability on downstream tasks, which may also shed some lights on real-world applications like model pruning. The project website including implementation code is available at https://foundation-model-research.github.io/NeuronPath/.