关于大型语言模型中的关系特定神经元

On Relation-Specific Neurons in Large Language Models

February 24, 2025

作者: Yihong Liu, Runsheng Chen, Lea Hirlimann, Ahmad Dawar Hakimi, Mingyang Wang, Amir Hossein Kargaran, Sascha Rothe, François Yvon, Hinrich Schütze

cs.AI

摘要

在大型语言模型（LLMs）中，某些神经元可以存储在预训练期间学到的不同知识片段。虽然知识通常表现为关系和实体的组合，但目前尚不清楚是否有些神经元专注于关系本身 -- 而与任何实体无关。我们假设这样的神经元可以检测输入文本中的关系，并指导涉及这种关系的生成。为了研究这一点，我们使用基于统计的方法在选择的一组关系上研究了Llama-2系列。我们的实验证明了关系特定神经元的存在。我们测量了有选择性地停用与关系r特定的候选神经元对LLM处理以下内容的能力的影响：（1）其关系为r的事实和（2）其关系为不同关系r'（r不等于r'）的事实。关于它们对编码关系信息的能力，我们提供了关于关系特定神经元的以下三个属性的证据。 （i）神经元累积性。与r相关的神经元具有累积效应，因此停用其中更大一部分会导致r中更多事实的退化。 （ii）神经元多功能性。神经元可以跨多个密切相关和不太相关的关系共享。一些关系神经元可以跨语言传递。 （iii）神经元干扰。停用特定于一个关系的神经元可以提高LLM对其他关系事实的生成性能。我们将在以下网址公开我们的代码：https://github.com/cisnlp/relation-specific-neurons。

English

In large language models (LLMs), certain neurons can store distinct pieces of knowledge learned during pretraining. While knowledge typically appears as a combination of relations and entities, it remains unclear whether some neurons focus on a relation itself -- independent of any entity. We hypothesize such neurons detect a relation in the input text and guide generation involving such a relation. To investigate this, we study the Llama-2 family on a chosen set of relations with a statistics-based method. Our experiments demonstrate the existence of relation-specific neurons. We measure the effect of selectively deactivating candidate neurons specific to relation r on the LLM's ability to handle (1) facts whose relation is r and (2) facts whose relation is a different relation r' neq r. With respect to their capacity for encoding relation information, we give evidence for the following three properties of relation-specific neurons. (i) Neuron cumulativity. The neurons for r present a cumulative effect so that deactivating a larger portion of them results in the degradation of more facts in r. (ii) Neuron versatility. Neurons can be shared across multiple closely related as well as less related relations. Some relation neurons transfer across languages. (iii) Neuron interference. Deactivating neurons specific to one relation can improve LLM generation performance for facts of other relations. We will make our code publicly available at https://github.com/cisnlp/relation-specific-neurons.