LM的组合泛化和虚构中的线性相关

摘要

语言模型（LMs）的泛化正在经历积极的讨论，对比它们在通用智能方面的潜力与它们在基本知识组合（例如，逆/过渡诅咒）方面的挣扎。本文揭示了LMs在知识组合过程中的线性相关现象。举例来说，存在一种线性转换，将某些相关知识映射到下一个令牌预测logits，从一个提示到另一个提示，例如，“X lives in the city of” 转变为 “X lives in the country of” 对于每个给定的X。这反映了人类知识组合中的线性关系，比如 Paris 转变为 France。我们的发现表明，这种线性转换对大规模微调具有韧性，当与现实世界关系一致时，泛化更新的知识，但当偏离时会导致幻觉。实证结果表明，线性相关性可以作为LM泛化的潜在标识符。最后，我们展示这种线性相关性可以通过单个前馈网络和预训练的词汇表示来学习，表明LM的泛化在很大程度上依赖于后者。

English

The generalization of language models (LMs) is undergoing active debates, contrasting their potential for general intelligence with their struggles with basic knowledge composition (e.g., reverse/transition curse). This paper uncovers the phenomenon of linear correlations in LMs during knowledge composition. For explanation, there exists a linear transformation between certain related knowledge that maps the next token prediction logits from one prompt to another, e.g., "X lives in the city of" rightarrow "X lives in the country of" for every given X. This mirrors the linearity in human knowledge composition, such as Paris rightarrow France. Our findings indicate that the linear transformation is resilient to large-scale fine-tuning, generalizing updated knowledge when aligned with real-world relationships, but causing hallucinations when it deviates. Empirical results suggest that linear correlation can serve as a potential identifier of LM's generalization. Finally, we show such linear correlations can be learned with a single feedforward network and pre-trained vocabulary representations, indicating LM generalization heavily relies on the latter.

LM的组合泛化和虚构中的线性相关

Linear Correlation in LM's Compositional Generalization and Hallucination

摘要

Summary

Support