混沌邊緣的智能

Intelligence at the Edge of Chaos

October 3, 2024

作者: Shiyang Zhang, Aakash Patel, Syed A Rizvi, Nianchen Liu, Sizhuang He, Amin Karbasi, Emanuele Zappala, David van Dijk

cs.AI

摘要

通過研究基於規則系統的複雜性如何影響訓練模型以預測這些規則的能力，我們探索了人工系統中智能行為的出現。我們的研究聚焦於基本元胞自動機（ECA），這是一維系統，生成的行為從簡單到高度複雜不等。通過在不同的ECA上訓練不同的大型語言模型（LLMs），我們評估了規則行為的複雜性與LLMs展現的智能之間的關係，這在它們在下游任務上的表現中得以體現。我們的研究結果顯示，具有更高複雜性的規則導致模型展現出更高的智能，這表現在它們在推理和棋藝預測任務上的表現。無論是均勻的、周期性的系統，還是高度混沌的系統，都導致下游表現較差，突顯了有利於智能的複雜性的平衡點。我們推測智能來自於預測複雜性的能力，並且創造智能可能僅需要接觸複雜性。

English

We explore the emergence of intelligent behavior in artificial systems by investigating how the complexity of rule-based systems influences the capabilities of models trained to predict these rules. Our study focuses on elementary cellular automata (ECA), simple yet powerful one-dimensional systems that generate behaviors ranging from trivial to highly complex. By training distinct Large Language Models (LLMs) on different ECAs, we evaluated the relationship between the complexity of the rules' behavior and the intelligence exhibited by the LLMs, as reflected in their performance on downstream tasks. Our findings reveal that rules with higher complexity lead to models exhibiting greater intelligence, as demonstrated by their performance on reasoning and chess move prediction tasks. Both uniform and periodic systems, and often also highly chaotic systems, resulted in poorer downstream performance, highlighting a sweet spot of complexity conducive to intelligence. We conjecture that intelligence arises from the ability to predict complexity and that creating intelligence may require only exposure to complexity.