缺失前提加劇過度思考：推理模型是否正在喪失批判性思維能力？

Missing Premise exacerbates Overthinking: Are Reasoning Models losing Critical Thinking Skill?

April 9, 2025

作者: Chenrui Fan, Ming Li, Lichao Sun, Tianyi Zhou

cs.AI

摘要

我們發現，無論是通過強化學習還是監督學習訓練的推理大語言模型（LLMs），在面對前提缺失（MiP）的不適定問題時，其回應長度會急劇增加，最終產生冗長且無效的思考。這一新引入的情境在很大程度上加劇了普遍的過度思考問題，我們稱之為MiP-過度思考。此類失敗違背了「測試時縮放定律」，但在我們精心構建的多個MiP數據集上廣泛觀察到，揭示了廉價過度思考的危害以及批判性思維的缺乏。令人驚訝的是，未專門針對推理進行訓練的LLMs在MiP情境下表現得更好，生成的回應更短，能迅速識別不適定的查詢。這暗示了當前推理LLMs訓練方案的一個關鍵缺陷，即未能充分鼓勵高效思考，導致思維模式的濫用。為了深入探究這些失敗背後的原因，我們對不同類型LLMs的推理長度、過度思考模式及關鍵思維的位置進行了細緻分析。此外，我們擴展的消融研究表明，過度思考通過推理模型回應的蒸餾具有傳染性。這些結果增進了對過度思考的理解，並為緩解該問題提供了新的見解。

English

We find that the response length of reasoning LLMs, whether trained by reinforcement learning or supervised learning, drastically increases for ill-posed questions with missing premises (MiP), ending up with redundant and ineffective thinking. This newly introduced scenario exacerbates the general overthinking issue to a large extent, which we name as the MiP-Overthinking. Such failures are against the ``test-time scaling law'' but have been widely observed on multiple datasets we curated with MiP, indicating the harm of cheap overthinking and a lack of critical thinking. Surprisingly, LLMs not specifically trained for reasoning exhibit much better performance on the MiP scenario, producing much shorter responses that quickly identify ill-posed queries. This implies a critical flaw of the current training recipe for reasoning LLMs, which does not encourage efficient thinking adequately, leading to the abuse of thinking patterns. To further investigate the reasons behind such failures, we conduct fine-grained analyses of the reasoning length, overthinking patterns, and location of critical thinking on different types of LLMs. Moreover, our extended ablation study reveals that the overthinking is contagious through the distillation of reasoning models' responses. These results improve the understanding of overthinking and shed novel insights into mitigating the problem.