強健且細緻的AI生成文本檢測

Robust and Fine-Grained Detection of AI Generated Texts

April 16, 2025

作者: Ram Mohan Rao Kadiyala, Siddartha Pullakhandam, Kanwal Mehreen, Drishti Sharma, Siddhant Gupta, Jebish Purbey, Ashay Srivastava, Subhasya TippaReddy, Arvind Reddy Bobbili, Suraj Telugara Chandrashekhar, Modabbir Adeeb, Srinadh Vura, Hamza Farooq

cs.AI

摘要

理想的機器生成內容檢測系統應能有效應對任何生成器，因為日益先進的大型語言模型（LLM）不斷湧現。現有系統在準確識別短文本中的AI生成內容方面往往力不從心。此外，並非所有文本都完全由人類或LLM創作，因此我們更關注部分情況，即人機協作撰寫的文本。本文介紹了一組專為標記分類任務構建的模型，這些模型在大量人機協作文本上進行訓練，並在未見領域、未見生成器、非母語者撰寫的文本以及對抗性輸入的文本上表現出色。我們還引入了一個包含超過240萬條此類文本的新數據集，這些文本主要由多個流行的專有LLM在23種語言中協作完成。我們還展示了模型在各領域和各生成器文本上的性能表現。其他發現包括與每種對抗方法的性能比較、輸入文本的長度，以及生成文本與原始人類撰寫文本的特徵對比。

English

An ideal detection system for machine generated content is supposed to work well on any generator as many more advanced LLMs come into existence day by day. Existing systems often struggle with accurately identifying AI-generated content over shorter texts. Further, not all texts might be entirely authored by a human or LLM, hence we focused more over partial cases i.e human-LLM co-authored texts. Our paper introduces a set of models built for the task of token classification which are trained on an extensive collection of human-machine co-authored texts, which performed well over texts of unseen domains, unseen generators, texts by non-native speakers and those with adversarial inputs. We also introduce a new dataset of over 2.4M such texts mostly co-authored by several popular proprietary LLMs over 23 languages. We also present findings of our models' performance over each texts of each domain and generator. Additional findings include comparison of performance against each adversarial method, length of input texts and characteristics of generated texts compared to the original human authored texts.