基于LLM的鲁棒多比特文本水印技术

Robust Multi-bit Text Watermark with LLM-based Paraphrasers

December 4, 2024

作者: Xiaojun Xu, Jinghan Jia, Yuanshun Yao, Yang Liu, Hang Li

cs.AI

摘要

我们提出了一种通过使用LLM进行改写嵌入的难以察觉的多比特文本水印方案。我们微调了一对LLM改写器，这些改写器被设计成行为不同，以便它们在文本语义中反映的改写差异可以被训练有素的解码器识别。为了嵌入我们的多比特水印，我们交替使用两个改写器在句子级别对预定义的二进制代码进行编码。然后，我们使用文本分类器作为解码器来解码水印的每个比特。通过大量实验证明，我们的水印可以在保留原始句子的语义信息的同时，利用小型（1.1B）文本改写器实现超过99.99\%的检测AUC。更重要的是，我们的流程在词替换和句子改写扰动下表现出鲁棒性，并且很好地推广到超出分布范围的数据。我们还展示了基于LLM的评估显示我们水印的隐蔽性。我们已经开源了代码：https://github.com/xiaojunxu/multi-bit-text-watermark。

English

We propose an imperceptible multi-bit text watermark embedded by paraphrasing with LLMs. We fine-tune a pair of LLM paraphrasers that are designed to behave differently so that their paraphrasing difference reflected in the text semantics can be identified by a trained decoder. To embed our multi-bit watermark, we use two paraphrasers alternatively to encode the pre-defined binary code at the sentence level. Then we use a text classifier as the decoder to decode each bit of the watermark. Through extensive experiments, we show that our watermarks can achieve over 99.99\% detection AUC with small (1.1B) text paraphrasers while keeping the semantic information of the original sentence. More importantly, our pipeline is robust under word substitution and sentence paraphrasing perturbations and generalizes well to out-of-distributional data. We also show the stealthiness of our watermark with LLM-based evaluation. We open-source the code: https://github.com/xiaojunxu/multi-bit-text-watermark.