ShieldAgent：通過可驗證的安全策略推理來保護代理

ShieldAgent: Shielding Agents via Verifiable Safety Policy Reasoning

March 26, 2025

作者: Zhaorun Chen, Mintong Kang, Bo Li

cs.AI

摘要

基於基礎模型的自動化代理已在多種現實應用中廣泛採用。然而，這些代理仍極易受到惡意指令和攻擊的影響，可能導致隱私洩露和財務損失等嚴重後果。更為關鍵的是，由於代理的複雜性和動態性，現有的大型語言模型（LLM）防護措施並不適用。為應對這些挑戰，我們提出了ShieldAgent，這是首個旨在通過邏輯推理來確保其他受保護代理的行動軌跡符合明確安全策略的防護代理。具體而言，ShieldAgent首先從策略文件中提取可驗證的規則，並將其結構化為一組基於行動的概率規則電路，從而構建安全策略模型。針對受保護代理的行動軌跡，ShieldAgent檢索相關的規則電路，並利用其全面的工具庫和可執行代碼生成防護計劃，進行形式化驗證。此外，鑑於缺乏針對代理的防護基準，我們引入了ShieldAgent-Bench，這是一個包含3,000對與安全相關的代理指令和行動軌跡的數據集，這些數據通過在6個網絡環境和7個風險類別中的最先進攻擊收集而來。實驗表明，ShieldAgent在ShieldAgent-Bench和三個現有基準上達到了最先進水平，平均優於先前方法11.3%，並實現了90.1%的高召回率。此外，ShieldAgent將API查詢減少了64.7%，推理時間縮短了58.2%，展示了其在保護代理方面的高精度和高效性。

English

Autonomous agents powered by foundation models have seen widespread adoption across various real-world applications. However, they remain highly vulnerable to malicious instructions and attacks, which can result in severe consequences such as privacy breaches and financial losses. More critically, existing guardrails for LLMs are not applicable due to the complex and dynamic nature of agents. To tackle these challenges, we propose ShieldAgent, the first guardrail agent designed to enforce explicit safety policy compliance for the action trajectory of other protected agents through logical reasoning. Specifically, ShieldAgent first constructs a safety policy model by extracting verifiable rules from policy documents and structuring them into a set of action-based probabilistic rule circuits. Given the action trajectory of the protected agent, ShieldAgent retrieves relevant rule circuits and generates a shielding plan, leveraging its comprehensive tool library and executable code for formal verification. In addition, given the lack of guardrail benchmarks for agents, we introduce ShieldAgent-Bench, a dataset with 3K safety-related pairs of agent instructions and action trajectories, collected via SOTA attacks across 6 web environments and 7 risk categories. Experiments show that ShieldAgent achieves SOTA on ShieldAgent-Bench and three existing benchmarks, outperforming prior methods by 11.3% on average with a high recall of 90.1%. Additionally, ShieldAgent reduces API queries by 64.7% and inference time by 58.2%, demonstrating its high precision and efficiency in safeguarding agents.