KEPo: Knowledge Evolution Poison on Graph-based Retrieval-Augmented Generation
arXiv:2603.11501v1 Announce Type: new Abstract: Graph-based Retrieval-Augmented Generation (GraphRAG) constructs the Knowledge Graph (KG) from external databases to enhance the timeliness and accuracy of Large Language Model (LLM) generations.However,this reliance on external data introduces new attack surfaces.Attackers can inject poisoned texts into databases to manipulate LLMs into producing harmful target responses for attacker-chosen queries.Existing research primarily focuses on attacking conventional RAG systems.However,such methods are ineffective against GraphRAG.This robustness derives from the KG abstraction of GraphRAG,which reorganizes injected text into a graph before retrieval,thereby enabling the LLM to reason based on the restructured context instead of raw poisoned passages.To expose latent security vulnerabilities in GraphRAG,we propose Knowledge Evolution Poison (KEPo),a novel poisoning attack method specifically designed for GraphRAG.For each target query,KEPo fir
arXiv:2603.11501v1 Announce Type: new Abstract: Graph-based Retrieval-Augmented Generation (GraphRAG) constructs the Knowledge Graph (KG) from external databases to enhance the timeliness and accuracy of Large Language Model (LLM) generations.However,this reliance on external data introduces new attack surfaces.Attackers can inject poisoned texts into databases to manipulate LLMs into producing harmful target responses for attacker-chosen queries.Existing research primarily focuses on attacking conventional RAG systems.However,such methods are ineffective against GraphRAG.This robustness derives from the KG abstraction of GraphRAG,which reorganizes injected text into a graph before retrieval,thereby enabling the LLM to reason based on the restructured context instead of raw poisoned passages.To expose latent security vulnerabilities in GraphRAG,we propose Knowledge Evolution Poison (KEPo),a novel poisoning attack method specifically designed for GraphRAG.For each target query,KEPo first generates a toxic event containing poisoned knowledge based on the target answer.By fabricating event backgrounds and forging knowledge evolution paths from original facts to the toxic event,it then poisons the KG and misleads the LLM into treating the poisoned knowledge as the final result.In multi-target attack scenarios,KEPo further connects multiple attack corpora,enabling their poisoned knowledge to mutually reinforce while expanding the scale of poisoned communities,thereby amplifying attack effectiveness.Experimental results across multiple datasets demonstrate that KEPo achieves state-of-the-art attack success rates for both single-target and multi-target attacks,significantly outperforming previous methods.
Executive Summary
The article proposes a novel poisoning attack method, Knowledge Evolution Poison (KEPo), specifically designed for Graph-based Retrieval-Augmented Generation (GraphRAG) systems. GraphRAG constructs a Knowledge Graph (KG) from external databases to enhance the accuracy and timeliness of Large Language Model (LLM) generations. However, this reliance on external data introduces new attack surfaces, which existing research has failed to address effectively. KEPo generates a toxic event containing poisoned knowledge based on the target answer, fabricates event backgrounds, and forges knowledge evolution paths to mislead the LLM into treating the poisoned knowledge as the final result. Experimental results demonstrate that KEPo achieves state-of-the-art attack success rates for both single-target and multi-target attacks, significantly outperforming previous methods. This breakthrough highlights the need for robust security measures in GraphRAG systems to prevent poisoning attacks.
Key Points
- ▸ KEPo is a novel poisoning attack method specifically designed for GraphRAG systems.
- ▸ GraphRAG systems construct a KG from external databases to enhance LLM generations.
- ▸ Existing research has failed to address the attack surfaces introduced by GraphRAG effectively.
- ▸ KEPo generates a toxic event containing poisoned knowledge based on the target answer.
- ▸ KEPo achieves state-of-the-art attack success rates for both single-target and multi-target attacks.
Merits
Robustness
KEPo's novel approach to poisoning attacks demonstrates its robustness against existing defense mechanisms.
Effectiveness
Experimental results show that KEPo achieves state-of-the-art attack success rates for both single-target and multi-target attacks.
Scalability
KEPo's ability to connect multiple attack corpora enables the poisoned knowledge to mutually reinforce and expand the scale of poisoned communities.
Demerits
Limited scope
KEPo is specifically designed for GraphRAG systems, which may limit its applicability to other types of LLM generations.
Potential for over-reliance on external data
The reliance on external data in GraphRAG systems may introduce new attack surfaces and vulnerabilities.
Expert Commentary
The article's novel approach to poisoning attacks and its experimental results demonstrate a significant breakthrough in the field of LLM security. However, the limited scope of KEPo's applicability and the potential for over-reliance on external data in GraphRAG systems raise concerns about the overall security of these systems. To address these concerns, researchers and developers should prioritize the development of robust security measures and guidelines for the construction and maintenance of KGs. Furthermore, policymakers and regulatory bodies should consider implementing stricter guidelines and regulations for the development and deployment of GraphRAG systems to prevent poisoning attacks and ensure the safety and security of users.
Recommendations
- ✓ Develop robust security measures and guidelines for the construction and maintenance of KGs.
- ✓ Implement stricter guidelines and regulations for the development and deployment of GraphRAG systems.