From Weak Cues to Real Identities: Evaluating Inference-Driven De-Anonymization in LLM Agents
arXiv:2603.18382v1 Announce Type: new Abstract: Anonymization is widely treated as a practical safeguard because re-identifying anonymous records was historically costly, requiring domain expertise, tailored algorithms, and manual corroboration. We study a growing privacy risk that may weaken this barrier: LLM-based agents can autonomously reconstruct real-world identities from scattered, individually non-identifying cues. By combining these sparse cues with public information, agents resolve identities without bespoke engineering. We formalize this threat as \emph{inference-driven linkage} and systematically evaluate it across three settings: classical linkage scenarios (Netflix and AOL), \emph{InferLink} (a controlled benchmark varying task intent, shared cues, and attacker knowledge), and modern text-rich artifacts. Without task-specific heuristics, agents successfully execute both fixed-pool matching and open-ended identity resolution. In the Netflix Prize setting, an agent recons
arXiv:2603.18382v1 Announce Type: new Abstract: Anonymization is widely treated as a practical safeguard because re-identifying anonymous records was historically costly, requiring domain expertise, tailored algorithms, and manual corroboration. We study a growing privacy risk that may weaken this barrier: LLM-based agents can autonomously reconstruct real-world identities from scattered, individually non-identifying cues. By combining these sparse cues with public information, agents resolve identities without bespoke engineering. We formalize this threat as \emph{inference-driven linkage} and systematically evaluate it across three settings: classical linkage scenarios (Netflix and AOL), \emph{InferLink} (a controlled benchmark varying task intent, shared cues, and attacker knowledge), and modern text-rich artifacts. Without task-specific heuristics, agents successfully execute both fixed-pool matching and open-ended identity resolution. In the Netflix Prize setting, an agent reconstructs 79.2\% of identities, significantly outperforming a 56.0\% classical baseline. Furthermore, linkage emerges not only under explicit adversarial prompts but also as a byproduct of benign cross-source analysis in \emph{InferLink} and unstructured research narratives. These findings establish that identity inference -- not merely explicit information disclosure -- must be treated as a first-class privacy risk; evaluations must measure what identities an agent can infer.
Executive Summary
This article presents a comprehensive study on the growing risk of inference-driven de-anonymization in Large Language Model (LLM) agents, which can autonomously reconstruct real-world identities from scattered, non-identifying cues. The authors formalize this threat as 'inference-driven linkage' and evaluate it across three settings, demonstrating that agents can successfully execute both fixed-pool matching and open-ended identity resolution. The study highlights the need to treat identity inference as a first-class privacy risk, emphasizing the importance of evaluating what identities an agent can infer. The findings have significant implications for both practical applications and policy-making, underscoring the need for novel approaches to safeguarding anonymity in the face of emerging LLM-based threats.
Key Points
- ▸ Inference-driven de-anonymization poses a significant threat to anonymity in the era of LLM agents.
- ▸ LLM agents can autonomously reconstruct real-world identities from scattered, non-identifying cues.
- ▸ Identity inference must be treated as a first-class privacy risk, and evaluations must measure what identities an agent can infer.
Merits
Strength
The study's comprehensive evaluation across three settings provides robust insights into the capabilities of LLM agents and the risks associated with inference-driven de-anonymization.
Methodological Rigor
The authors' formalization of the threat as 'inference-driven linkage' and their systematic evaluation approach establish a solid foundation for future research in this area.
Practical Relevance
The study's findings have significant implications for both practical applications and policy-making, highlighting the need for novel approaches to safeguarding anonymity in the face of emerging LLM-based threats.
Demerits
Limitation
The study primarily focuses on the capabilities of LLM agents, and future research should explore the development of countermeasures and mitigation strategies to address the identified risks.
Scalability
The study's evaluation is limited to relatively small-scale settings, and future research should investigate the scalability of LLM agents' capabilities and the potential impact of larger-scale inference-driven de-anonymization.
Interdisciplinary Collaboration
The study could benefit from interdisciplinary collaboration, engaging with experts from fields such as computer science, law, and social sciences to develop a more comprehensive understanding of the risks and implications associated with inference-driven de-anonymization.
Expert Commentary
The study presents a timely and comprehensive analysis of the risks associated with inference-driven de-anonymization in LLM agents. The authors' formalization of the threat as 'inference-driven linkage' and their systematic evaluation approach establish a solid foundation for future research in this area. The study's findings have significant implications for both practical applications and policy-making, highlighting the need for novel approaches to safeguarding anonymity in the face of emerging LLM-based threats. However, the study's limitations, such as its focus on relatively small-scale settings and the need for interdisciplinary collaboration, should be addressed in future research. Overall, the study contributes significantly to the ongoing conversation on the intersection of AI, surveillance, and anonymity, and its findings have far-reaching implications for the development of novel countermeasures and mitigation strategies.
Recommendations
- ✓ Future research should focus on developing novel countermeasures and mitigation strategies to address the risks associated with inference-driven de-anonymization.
- ✓ Interdisciplinary collaboration should be encouraged to develop a more comprehensive understanding of the risks and implications associated with inference-driven de-anonymization.
- ✓ Policymakers should develop novel regulations and guidelines to address the risks associated with inference-driven de-anonymization.
