Assessing Cognitive Biases in LLMs for Judicial Decision Support: Virtuous Victim and Halo Effects
arXiv:2603.10016v1 Announce Type: cross Abstract: We investigate whether large language models (LLMs) display human-like cognitive biases, focusing on potential implications for assistance in judicial sentencing, a decision-making system where fairness is paramount. Two of the most relevant biases were chosen: the virtuous victim effect (VVE), with emphasis given to its reduction when adjacent consent is present, and prestige-based halo effects (occupation, company, and credentials). Using vignettes that were altered from prior literature to avoid LLMs recalling from their training data, we isolate each manipulation by holding all other details consistent, then measuring the percentage difference in outcomes. Five models were evaluated as representative LLMs in independent multi-run trials per condition (ChatGPT 5 Instant, ChatGPT 5 Thinking, DeepSeek V3.1, Claude Sonnet 4, Gemini 2.5 Flash). Our research discovers that there is larger VVE, there is no statistically significant penalt
Video Coverage
LLMs, Cognitive Biases, and Judicial Decision Support: What Does the Future Hold?
arXiv:2603.10016v1 Announce Type: cross Abstract: We investigate whether large language models (LLMs) display human-like cognitive biases, focusing on potential implications for assistance in judicial sentencing, a decision-making system where fairness is paramount. Two of the most relevant biases were chosen: the virtuous victim effect (VVE), with emphasis given to its reduction when adjacent consent is present, and prestige-based halo effects (occupation, company, and credentials). Using vignettes that were altered from prior literature to avoid LLMs recalling from their training data, we isolate each manipulation by holding all other details consistent, then measuring the percentage difference in outcomes. Five models were evaluated as representative LLMs in independent multi-run trials per condition (ChatGPT 5 Instant, ChatGPT 5 Thinking, DeepSeek V3.1, Claude Sonnet 4, Gemini 2.5 Flash). Our research discovers that there is larger VVE, there is no statistically significant penalty for adjacent-consent, and the halo effect is slightly reduced when compared to humans, with an exception for credential based prestige, which had a large reduction. Despite the variation across different models and outputs restricting current judicial usage, there were modest improvements compared to human benchmarks.
Executive Summary
This study examines the presence of cognitive biases in large language models (LLMs) and their implications for judicial decision support. The research focuses on the virtuous victim effect and halo effects, finding that LLMs exhibit these biases, albeit with some variations compared to human benchmarks. The study suggests that while LLMs can be useful tools, their current limitations restrict their application in judicial settings, highlighting the need for further refinement and evaluation.
Key Points
- ▸ LLMs exhibit cognitive biases similar to humans, including the virtuous victim effect and halo effects
- ▸ The presence of adjacent consent does not significantly reduce the virtuous victim effect in LLMs
- ▸ Credential-based prestige has a large reduction in halo effect compared to human benchmarks
Merits
Comprehensive Evaluation
The study evaluates multiple LLMs, providing a comprehensive understanding of their biases and limitations
Methodological Rigor
The use of vignettes and controlled experiments allows for a nuanced analysis of LLM biases
Demerits
Limited Generalizability
The study's findings may not be generalizable to all LLMs or judicial decision-making contexts
Variation Across Models
The significant variation in results across different LLMs may limit their practical application
Expert Commentary
This study contributes significantly to our understanding of LLM biases and their implications for judicial decision support. The findings suggest that while LLMs have the potential to support fair and informed decision-making, their limitations and variations must be carefully addressed through further research and development. The study's methodological rigor and comprehensive evaluation of multiple LLMs provide a foundation for future research and refinement of these models.
Recommendations
- ✓ Further research into the development of LLMs that can mitigate cognitive biases and ensure fairness in judicial decision-making
- ✓ The establishment of regulatory frameworks and guidelines for the use of LLMs in judicial decision support to ensure transparency and accountability
