Compression Favors Consistency, Not Truth: When and Why Language Models Prefer Correct Information
arXiv:2603.11749v1 Announce Type: new Abstract: Why do language models sometimes prefer correct statements even when trained on mixed-quality data? We introduce the Compression--Consistency Principle: next-token …
Konstantin Krestnikov
7 views