Left-right asymmetry in predicting brain activity from LLMs' representations emerges with their formal linguistic competence
arXiv:2602.12811v1 Announce Type: new Abstract: When humans and large language models (LLMs) process the same text, activations in the LLMs correlate with brain activity measured, e.g., with functional magnetic resonance imaging (fMRI). Moreover, it has been shown that, as the training of an LLM progresses, the performance in predicting brain activity from its internal activations improves more in the left hemisphere than in the right one. The aim of the present work is to understand which kind of competence acquired by the LLMs underlies the emergence of this left-right asymmetry. Using the OLMo-2 7B language model at various training checkpoints and fMRI data from English participants, we compare the evolution of the left-right asymmetry in brain scores alongside performance on several benchmarks. We observe that the asymmetry co-emerges with the formal linguistic abilities of the LLM. These abilities are demonstrated in two ways: by the model's capacity to assign a higher probabili
arXiv:2602.12811v1 Announce Type: new Abstract: When humans and large language models (LLMs) process the same text, activations in the LLMs correlate with brain activity measured, e.g., with functional magnetic resonance imaging (fMRI). Moreover, it has been shown that, as the training of an LLM progresses, the performance in predicting brain activity from its internal activations improves more in the left hemisphere than in the right one. The aim of the present work is to understand which kind of competence acquired by the LLMs underlies the emergence of this left-right asymmetry. Using the OLMo-2 7B language model at various training checkpoints and fMRI data from English participants, we compare the evolution of the left-right asymmetry in brain scores alongside performance on several benchmarks. We observe that the asymmetry co-emerges with the formal linguistic abilities of the LLM. These abilities are demonstrated in two ways: by the model's capacity to assign a higher probability to an acceptable sentence than to a grammatically unacceptable one within a minimal contrasting pair, or its ability to produce well-formed text. On the opposite, the left-right asymmetry does not correlate with the performance on arithmetic or Dyck language tasks; nor with text-based tasks involving world knowledge and reasoning. We generalize these results to another family of LLMs (Pythia) and another language, namely French. Our observations indicate that the left-right asymmetry in brain predictivity matches the progress in formal linguistic competence (knowledge of linguistic patterns).
Executive Summary
The article investigates the left-right asymmetry in predicting brain activity from large language models (LLMs) representations, focusing on the emergence of this asymmetry in relation to the LLMs' formal linguistic competence. Using the OLMo-2 7B language model and fMRI data from English participants, the study finds that the asymmetry in brain predictivity correlates with the model's formal linguistic abilities, such as grammar and sentence formation, but not with arithmetic, Dyck language tasks, or tasks involving world knowledge and reasoning. The findings are generalized to the Pythia family of LLMs and the French language, suggesting that the left-right asymmetry in brain predictivity is specifically tied to the progress in formal linguistic competence.
Key Points
- ▸ Left-right asymmetry in brain predictivity from LLMs emerges with formal linguistic competence.
- ▸ Asymmetry correlates with grammar and sentence formation abilities but not with arithmetic or reasoning tasks.
- ▸ Findings are consistent across different LLMs (OLMo-2, Pythia) and languages (English, French).
Merits
Rigorous Methodology
The study employs a robust experimental design, utilizing multiple LLMs and languages, which strengthens the generalizability of the findings.
Novel Insight
The article provides a novel perspective on the relationship between LLMs' linguistic competence and brain activity predictivity, offering valuable insights into cognitive processing.
Demerits
Limited Scope
The study focuses primarily on formal linguistic competence, potentially overlooking other cognitive processes that might influence brain predictivity.
Data Limitations
The reliance on fMRI data from a single language (English) for the primary analysis, despite generalization to French, may limit the broader applicability of the findings.
Expert Commentary
The article presents a compelling investigation into the relationship between LLMs' linguistic competence and brain activity predictivity. The rigorous methodology and the novel insights offered are significant contributions to the field. However, the study's focus on formal linguistic competence, while valuable, may overlook other cognitive processes that could also influence brain predictivity. The generalization to different LLMs and languages is a strength, but the reliance on fMRI data from a single language for the primary analysis could limit the broader applicability of the findings. The implications for AI development and cognitive neuroscience are profound, suggesting a need for further interdisciplinary research to fully understand the complexities of language processing in both human and artificial systems.
Recommendations
- ✓ Future research should explore the relationship between LLMs and brain activity predictivity in a broader range of cognitive tasks beyond formal linguistics.
- ✓ Incorporating diverse linguistic and cultural contexts in the analysis could enhance the generalizability of the findings.
