Academic

Revisiting Real-Time Digging-In Effects: No Evidence from NP/Z Garden-Paths

arXiv:2603.23624v1 Announce Type: new Abstract: Digging-in effects, where disambiguation difficulty increases with longer ambiguous regions, have been cited as evidence for self-organized sentence processing, in which structural commitments strengthen over time. In contrast, surprisal theory predicts no such effect unless lengthening genuinely shifts statistical expectations, and neural language models appear to show the opposite pattern. Whether digging-in is a robust real-time phenomenon in human sentence processing -- or an artifact of wrap-up processes or methodological confounds -- remains unclear. We report two experiments on English NP/Z garden-path sentences using Maze and self-paced reading, comparing human behavior with predictions from an ensemble of large language models. We find no evidence for real-time digging-in effects. Critically, items with sentence-final versus nonfinal disambiguation show qualitatively different patterns: positive digging-in trends appear only sen

A
Amani Maina-Kilaas, Roger Levy · · 1 min read · 21 views
#cs.CL

arXiv:2603.23624v1 Announce Type: new Abstract: Digging-in effects, where disambiguation difficulty increases with longer ambiguous regions, have been cited as evidence for self-organized sentence processing, in which structural commitments strengthen over time. In contrast, surprisal theory predicts no such effect unless lengthening genuinely shifts statistical expectations, and neural language models appear to show the opposite pattern. Whether digging-in is a robust real-time phenomenon in human sentence processing -- or an artifact of wrap-up processes or methodological confounds -- remains unclear. We report two experiments on English NP/Z garden-path sentences using Maze and self-paced reading, comparing human behavior with predictions from an ensemble of large language models. We find no evidence for real-time digging-in effects. Critically, items with sentence-final versus nonfinal disambiguation show qualitatively different patterns: positive digging-in trends appear only sentence-finally, where wrap-up effects confound interpretation. Nonfinal items -- the cleaner test of real-time processing -- show reverse trends consistent with neural model predictions.

Executive Summary

This article presents a thorough investigation into the phenomenon of real-time digging-in effects in human sentence processing. The authors conducted two experiments on English NP/Z garden-path sentences, utilizing Maze and self-paced reading, and compared human behavior with predictions from large language models. The results indicate no evidence for real-time digging-in effects, with positive trends appearing only in sentence-final disambiguation. Critically, nonfinal items, which provide a cleaner test of real-time processing, show reverse trends consistent with neural model predictions. This study provides significant insights into the nature of human sentence processing and its relationship with neural language models.

Key Points

  • ▸ The article challenges the existence of real-time digging-in effects in human sentence processing.
  • ▸ The authors present two experiments on English NP/Z garden-path sentences using Maze and self-paced reading.
  • ▸ The results show no evidence for real-time digging-in effects, with positive trends appearing only in sentence-final disambiguation.

Merits

Strength of Experimental Design

The authors employed a robust experimental design, utilizing both Maze and self-paced reading, to investigate real-time digging-in effects.

Comparison with Neural Language Models

The study's comparison with large language model predictions provides valuable insights into the relationship between human sentence processing and neural language models.

Demerits

Limited Generalizability

The study's findings may be limited to NP/Z garden-path sentences and may not generalize to other types of sentences or linguistic structures.

Potential for Wrap-Up Effects

The authors acknowledge the potential for wrap-up effects to confound interpretation, particularly in sentence-final disambiguation.

Expert Commentary

The article presents a significant contribution to the field of sentence processing, challenging the existence of real-time digging-in effects. The authors' use of a robust experimental design and comparison with large language model predictions provides valuable insights into the relationship between human sentence processing and neural language models. However, the study's findings may be limited to NP/Z garden-path sentences and may not generalize to other types of sentences or linguistic structures. Additionally, the potential for wrap-up effects to confound interpretation, particularly in sentence-final disambiguation, is a significant limitation of the study. Overall, the article provides a nuanced understanding of sentence processing and its relationship with neural language models.

Recommendations

  • ✓ Future studies should investigate the existence of real-time digging-in effects in other types of sentences and linguistic structures.
  • ✓ Researchers should develop more sophisticated methods to control for wrap-up effects and sentence finality in sentence processing experiments.

Sources

Original: arXiv - cs.CL

Related Articles