Frayed RoPE and Long Inputs: A Geometric Perspective
arXiv:2603.18017v1 Announce Type: new Abstract: Rotary Positional Embedding (RoPE) is a widely adopted technique for encoding position in language models, which, while effective, causes performance …
Davis Wertheimer, Aozhong Zhang, Derrick Liu, Penghang Yin, Naigang Wang
6 views
