Sequence Modeling

The article presents GateLoop, a sequence model that enhances linear recurrent models such as S4, S5, LRU, and RetNet by using data-controlled state transitions. GateLoop surpasses existing models in auto-regressive…