GateLoop: Fully Data-Controlled Linear Recurrence for Sequence Modeling
The article presents GateLoop, a sequence model that enhances linear recurrent models such as S4, S5, LRU, and RetNet by using data-controlled state transitions. GateLoop surpasses existing models in auto-regressive…
Continue reading