This repository contains the papers published in the research area of non-saturating recurrent units. The first paper published in this area, to the best of my knowledge, is the Statistical Recurrent Unit (SRU) [1]. Fourier Recurrent Unit [2] and Non-Saturating Recurrunt Unit (NRU)[3] are two proposed RNNs which improve on the original SRU.
[1] Oliva, Junier B., Barnabás Póczos, and Jeff Schneider. "The statistical recurrent unit." International Conference on Machine Learning. PMLR, 2017.
[2] Zhang, Jiong, et al. "Learning long term dependencies via fourier recurrent units." International Conference on Machine Learning. PMLR, 2018.
[3] Chandar, Sarath, et al. "Towards non-saturating recurrent units for modelling long-term dependencies." Proceedings of the AAAI Conference on Artificial Intelligence. Vol. 33. No. 01. 2019.