Orthogonal Laguerre Recurrent Neural Networks

Sergio A. Dorado-Rojas, Bhanukiran Vinzamuri, and Luigi Vanfretti

• Author's copy

• URL

bib

@inproceedings{dorado-rojas2020,
  title = {Orthogonal {{Laguerre Recurrent Neural Networks}}},
  booktitle = {Machine {{Learning}} and the {{Physical Sciences Workshop}} at the 34th {{Annual Conference}} on {{Neural Information Processing Systems}}},
  author = {{Dorado-Rojas}, Sergio A. and Vinzamuri, Bhanukiran and Vanfretti, Luigi},
  year = 2020,
  month = dec
}

Abstract

The inability of RNN architectures to incorporate stable discrete-time linear time invariant dynamics has been a long-standing problem that has also affected their performance in practice. To mitigate this problem, in this paper, we propose a RNN architecture that embeds a linear time-invariant system basis comprising of Laguerre polynomials inside its structure. Laguerre functions are a family of Eigenfunctions arising from the Sturm-Liouville problem characterized by their orthonormality. Our RNN uses the embedded full state-space representation provided by such orthonormal functions in two different variants: a general orthogonal Laguerre network and a Ladder network. The state of such systems is used as a means to preserve input information for more extended periods by an orthogonal encoding. Both variants are enhanced by a non-linear static output layer that projects the state, the memory, and the past output for the hidden state. The proposed models are benchmarked exhaustively against other dynamical systems inspired-RNN's on two physics-based benchmarks that demonstrate their better performance.

Back