- source
- (Gauthier et al. 2021)
- tags
- Reservoir computing
DONE Summary
This paper bases itself on the demonstration that some reservoir computers (echo-state networks) are mathematically identical to nonlinear vector autoregression (NVAR) machines (Bollt 2021). A NVAR is just a regression over a feature vector composed of \(k\) time-delay observations of the dynamical system to be learned and nonlinear functions of these observations.
The authors introduce Next-Generation Reservoir computing (NG-RC) which is essentially a NVAR. Instead of the standard RC setup, they regress the outer layer over a vector
\[\mathbb{O}_{\text{total} }= c + \mathbb{O}_{\text{lin}} + \mathbb{O}_{\text{nonlin}}\]
where \(c\) is a constant, \(\mathbb{O}_{\text{lin}}\) is the vector of time delayed input observations and \(\mathbb{O}_{\text{nonlin}}\) is a vector of non-linear transformations of these observations.
Then, the output layer computes the NVAR output as a linear transformation of the feature vector, through \(\mathbf{Y}_i = \mathbf{W}_{\text{out}} \mathbb{O}_{\text{total}, i}\).
For a training dataset \(\mathbf{Y}_d\), output \(\mathbf{Y}\) is matched to it by solving a least-square linear regression problem.
The authors then evaluate the NG-RC model on two chaotic dynamical systems and show some performance scores and a few train/test trajectories comparisons.
DONE Comments
There doesn’t seem to be any comparison against standard RC, which is what this model is supposed to replace.
The benefits of reservoir computing probably lie beyond echo-state networks. With different reservoirs, one can harness the computations of complex non-linear dynamical systems which might not be possible using only recurrent neural networks.
Loading comments...