Theory of Temporal Pattern Learning in Echo State Networks
Hakim, V.; Karma, A.
Show abstract
Echo state networks are well-known for their ability to learn temporal patterns through simple feedback to a large recurrent network with random connections. However, the learning process itself remains poorly understood. We develop a quantitative theory that explains learning in a regime where the network dynamics is stable and the feedback is weak. We show that the dynamics is governed by a finite number of master modes whose nonlinear interactions can be described by a normal form. This formulation provides a simple picture of learning as a Fourier decomposition of the target pattern with amplitudes determined by nonlinear interactions that, remarkably, become independent of the network randomness in the limit of large network size. We further show that the description extends to moderate feedback and recurrent networks with multiple unstable modes.
Matching journals
The top 5 journals account for 50% of the predicted probability mass.
Similar papers in this journal
Similar papers in this journal
"Similar papers" are the closest papers from that journal in the model's embedding space. They show what the match is built on, but the ranking comes mostly from a classifier over the whole training set, not from these examples alone.