The extension of Recurrent Neural Networks (RNNs) in the direction of deep learning is a topic that is gaining more and more attention in the neural networks community. The study of deep RNNs opened a number of intriguing research questions on the actual role played by layering in the architectural design of RNNs. Recently, the introduction of the Deep Echo State Network (DeepESN) model allowed to start addressing such open issues in literature, contributing to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers. This contribution intends to present a unified view over the major advancements in the study of DeepESNs, enabling to directly point out the natural advantages of a layered construction of recurrent networks for temporal data processing.

Why Layering in Recurrent Neural Networks? A DeepESN Survey

Gallicchio, Claudio;Micheli, Alessio
2018

Abstract

The extension of Recurrent Neural Networks (RNNs) in the direction of deep learning is a topic that is gaining more and more attention in the neural networks community. The study of deep RNNs opened a number of intriguing research questions on the actual role played by layering in the architectural design of RNNs. Recently, the introduction of the Deep Echo State Network (DeepESN) model allowed to start addressing such open issues in literature, contributing to shed light on the intrinsic properties of state dynamics developed by hierarchical compositions of recurrent layers. This contribution intends to present a unified view over the major advancements in the study of DeepESNs, enabling to directly point out the natural advantages of a layered construction of recurrent networks for temporal data processing.
9781509060146
File in questo prodotto:
Non ci sono file associati a questo prodotto.

I documenti in IRIS sono protetti da copyright e tutti i diritti sono riservati, salvo diversa indicazione.

Utilizza questo identificativo per citare o creare un link a questo documento: http://hdl.handle.net/11568/937439
 Attenzione

Attenzione! I dati visualizzati non sono stati sottoposti a validazione da parte dell'ateneo

Citazioni
  • ???jsp.display-item.citation.pmc??? ND
  • Scopus 3
  • ???jsp.display-item.citation.isi??? ND
social impact