Abstract

This paper presents sufficient conditions for the stability and $\ell_2$-gainperformance of recurrent neural networks (RNNs) with ReLU activation functions.These conditions are derived by combining Lyapunov/dissipativity theory withQuadratic Constraints (QCs) satisfied by repeated ReLUs. We write a generalclass of QCs for repeated RELUs using known properties for the scalar ReLU. Ourstability and performance condition uses these QCs along with a "lifted"representation for the ReLU RNN. We show that the positive homogeneity propertysatisfied by a scalar ReLU does not expand the class of QCs for the repeatedReLU. We present examples to demonstrate the stability / performance conditionand study the effect of the lifting horizon.

 

Quick Read (beta)