Stability and Performance Analysis of Discrete-Time ReLU Recurrent Neural Networks

  • 2024-05-14 18:53:13
  • Sahel Vahedi Noori, Bin Hu, Geir Dullerud, Peter Seiler
  • 0


This paper presents sufficient conditions for the stability and $\ell_2$-gainperformance of recurrent neural networks (RNNs) with ReLU activation functions.These conditions are derived by combining Lyapunov/dissipativity theory withQuadratic Constraints (QCs) satisfied by repeated ReLUs. We write a generalclass of QCs for repeated RELUs using known properties for the scalar ReLU. Ourstability and performance condition uses these QCs along with a "lifted"representation for the ReLU RNN. We show that the positive homogeneity propertysatisfied by a scalar ReLU does not expand the class of QCs for the repeatedReLU. We present examples to demonstrate the stability / performance conditionand study the effect of the lifting horizon.


Quick Read (beta)

loading the full paper ...