A Small Gain Analysis of Single Timescale Actor Critic

  • 2023-05-25 18:59:20
  • Alex Olshevsky, Bahman Gharesifard
  • 0


We consider a version of actor-critic which uses proportional step-sizes andonly one critic update with a single sample from the stationary distributionper actor step. We provide an analysis of this method using the small-gaintheorem. Specifically, we prove that this method can be used to find astationary point, and that the resulting sample complexity improves the stateof the art for actor-critic methods to $O \left(\mu^{-2} \epsilon^{-2} \right)$to find an $\epsilon$-approximate stationary point where $\mu$ is the conditionnumber associated with the critic.


Quick Read (beta)

loading the full paper ...