Abstract
Caching is envisioned to play a critical role in next-generation contentdelivery infrastructure, cellular networks, and Internet architectures. Bysmartly storing the most popular contents at the storage-enabled networkentities during off-peak demand instances, caching can benefit both networkinfrastructure as well as end users, during on-peak periods. In this context,distributing the limited storage capacity across network entities calls fordecentralized caching schemes. Many practical caching systems involve a parentcaching node connected to multiple leaf nodes to serve user file requests. Tomodel the two-way interactive influence between caching decisions at the parentand leaf nodes, a reinforcement learning framework is put forth. To handle thelarge continuous state space, a scalable deep reinforcement learning approachis pursued. The novel approach relies on a deep Q-network to learn theQ-function, and thus the optimal caching policy, in an online fashion.Reinforcing the parent node with ability to learn-and-adapt to unknown policiesof leaf nodes as well as spatio-temporal dynamic evolution of file requests,results in remarkable caching performance, as corroborated through numericaltests.