This paper discusses learning a structured feedback control to obtainsufficient robustness to exogenous inputs for linear dynamic systems withunknown state matrix. The structural constraint on the controller is necessaryfor many cyber-physical systems, and our approach presents a design for anygeneric structure, paving the way for distributed learning control. The ideasfrom reinforcement learning (RL) in conjunction with control-theoreticsufficient stability and performance guarantees are used to develop themethodology. First, a model-based framework is formulated using dynamicprogramming to embed the structural constraint in the linear quadraticregulator (LQR) setting along with sufficient robustness conditions.Thereafter, we translate these conditions to a data-driven learning-basedframework - robust structured reinforcement learning (RSRL) that enjoys thecontrol-theoretic guarantees on stability and convergence. We validate ourtheoretical results with a simulation on a multi-agent network with $6$ agents.