Reinforcement learning agents have demonstrated remarkable achievements insimulated environments. Data efficiency poses an impediment to carrying thissuccess over to real environments. The design of data-efficient agents callsfor a deeper understanding of information acquisition and representation. Wedevelop concepts and establish a regret bound that together offer principledguidance. The bound sheds light on questions of what information to seek, howto seek that information, and it what information to retain. To illustrateconcepts, we design simple agents that build on them and present computationalresults that demonstrate improvements in data efficiency.