Seamlessly interacting with humans or robots is hard because these agents arenon-stationary. They update their policy in response to the ego agent'sbehavior, and the ego agent must anticipate these changes to co-adapt. Inspiredby humans, we recognize that robots do not need to explicitly model everylow-level action another agent will make; instead, we can capture the latentstrategy of other agents through high-level representations. We propose areinforcement learning-based framework for learning latent representations ofan agent's policy, where the ego agent identifies the relationship between itsbehavior and the other agent's future strategy. The ego agent then leveragesthese latent dynamics to influence the other agent, purposely guiding themtowards policies suitable for co-adaptation. Across several simulated domainsand a real-world air hockey game, our approach outperforms the alternatives andlearns to influence the other agent.