Deep reinforcement learning (RL) has achieved outstanding results in recentyears, which has led a dramatic increase in the number of methods andapplications. Recent works are exploring learning beyond single-agent scenariosand considering multi-agent scenarios. However, they are faced with lots ofchallenges and are seeking for help from traditional game-theoretic algorithms,which, in turn, show bright application promise combined with modern algorithmsand boosting computing power. In this survey, we first introduce basic conceptsand algorithms in single agent RL and multi-agent systems; then, we summarizethe related algorithms from three aspects. Solution concepts from game theorygive inspiration to algorithms which try to evaluate the agents or find bettersolutions in multi-agent systems. Fictitious self-play becomes popular and hasa great impact on the algorithm of multi-agent reinforcement learning.Counterfactual regret minimization is an important tool to solve games withincomplete information, and has shown great strength when combined with deeplearning.