Abstract
In Reinforcement Learning-based Recommender Systems (RLRS), the complexityand dynamism of user interactions often result in high-dimensional and noisystate spaces, making it challenging to discern which aspects of the state aretruly influential in driving the decision-making process. This issue isexacerbated by the evolving nature of user preferences and behaviors, requiringthe recommender system to adaptively focus on the most relevant information fordecision-making while preserving generaliability. To tackle this problem, weintroduce an innovative causal approach for decomposing the state andextracting \textbf{C}ausal-\textbf{I}n\textbf{D}ispensable \textbf{S}tateRepresentations (CIDS) in RLRS. Our method concentrates on identifying the\textbf{D}irectly \textbf{A}ction-\textbf{I}nfluenced \textbf{S}tate Variables(DAIS) and \textbf{A}ction-\textbf{I}nfluence \textbf{A}ncestors (AIA), whichare essential for making effective recommendations. By leveraging conditionalmutual information, we develop a framework that not only discerns the causalrelationships within the generative process but also isolates critical statevariables from the typically dense and high-dimensional state representations.We provide theoretical evidence for the identifiability of these variables.Then, by making use of the identified causal relationship, we constructcausal-indispensable state representations, enabling the training of policiesover a more advantageous subset of the agent's state space. We demonstrate theefficacy of our approach through extensive experiments, showcasing our methodoutperforms state-of-the-art methods.