Unstructured Persona-oriented Dialogue Systems (UPDS) has been demonstratedeffective in generating persona consistent responses by utilizing predefinednatural language user persona descriptions (e.g., "I am a vegan"). However, thepredefined user persona descriptions are usually short and limited to only afew descriptive words, which makes it hard to correlate them with thedialogues. As a result, existing methods either fail to use the personadescription or use them improperly when generating persona consistentresponses. To address this, we propose a neural topical expansion framework,namely Persona Exploration and Exploitation (PEE), which is able to extend thepredefined user persona description with semantically correlated content beforeutilizing them to generate dialogue responses. PEE consists of two mainmodules: persona exploration and persona exploitation. The former learns toextend the predefined user persona description by mining and correlating withexisting dialogue corpus using a variational auto-encoder (VAE) based topicmodel. The latter learns to generate persona consistent responses by utilizingthe predefined and extended user persona description. In order to make personaexploitation learn to utilize user persona description more properly, we alsointroduce two persona-oriented loss functions: Persona-oriented Matching(P-Match) loss and Persona-oriented Bag-of-Words (P-BoWs) loss whichrespectively supervise persona selection in encoder and decoder. Experimentalresults show that our approach outperforms state-of-the-art baselines, in termsof both automatic and human evaluations.