The objective of this paper is to verify that current cutting-edge artificialintelligence technology, deep reinforcement learning, can be applied toportfolio management. We improve on the existing Deep Reinforcement LearningPortfolio model and make many innovations. Unlike many previous studies ondiscrete trading signals in portfolio management, we make the agent to short ina continuous action space, design an arbitrage mechanism based on ArbitragePricing Theory,and redesign the activation function for acquiring actionvectors, in addition, we redesign neural networks for reinforcement learningwith reference to deep neural networks that process image data. In experiments,we use our model in several randomly selected portfolios which include CSI300that represents the market's rate of return and the randomly selectedconstituents of CSI500. The experimental results show that no matter whatstocks we select in our portfolios, we can almost get a higher return than themarket itself. That is to say, we can defeat market by using deep reinforcementlearning.