In this paper we explore the usage of deep reinforcement learning algorithmsto automatically generate consistently profitable, robust, uncorrelated tradingsignals in any general financial market. In order to do this, we present anovel Markov decision process (MDP) model to capture the financial tradingmarkets. We review and propose various modifications to existing approaches andexplore different techniques to succinctly capture the market dynamics to modelthe markets. We then go on to use deep reinforcement learning to enable theagent (the algorithm) to learn how to take profitable trades in any market onits own, while suggesting various methodology changes and leveraging the uniquerepresentation of the FMDP (financial MDP) to tackle the primary challengesfaced in similar works. Through our experimentation results, we go on to showthat our model could be easily extended to two very different financial marketsand generates a positively robust performance in all conducted experiments.