In this paper, we present a coarse to fine question answering (CFQA) systembased on reinforcement learning which can efficiently processes documents withdifferent lengths by choosing appropriate actions. The system is designed usingan actor-critic based deep reinforcement learning model to achieve multi-stepquestion answering. Compared to previous QA models targeting on datasets mainlycontaining either short or long documents, our multi-step coarse to fine modeltakes the merits from multiple system modules, which can handle both short andlong documents. The system hence obtains a much better accuracy and fastertrainings speed compared to the current state-of-the-art models. We test ourmodel on four QA datasets, WIKEREADING, WIKIREADING LONG, CNN and SQuAD, anddemonstrate 1.3$\%$-1.7$\%$ accuracy improvements with 1.5x-3.4x trainingspeed-ups in comparison to the baselines using state-of-the-art models.