Deep reinforcement learning has been recognized as an efficient technique todesign optimal strategies for different complex systems without prior knowledgeof the control landscape. To achieve a fast and precise control for quantumsystems, we propose a novel deep reinforcement learning approach byconstructing a curriculum consisting of a set of intermediate tasks defined bya fidelity threshold. Tasks among a curriculum can be statically determinedusing empirical knowledge or adaptively generated with the learning process. Bytransferring knowledge between two successive tasks and sequencing tasksaccording to their difficulties, the proposed curriculum-based deepreinforcement learning (CDRL) method enables the agent to focus on easy tasksin the early stage, then move onto difficult tasks, and eventually approachesthe final task. Numerical simulations on closed quantum systems and openquantum systems demonstrate that the proposed method exhibits improved controlperformance for quantum systems and also provides an efficient way to identifyoptimal strategies with fewer control pulses.