Deep neural networks have achieved substantial achievements in severalcomputer vision areas, but have vulnerabilities that are often fooled byadversarial examples that are not recognized by humans. This is an importantissue for security or medical applications. In this paper, we propose anensemble model training framework with random layer sampling to improve therobustness of deep neural networks. In the proposed training framework, wegenerate various sampled model through the random layer sampling and update theweight of the sampled model. After the ensemble models are trained, it can hidethe gradient efficiently and avoid the gradient-based attack by the randomlayer sampling method. To evaluate our proposed method, comprehensive andcomparative experiments have been conducted on three datasets. Experimentalresults show that the proposed method improves the adversarial robustness.