Adversarial learning has emerged as one of the successful techniques tocircumvent the susceptibility of existing methods against adversarialperturbations. However, the majority of existing defense methods are tailoredto defend against a single category of adversarial perturbation (e.g.$\ell_\infty$-attack). In safety-critical applications, this makes thesemethods extraneous as the attacker can adopt diverse adversaries to deceive thesystem. Moreover, training on multiple perturbations simultaneouslysignificantly increases the computational overhead during training. To addressthese challenges, we propose a novel meta-learning framework that explicitlylearns to generate noise to improve the model's robustness against multipletypes of attacks. Its key component is Meta Noise Generator (MNG) that outputsoptimal noise to stochastically perturb a given sample, such that it helpslower the error on diverse adversarial perturbations. By utilizing samplesgenerated by MNG, we train a model by enforcing the label consistency acrossmultiple perturbations. We validate the robustness of models trained by ourscheme on various datasets and against a wide variety of perturbations,demonstrating that it significantly outperforms the baselines across multipleperturbations with a marginal computational cost.