Adversarial examples are perturbed inputs that are designed (from a deeplearning network's (DLN) parameter gradients) to mislead the DLN during testtime. Intuitively, constraining the dimensionality of inputs or parameters of anetwork reduces the 'space' in which adversarial examples exist. Guided by thisintuition, we demonstrate that discretization greatly improves the robustnessof DLNs against adversarial attacks. Specifically, discretizing the input space(or allowed pixel levels from 256 values or 8-bit to 4 values or 2-bit)extensively improves the adversarial robustness of DLNs for a substantial rangeof perturbations for minimal loss in test accuracy. Furthermore, we find thatBinary Neural Networks (BNNs) and related variants are intrinsically morerobust than their full precision counterparts in adversarial scenarios.Combining input discretization with BNNs furthers the robustness even waivingthe need for adversarial training for certain magnitude of perturbation values.We evaluate the effect of discretization on MNIST, CIFAR10, CIFAR100 andImagenet datasets. Across all datasets, we observe maximal adversarialresistance with 2-bit input discretization that incurs an adversarial accuracyloss of just ~1-2% as compared to clean test accuracy.