Fully Connected Neural Networks (FCNNs) have been the core of moststate-of-the-art Machine Learning (ML) applications in recent years and alsohave been widely used for Intrusion Detection Systems (IDSs). Experimentalresults from the last years show that generally deeper neural networks withmore layers perform better than shallow models. Nonetheless, with the growingnumber of layers, obtaining fast predictions with less resources has become adifficult task despite the use of special hardware such as GPUs. We propose anew architecture to detect network attacks with minimal resources. Thearchitecture is able to deal with either binary or multiclass classificationproblems and trades prediction speed for the accuracy of the network. Weevaluate our proposal with two different network intrusion detection datasets.Results suggest that it is possible to obtain comparable accuracies to simpleFCNNs without evaluating all layers for the majority of samples, thus obtainingearly predictions and saving energy and computational efforts.