Abstract
Pruning encompasses a range of techniques aimed at increasing the sparsity ofneural networks (NNs). These techniques can generally be framed as minimizing aloss function subject to an $L_0$ norm constraint. This paper introducesCoNNect, a novel differentiable regularizer for sparse NN training that ensuresconnectivity between input and output layers. We prove that CoNNectapproximates $L_0$ regularization, guaranteeing maximally connected networkstructures while avoiding issues like layer collapse. Moreover, CoNNect iseasily integrated with established structural pruning strategies. Numericalexperiments demonstrate that CoNNect can improve classical pruning strategiesand enhance state-of-the-art one-shot pruners, such as DepGraph and LLM-pruner.