Abstract
Incorporating prior knowledge or specifications of input-output relationshipsinto machine learning models has gained significant attention, as it enhancesgeneralization from limited data and leads to conforming outputs. However, mostexisting approaches use soft constraints by penalizing violations throughregularization, which offers no guarantee of constraint satisfaction--anessential requirement in safety-critical applications. On the other hand,imposing hard constraints on neural networks may hinder their representationalpower, adversely affecting performance. To address this, we propose HardNet, apractical framework for constructing neural networks that inherently satisfyhard constraints without sacrificing model capacity. Unlike approaches thatmodify outputs only at inference time, HardNet enables end-to-end training withhard constraint guarantees, leading to improved performance. To the best of ourknowledge, HardNet is the first method with an efficient forward pass toenforce more than one input-dependent inequality constraint. It allowsunconstrained optimization of the network parameters using standard algorithmsby appending a differentiable closed-form enforcement layer to the network'soutput. Furthermore, we show that HardNet retains the universal approximationcapabilities of neural networks. We demonstrate the versatility andeffectiveness of HardNet across various applications: learning with piecewiseconstraints, learning optimization solvers, optimizing control policies insafety-critical systems, and learning safe decision logic for aircraft systems.