Abstract
Incorporating prior knowledge or specifications of input-output relationshipsinto machine learning models has gained significant attention, as it enhancesgeneralization from limited data and leads to conforming outputs. However, mostexisting approaches use soft constraints by penalizing violations throughregularization, which offers no guarantee of constraint satisfaction -- anessential requirement in safety-critical applications. On the other hand,imposing hard constraints on neural networks may hinder their representationalpower, adversely affecting performance. To address this, we propose HardNet, apractical framework for constructing neural networks that inherently satisfyhard constraints without sacrificing model capacity. Specifically, we encodeaffine and convex hard constraints, dependent on both inputs and outputs, byappending a differentiable projection layer to the network's output. Thisarchitecture allows unconstrained optimization of the network parameters usingstandard algorithms while ensuring constraint satisfaction by construction.Furthermore, we show that HardNet retains the universal approximationcapabilities of neural networks. We demonstrate the versatility andeffectiveness of HardNet across various applications: fitting functions underconstraints, learning optimization solvers, optimizing control policies insafety-critical systems, and learning safe decision logic for aircraft systems.