Abstract
We show that a one-layer feedforward neural network with exponentialactivation functions in the inner layer and logarithmic activation in theoutput neuron is a universal approximator of convex functions. Such a networkrepresents a family of scaled log-sum exponential functions, here named LSET.The proof uses a dequantization argument from tropical geometry. Under asuitable exponential transformation LSE maps to a family of generalizedposynomial functions GPOST, which we also show to be universal approximatorsfor log-log-convex functions. The key feature of interest in the proposedapproach is that, once a LSET network is trained on data, the resulting modelis convex in the variables, which makes it readily amenable to efficient designbased on convex optimization. Similarly, once a GPOST model is trained on data,it yields a posynomial model that can be efficiently optimized with respect toits variables by using Geometric Programming (GP). Many relevant phenomena inphysics and engineering can indeed be modeled, either exactly or approximately,via convex or log-log-convex models. The proposed methodology is illustrated bytwo numerical examples in which LSET and GPOST models are used to firstapproximate data gathered from the simulations of two physical processes (thevibration from a vehicle suspension system, and the peak power generated by thecombustion of propane), and to later optimize these models.