Abstract
Despite the success of neural networks (NNs), there is still a concern amongmany over their "black box" nature. Why do they work? Here we present a simpleanalytic argument that NNs are in fact essentially polynomial regressionmodels. This view will have various implications for NNs, e.g. providing anexplanation for why convergence problems arise in NNs, and it gives roughguidance on avoiding overfitting. In addition, we use this phenomenon topredict and confirm a multicollinearity property of NNs not previously reportedin the literature. Most importantly, given this loose correspondence, one maychoose to routinely use polynomial models instead of NNs, thus avoiding somemajor problems of the latter, such as having to set many tuning parameters anddealing with convergence issues. We present a number of empirical results; ineach case, the accuracy of the polynomial approach matches or exceeds that ofNN approaches. A many-featured, open-source software package, polyreg, isavailable.