An Initialization Schema for Neuronal Networks on Tabular Data

  • 2023-11-07 13:52:35
  • Wolfgang Fuhl
  • 0


Nowadays, many modern applications require heterogeneous tabular data, whichis still a challenging task in terms of regression and classification. Manyapproaches have been proposed to adapt neural networks for this task, butstill, boosting and bagging of decision trees are the best-performing methodsfor this task. In this paper, we show that a binomial initialized neuralnetwork can be used effectively on tabular data. The proposed approach shows asimple but effective approach for initializing the first hidden layer in neuralnetworks. We also show that this initializing schema can be used to jointlytrain ensembles by adding gradient masking to batch entries and using thebinomial initialization for the last layer in a neural network. For thispurpose, we modified the hinge binary loss and the soft max loss to make themapplicable for joint ensemble training. We evaluate our approach on multiplepublic datasets and showcase the improved performance compared to other neuralnetwork-based approaches. In addition, we discuss the limitations and possiblefurther research of our approach for improving the applicability of neuralnetworks to tabular data. Link:


Quick Read (beta)

loading the full paper ...