Lai Loss: A Novel Loss Integrating Regularization

  • 2024-05-13 17:17:57
  • YuFei Lai
  • 0

Abstract

In the field of machine learning, traditional regularization methodsgenerally tend to directly add regularization terms to the loss function. Thispaper introduces the "Lai loss", a novel loss design that integrates theregularization terms (gradient component) into the traditional loss functionthrough a straightforward geometric ideation. This design innovativelypenalizes the gradient vectors through the loss, effectively controlling themodel's smoothness and offering the dual benefits of reducing overfitting andavoiding underfitting. Subsequently, we proposed a random sampling method thatsuccessfully addresses the challenges associated with its application underlarge sample conditions. We conducted preliminary experiments using publiclyavailable datasets from Kaggle, demonstrating that the design of Lai loss cancontrol the model's smoothness while ensuring maximum accuracy.

 

Quick Read (beta)

loading the full paper ...