The Extrapolation Power of Implicit Models

  • 2024-07-19 17:01:37
  • Juliette Decugis, Alicia Y. Tsai, Max Emerling, Ashwin Ganesh, Laurent El Ghaoui
  • 0

Abstract

In this paper, we investigate the extrapolation capabilities of implicit deeplearning models in handling unobserved data, where traditional deep neuralnetworks may falter. Implicit models, distinguished by their adaptability inlayer depth and incorporation of feedback within their computational graph, areput to the test across various extrapolation scenarios: out-of-distribution,geographical, and temporal shifts. Our experiments consistently demonstratesignificant performance advantage with implicit models. Unlike theirnon-implicit counterparts, which often rely on meticulous architectural designfor each task, implicit models demonstrate the ability to learn complex modelstructures without the need for task-specific design, highlighting theirrobustness in handling unseen data.

 

Quick Read (beta)

loading the full paper ...