Embed Any NeRF: Graph Meta-Networks for Neural Tasks on Arbitrary NeRF Architectures

  • 2025-02-13 18:59:50
  • Francesco Ballerini, Pierluigi Zama Ramirez, Samuele Salti, Luigi Di Stefano
  • 0

Abstract

Neural Radiance Fields (NeRFs) have emerged as a groundbreaking paradigm forrepresenting 3D objects and scenes by encoding shape and appearance informationinto the weights of a neural network. Recent works have shown how such weightscan be used as input to frameworks processing them to solve deep learningtasks. Yet, these frameworks can only process NeRFs with a specific, predefinedarchitecture. In this paper, we present the first framework that can ingestNeRFs with multiple architectures and perform inference on architectures unseenat training time. We achieve this goal by training a Graph Meta-Network in arepresentation learning framework. Moreover, we show how a contrastiveobjective is conducive to obtaining an architecture-agnostic latent space. Inexperiments on both MLP-based and tri-planar NeRFs, our approach demonstratesrobust performance in classification and retrieval tasks that either matches orexceeds that of existing frameworks constrained to single architectures, thusproviding the first architecture-agnostic method to perform tasks on NeRFs byprocessing their weights.

 

Quick Read (beta)

loading the full paper ...