Abstract
With the advent of virtual reality technology, omnidirectional image (ODI)rescaling techniques are increasingly embraced for reducing transmitted andstored file sizes while preserving high image quality. Despite this progress,current ODI rescaling methods predominantly focus on enhancing the quality ofimages in equirectangular projection (ERP) format, which overlooks the factthat the content viewed on head mounted displays (HMDs) is actually a renderedviewport instead of an ERP image. In this work, we emphasize that focusingsolely on ERP quality results in inferior viewport visual experiences forusers. Thus, we propose ResVR, which is the first comprehensive framework forthe joint Rescaling and Viewport Rendering of ODIs. ResVR allows obtaining LRERP images for transmission while rendering high-quality viewports for users towatch on HMDs. In our ResVR, a novel discrete pixel sampling strategy isdeveloped to tackle the complex mapping between the viewport and ERP, enablingend-to-end training of ResVR pipeline. Furthermore, a spherical pixel shaperepresentation technique is innovatively derived from spherical differentiationto significantly improve the visual quality of rendered viewports. Extensiveexperiments demonstrate that our ResVR outperforms existing methods in viewportrendering tasks across different fields of view, resolutions, and viewdirections while keeping a low transmission overhead.