Abstract
Like image coding in visual data transmission, feature coding is essentialfor the distributed deployment of large models by significantly reducingtransmission and storage burden. However, prior studies have mostly targetedtask- or model-specific scenarios, leaving the challenge of universal featurecoding across diverse large models largely unexplored. In this paper, wepresent the first systematic study on universal feature coding for largemodels. The key challenge lies in the inherently diverse and distributionallyincompatible nature of features extracted from different models. For example,features from DINOv2 exhibit highly peaky, concentrated distributions, whilethose from Stable Diffusion 3 (SD3) are more dispersed and uniform. Thisdistributional heterogeneity severely hampers both compression efficiency andcross-model generalization. To address this, we propose a learnedpeaky-to-balanced distribution transformation, which reshapes highly skewedfeature distributions into a common, balanced target space. This transformationis non-uniform, data-driven, and plug-and-play, enabling effective alignment ofheterogeneous distributions without modifying downstream codecs. With thisalignment, a universal codec trained on the balanced target distribution caneffectively generalize to features from different models and tasks. We validateour approach on three representative large models (LLaMA3, DINOv2, and SD3)across multiple tasks and modalities. Extensive experiments show that ourmethod achieves notable improvements in both compression efficiency andcross-model generalization over task-specific baselines. All source code hasbeen made available at https://github.com/chansongoal/DT-UFC.