Abstract
The widespread deployment of large models in resource-constrainedenvironments has underscored the need for efficient transmission ofintermediate feature representations. In this context, feature coding, whichcompresses features into compact bitstreams, becomes a critical component forscenarios involving feature transmission, storage, and reuse. However, thiscompression process inevitably introduces semantic degradation that isdifficult to quantify with traditional metrics. To address this, we formalizethe research problem of Compressed Feature Quality Assessment (CFQA), aiming toevaluate the semantic fidelity of compressed features. To advance CFQAresearch, we propose the first benchmark dataset, comprising 300 originalfeatures and 12000 compressed features derived from three vision tasks and fourfeature codecs. Task-specific performance degradation is provided as truesemantic distortion for evaluating CFQA metrics. We systematically assess threewidely used metrics -- MSE, cosine similarity, and Centered Kernel Alignment(CKA) -- in terms of their ability to capture semantic degradation. Ourfindings demonstrate the representativeness of the proposed dataset whileunderscoring the need for more sophisticated metrics capable of measuringsemantic distortion in compressed features. This work advances the field byestablishing a foundational benchmark and providing a critical resource for thecommunity to explore CFQA. To foster further research, we release the datasetand all associated source code athttps://github.com/chansongoal/Compressed-Feature-Quality-Assessment.