Abstract
The quality of synthetically generated images (e.g. those produced bydiffusion models) are often evaluated using information about image contentsencoded by pretrained auxiliary models. For example, the Fr\'{e}chet InceptionDistance (FID) uses embeddings from an InceptionV3 model pretrained to classifyImageNet. The effectiveness of this feature embedding model has considerableimpact on the trustworthiness of the calculated metric (affecting itssuitability in several domains, including medical imaging). Here, uncertaintyquantification (UQ) is used to provide a heuristic measure of thetrustworthiness of the feature embedding model and an FID-like metric calledthe Fr\'{e}chet Autoencoder Distance (FAED). We apply Monte Carlo dropout to afeature embedding model (convolutional autoencoder) to model the uncertainty inits embeddings. The distribution of embeddings for each input are then used tocompute a distribution of FAED values. We express uncertainty as the predictivevariance of the embeddings as well as the standard deviation of the computedFAED values. We find that their magnitude correlates with the extent to whichthe inputs are out-of-distribution to the model's training data, providing somevalidation of its ability to assess the trustworthiness of the FAED.