Abstract
What metrics should guide the development of more realistic models of thebrain? One proposal is to quantify the similarity between models and brainsusing methods such as linear regression, Centered Kernel Alignment (CKA), andangular Procrustes distance. To better understand the limitations of thesesimilarity measures we analyze neural activity recorded in five experiments onnonhuman primates, and optimize synthetic datasets to become more similar tothese neural recordings. How similar can these synthetic datasets be to neuralactivity while failing to encode task relevant variables? We find that somemeasures like linear regression and CKA, differ from angular Procrustes, andyield high similarity scores even when task relevant variables cannot belinearly decoded from the synthetic datasets. Synthetic datasets optimized tomaximize similarity scores initially learn the first principal component of thetarget dataset, but angular Procrustes captures higher variance dimensions muchearlier than methods like linear regression and CKA. We show in both theory andsimulations how these scores change when different principal components areperturbed. And finally, we jointly optimize multiple similarity scores to findtheir allowed ranges, and show that a high angular Procrustes similarity, forexample, implies a high CKA score, but not the converse.