Abstract
With the increasing numbers of publicly available models, there are probablypretrained, online models for most tasks users require. However, current modelsearch methods are rudimentary, essentially a text-based search in thedocumentation, thus users cannot find the relevant models. This paper presentsProbeLog, a method for retrieving classification models that can recognize atarget concept, such as "Dog", without access to model metadata or trainingdata. Differently from previous probing methods, ProbeLog computes a descriptorfor each output dimension (logit) of each model, by observing its responses ona fixed set of inputs (probes). Our method supports both logit-based retrieval("find more logits like this") and zero-shot, text-based retrieval ("find alllogits corresponding to dogs"). As probing-based representations requiremultiple costly feedforward passes through the model, we develop a method,based on collaborative filtering, that reduces the cost of encodingrepositories by 3x. We demonstrate that ProbeLog achieves high retrievalaccuracy, both in real-world and fine-grained search tasks and is scalable tofull-size repositories.