Abstract
Whole Slide Image (WSI) classification with multiple instance learning (MIL)in digital pathology faces significant computational challenges. Currentmethods mostly rely on extensive self-supervised learning (SSL) forsatisfactory performance, requiring long training periods and considerablecomputational resources. At the same time, no pre-training affects performancedue to domain shifts from natural images to WSIs. We introduce\textbf{\textit{Snuffy}} architecture, a novel MIL-pooling method based onsparse transformers that mitigates performance loss with limited pre-trainingand enables continual few-shot pre-training as a competitive option. Oursparsity pattern is tailored for pathology and is theoretically proven to be auniversal approximator with the tightest probabilistic sharp bound on thenumber of layers for sparse transformers, to date. We demonstrate Snuffy'seffectiveness on CAMELYON16 and TCGA Lung cancer datasets, achieving superiorWSI and patch-level accuracies. The code is available on\url{https://github.com/jafarinia/snuffy}.