Structured State Space Models for Multiple Instance Learning in Digital Pathology

by   Leo Fillioux, et al.

Multiple instance learning is an ideal mode of analysis for histopathology data, where vast whole slide images are typically annotated with a single global label. In such cases, a whole slide image is modelled as a collection of tissue patches to be aggregated and classified. Common models for performing this classification include recurrent neural networks and transformers. Although powerful compression algorithms, such as deep pre-trained neural networks, are used to reduce the dimensionality of each patch, the sequences arising from whole slide images remain excessively long, routinely containing tens of thousands of patches. Structured state space models are an emerging alternative for sequence modelling, specifically designed for the efficient modelling of long sequences. These models invoke an optimal projection of an input sequence into memory units that compress the entire sequence. In this paper, we propose the use of state space models as a multiple instance learner to a variety of problems in digital pathology. Across experiments in metastasis detection, cancer subtyping, mutation classification, and multitask learning, we demonstrate the competitiveness of this new class of models with existing state of the art approaches. Our code is available at


CellViT: Vision Transformers for Precise Cell Segmentation and Classification

Nuclei detection and segmentation in hematoxylin and eosin-stained (H ...

Detection of prostate cancer in whole-slide images through end-to-end training with image-level labels

Prostate cancer is the most prevalent cancer among men in Western countr...

FlexiViT: One Model for All Patch Sizes

Vision Transformers convert images to sequences by slicing them into pat...

Ensembling Neural Networks for Digital Pathology Images Classification and Segmentation

In the last years, neural networks have proven to be a powerful framewor...

Sparse convolutional context-aware multiple instance learning for whole slide image classification

Whole slide microscopic slides display many cues about the underlying ti...

State-Space Constraints Improve the Generalization of the Differentiable Neural Computer in some Algorithmic Tasks

Memory-augmented neural networks (MANNs) can solve algorithmic tasks lik...

DTFD-MIL: Double-Tier Feature Distillation Multiple Instance Learning for Histopathology Whole Slide Image Classification

Multiple instance learning (MIL) has been increasingly used in the class...

Please sign up or login with your details

Forgot password? Click here to reset