Foveation for Segmentation of Ultra-High Resolution Images

by   Chen Jin, et al.

Segmentation of ultra-high resolution images is challenging because of their enormous size, consisting of millions or even billions of pixels. Typical solutions include dividing input images into patches of fixed size and/or down-sampling to meet memory constraints. Such operations incur information loss in the field-of-view (FoV) i.e., spatial coverage and the image resolution. The impact on segmentation performance is, however, as yet understudied. In this work, we start with a motivational experiment which demonstrates that the trade-off between FoV and resolution affects the segmentation performance on ultra-high resolution images—and furthermore, its influence also varies spatially according to the local patterns in different areas. We then introduce foveation module, a learnable "dataloader" which, for a given ultra-high resolution image, adaptively chooses the appropriate configuration (FoV/resolution trade-off) of the input patch to feed to the downstream segmentation model at each spatial location of the image. The foveation module is jointly trained with the segmentation network to maximise the task performance. We demonstrate on three publicly available high-resolution image datasets that the foveation module consistently improves segmentation performance over the cases trained with patches of fixed FoV/resolution trade-off. Our approach achieves the SoTA performance on the DeepGlobe aerial image dataset. On the Gleason2019 histopathology dataset, our model achieves better segmentation accuracy for the two most clinically important and ambiguous classes (Gleason Grade 3 and 4) than the top performers in the challenge by 13.1 6 human experts by 6.5


page 2

page 7

page 8

page 16

page 17

page 18

page 19

page 20


Learning to Downsample for Segmentation of Ultra-High Resolution Images

Segmentation of ultra-high resolution images with deep learning is chall...

Collaborative Global-Local Networks for Memory-Efficient Segmentation of Ultra-High Resolution Images

Segmentation of ultra-high resolution images is increasingly demanded, y...

From Contexts to Locality: Ultra-high Resolution Image Segmentation via Locality-aware Contextual Correlation

Ultra-high resolution image segmentation has raised increasing interests...

Ultra-High Resolution Segmentation with Ultra-Rich Context: A Novel Benchmark

With the increasing interest and rapid development of methods for Ultra-...

Human Perception-based Evaluation Criterion for Ultra-high Resolution Cell Membrane Segmentation

Computer vision technology is widely used in biological and medical data...

Iterative Patch Selection for High-Resolution Image Recognition

High-resolution images are prevalent in various applications, such as au...

Patient-level Microsatellite Stability Assessment from Whole Slide Images By Combining Momentum Contrast Learning and Group Patch Embeddings

Assessing microsatellite stability status of a patient's colorectal canc...

Please sign up or login with your details

Forgot password? Click here to reset