Beyond Self-Attention: Deformable Large Kernel Attention for Medical Image Segmentation

by   Reza Azad, et al.

Medical image segmentation has seen significant improvements with transformer models, which excel in grasping far-reaching contexts and global contextual information. However, the increasing computational demands of these models, proportional to the squared token count, limit their depth and resolution capabilities. Most current methods process D volumetric image data slice-by-slice (called pseudo 3D), missing crucial inter-slice information and thus reducing the model's overall performance. To address these challenges, we introduce the concept of Deformable Large Kernel Attention (D-LKA Attention), a streamlined attention mechanism employing large convolution kernels to fully appreciate volumetric context. This mechanism operates within a receptive field akin to self-attention while sidestepping the computational overhead. Additionally, our proposed attention mechanism benefits from deformable convolutions to flexibly warp the sampling grid, enabling the model to adapt appropriately to diverse data patterns. We designed both 2D and 3D adaptations of the D-LKA Attention, with the latter excelling in cross-depth data understanding. Together, these components shape our novel hierarchical Vision Transformer architecture, the D-LKA Net. Evaluations of our model against leading methods on popular medical segmentation datasets (Synapse, NIH Pancreas, and Skin lesion) demonstrate its superior performance. Our code implementation is publicly available at the:


page 4

page 6

page 8

page 15

page 16

page 17


TEC-Net: Vision Transformer Embrace Convolutional Neural Networks for Medical Image Segmentation

The hybrid architecture of convolution neural networks (CNN) and Transfo...

3D UX-Net: A Large Kernel Volumetric ConvNet Modernizing Hierarchical Transformer for Medical Image Segmentation

Vision transformers (ViTs) have quickly superseded convolutional network...

Spatial Context-Aware Self-Attention Model For Multi-Organ Segmentation

Multi-organ segmentation is one of most successful applications of deep ...

CiT-Net: Convolutional Neural Networks Hand in Hand with Vision Transformers for Medical Image Segmentation

The hybrid architecture of convolutional neural networks (CNNs) and Tran...

MDA-Net: Multi-Dimensional Attention-Based Neural Network for 3D Image Segmentation

Segmenting an entire 3D image often has high computational complexity an...

Swin Deformable Attention Hybrid U-Net for Medical Image Segmentation

How to harmonize convolution and multi-head self-attention mechanisms ha...

UNet-2022: Exploring Dynamics in Non-isomorphic Architecture

Recent medical image segmentation models are mostly hybrid, which integr...

Please sign up or login with your details

Forgot password? Click here to reset