Fourier Test-time Adaptation with Multi-level Consistency for Robust Classification

by   Yuhao Huang, et al.

Deep classifiers may encounter significant performance degradation when processing unseen testing data from varying centers, vendors, and protocols. Ensuring the robustness of deep models against these domain shifts is crucial for their widespread clinical application. In this study, we propose a novel approach called Fourier Test-time Adaptation (FTTA), which employs a dual-adaptation design to integrate input and model tuning, thereby jointly improving the model robustness. The main idea of FTTA is to build a reliable multi-level consistency measurement of paired inputs for achieving self-correction of prediction. Our contribution is two-fold. First, we encourage consistency in global features and local attention maps between the two transformed images of the same input. Here, the transformation refers to Fourier-based input adaptation, which can transfer one unseen image into source style to reduce the domain gap. Furthermore, we leverage style-interpolated images to enhance the global and local features with learnable parameters, which can smooth the consistency measurement and accelerate convergence. Second, we introduce a regularization technique that utilizes style interpolation consistency in the frequency space to encourage self-consistency in the logit space of the model output. This regularization provides strong self-supervised signals for robustness enhancement. FTTA was extensively validated on three large classification datasets with different modalities and organs. Experimental results show that FTTA is general and outperforms other strong state-of-the-art methods.


page 2

page 3

page 4

page 8


Gradual Test-Time Adaptation by Self-Training and Style Transfer

Domain shifts at test-time are inevitable in practice. Test-time adaptat...

GeoAdapt: Self-Supervised Test-Time Adaption in LiDAR Place Recognition Using Geometric Priors

LiDAR place recognition approaches based on deep learning suffer a signi...

OptTTA: Learnable Test-Time Augmentation for Source-Free Medical Image Segmentation Under Domain Shift

As distribution shifts are inescapable in realistic clinical scenarios d...

Feather-Light Fourier Domain Adaptation in Magnetic Resonance Imaging

Generalizability of deep learning models may be severely affected by the...

Test-time Fourier Style Calibration for Domain Generalization

The topic of generalizing machine learning models learned on a collectio...

Unseen Object Instance Segmentation with Fully Test-time RGB-D Embeddings Adaptation

Segmenting unseen objects is a crucial ability for the robot since it ma...

Multi-IMU with Online Self-Consistency for Freehand 3D Ultrasound Reconstruction

Ultrasound (US) imaging is a popular tool in clinical diagnosis, offerin...

Please sign up or login with your details

Forgot password? Click here to reset