MTLSegFormer: Multi-task Learning with Transformers for Semantic Segmentation in Precision Agriculture

by   Diogo Nunes Gonçalves, et al.
University of Stirling
University of Waterloo
Universidade Federal de Mato Grosso do Sul

Multi-task learning has proven to be effective in improving the performance of correlated tasks. Most of the existing methods use a backbone to extract initial features with independent branches for each task, and the exchange of information between the branches usually occurs through the concatenation or sum of the feature maps of the branches. However, this type of information exchange does not directly consider the local characteristics of the image nor the level of importance or correlation between the tasks. In this paper, we propose a semantic segmentation method, MTLSegFormer, which combines multi-task learning and attention mechanisms. After the backbone feature extraction, two feature maps are learned for each task. The first map is proposed to learn features related to its task, while the second map is obtained by applying learned visual attention to locally re-weigh the feature maps of the other tasks. In this way, weights are assigned to local regions of the image of other tasks that have greater importance for the specific task. Finally, the two maps are combined and used to solve a task. We tested the performance in two challenging problems with correlated tasks and observed a significant improvement in accuracy, mainly in tasks with high dependence on the others.


page 4

page 6

page 7


Sequential Cross Attention Based Multi-task Learning

In multi-task learning (MTL) for visual scene understanding, it is cruci...

News Cover Assessment via Multi-task Learning

Online personalized news product needs a suitable cover for the article....

MuraNet: Multi-task Floor Plan Recognition with Relation Attention

The recognition of information in floor plan data requires the use of de...

TBGC: Task-level Backbone-Oriented Gradient Clip for Multi-Task Foundation Model Learning

The AllInOne training paradigm squeezes a wide range of tasks into a uni...

Shape-Margin Knowledge Augmented Network for Thyroid Nodule Segmentation and Diagnosis

Thyroid nodule segmentation is a crucial step in the diagnostic procedur...

Medusa: Universal Feature Learning via Attentional Multitasking

Recent approaches to multi-task learning (MTL) have focused on modelling...

Counting with Adaptive Auxiliary Learning

This paper proposes an adaptive auxiliary task learning based approach f...

Please sign up or login with your details

Forgot password? Click here to reset