Navigating the Trade-Off between Multi-Task Learning and Learning to Multitask in Deep Neural Networks

by   Sachin Ravi, et al.

The terms multi-task learning and multitasking are easily confused. Multi-task learning refers to a paradigm in machine learning in which a network is trained on various related tasks to facilitate the acquisition of tasks. In contrast, multitasking is used to indicate, especially in the cognitive science literature, the ability to execute multiple tasks simultaneously. While multi-task learning exploits the discovery of common structure between tasks in the form of shared representations, multitasking is promoted by separating representations between tasks to avoid processing interference. Here, we build on previous work involving shallow networks and simple task settings suggesting that there is a trade-off between multi-task learning and multitasking, mediated by the use of shared versus separated representations. We show that the same tension arises in deep networks and discuss a meta-learning algorithm for an agent to manage this trade-off in an unfamiliar environment. We display through different experiments that the agent is able to successfully optimize its training strategy as a function of the environment.


page 1

page 2

page 3

page 4


Which Tasks Should Be Learned Together in Multi-task Learning?

Many computer vision applications require solving multiple tasks in real...

Powerful, transferable representations for molecules through intelligent task selection in deep multitask networks

Chemical representations derived from deep learning are emerging as a po...

Decision Making Problems with Funnel Structure: A Multi-Task Learning Approach with Application to Email Marketing Campaigns

This paper studies the decision making problem with Funnel Structure. Fu...

Transformative Machine Learning

The key to success in machine learning (ML) is the use of effective data...

PaRT: Parallel Learning Towards Robust and Transparent AI

This paper takes a parallel learning approach for robust and transparent...

Efficient and robust multi-task learning in the brain with modular task primitives

In a real-world setting biological agents do not have infinite resources...

Please sign up or login with your details

Forgot password? Click here to reset