Intra-domain and cross-domain transfer learning for time series data – How transferable are the features?

by   Erik Otović, et al.

In practice, it is very demanding and sometimes impossible to collect datasets of tagged data large enough to successfully train a machine learning model, and one possible solution to this problem is transfer learning. This study aims to assess how transferable are the features between different domains of time series data and under which conditions. The effects of transfer learning are observed in terms of predictive performance of the models and their convergence rate during training. In our experiment, we use reduced data sets of 1,500 and 9,000 data instances to mimic real world conditions. Using the same scaled-down datasets, we trained two sets of machine learning models: those that were trained with transfer learning and those that were trained from scratch. Four machine learning models were used for the experiment. Transfer of knowledge was performed within the same domain of application (seismology), as well as between mutually different domains of application (seismology, speech, medicine, finance). We observe the predictive performance of the models and the convergence rate during the training. In order to confirm the validity of the obtained results, we repeated the experiments seven times and applied statistical tests to confirm the significance of the results. The general conclusion of our study is that transfer learning is very likely to either increase or not negatively affect the predictive performance of the model or its convergence rate. The collected data is analysed in more details to determine which source and target domains are compatible for transfer of knowledge. We also analyse the effect of target dataset size and the selection of model and its hyperparameters on the effects of transfer learning.


page 1

page 29

page 39

page 42


A Comprehensive Survey on Transfer Learning

Transfer learning aims at improving the performance of target learners o...

Boosting Kidney Stone Identification in Endoscopic Images Using Two-Step Transfer Learning

Knowing the cause of kidney stone formation is crucial to establish trea...

Explicit Domain Adaptation with Loosely Coupled Samples

Transfer learning is an important field of machine learning in general, ...

From fat droplets to floating forests: cross-domain transfer learning using a PatchGAN-based segmentation model

Many scientific domains gather sufficient labels to train machine algori...

HUST bearing: a practical dataset for ball bearing fault diagnosis

In this work, we introduce a practical dataset named HUST bearing, that ...

On-Device Transfer Learning for Personalising Psychological Stress Modelling using a Convolutional Neural Network

Stress is a growing concern in modern society adversely impacting the wi...

Size doesn't matter: predicting physico- or biochemical properties based on dozens of molecules

The use of machine learning in chemistry has become a common practice. A...

Please sign up or login with your details

Forgot password? Click here to reset