Transfer Learning for Clinical Time Series Analysis using Deep Neural Networks

by   Priyanka Gupta, et al.

Deep neural networks have shown promising results for various clinical prediction tasks. However, training deep networks such as those based on Recurrent Neural Networks (RNNs) requires large labeled data, significant hyper-parameter tuning effort and expertise, and high computational resources. In this work, we investigate as to what extent can transfer learning address these issues when using deep RNNs to model multivariate clinical time series. We consider two scenarios for transfer learning using RNNs: i) domain-adaptation, i.e., leveraging a deep RNN - namely, TimeNet - pre-trained for feature extraction on time series from diverse domains, and adapting it for feature extraction and subsequent target tasks in healthcare domain, ii) task-adaptation, i.e., pre-training a deep RNN - namely, HealthNet - on diverse tasks in healthcare domain, and adapting it to new target tasks in the same domain. We evaluate the above approaches on publicly available MIMIC-III benchmark dataset, and demonstrate that (a) computationally-efficient linear models trained using features extracted via pre-trained RNNs outperform or, in the worst case, perform as well as deep RNNs and statistical hand-crafted features based models trained specifically for target task; (b) models obtained by adapting pre-trained models for target tasks are significantly more robust to the size of labeled data compared to task-specific RNNs, while also being computationally efficient. We, therefore, conclude that pre-trained deep models like TimeNet and HealthNet allow leveraging the advantages of deep learning for clinical time series analysis tasks, while also minimize dependence on hand-crafted features, deal robustly with scarce labeled training data scenarios without overfitting, as well as reduce dependence on expertise and resources required to train deep networks from scratch.


Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks

Deep neural networks have shown promising results for various clinical p...

When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets

Semi-Supervised Learning (SSL) has been proved to be an effective way to...

ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification

Training deep neural networks often requires careful hyper-parameter tun...

Representation Similarity Analysis for Efficient Task taxonomy & Transfer Learning

Transfer learning is widely used in deep neural network models when ther...

A Survey on Time-Series Pre-Trained Models

Time-Series Mining (TSM) is an important research area since it shows gr...

Train and You'll Miss It: Interactive Model Iteration with Weak Supervision and Pre-Trained Embeddings

Our goal is to enable machine learning systems to be trained interactive...

Simplified Learning of CAD Features Leveraging a Deep Residual Autoencoder

In the domain of computer vision, deep residual neural networks like Eff...

Please sign up or login with your details

Forgot password? Click here to reset