Transfer Learning for Clinical Time Series Analysis using Recurrent Neural Networks

by   Priyanka Gupta, et al.

Deep neural networks have shown promising results for various clinical prediction tasks such as diagnosis, mortality prediction, predicting duration of stay in hospital, etc. However, training deep networks -- such as those based on Recurrent Neural Networks (RNNs) -- requires large labeled data, high computational resources, and significant hyperparameter tuning effort. In this work, we investigate as to what extent can transfer learning address these issues when using deep RNNs to model multivariate clinical time series. We consider transferring the knowledge captured in an RNN trained on several source tasks simultaneously using a large labeled dataset to build the model for a target task with limited labeled data. An RNN pre-trained on several tasks provides generic features, which are then used to build simpler linear models for new target tasks without training task-specific RNNs. For evaluation, we train a deep RNN to identify several patient phenotypes on time series from MIMIC-III database, and then use the features extracted using that RNN to build classifiers for identifying previously unseen phenotypes, and also for a seemingly unrelated task of in-hospital mortality. We demonstrate that (i) models trained on features extracted using pre-trained RNN outperform or, in the worst case, perform as well as task-specific RNNs; (ii) the models using features from pre-trained models are more robust to the size of labeled data than task-specific RNNs; and (iii) features extracted using pre-trained RNN are generic enough and perform better than typical statistical hand-crafted features.


Transfer Learning for Clinical Time Series Analysis using Deep Neural Networks

Deep neural networks have shown promising results for various clinical p...

Gated Transfer Network for Transfer Learning

Deep neural networks have led to a series of breakthroughs in computer v...

Interpretable Additive Recurrent Neural Networks For Multivariate Clinical Time Series

Time series models with recurrent neural networks (RNNs) can have high a...

MEME: Generating RNN Model Explanations via Model Extraction

Recurrent Neural Networks (RNNs) have achieved remarkable performance on...

Deep image representations using caption generators

Deep learning exploits large volumes of labeled data to learn powerful m...

ConvTimeNet: A Pre-trained Deep Convolutional Neural Network for Time Series Classification

Training deep neural networks often requires careful hyper-parameter tun...

Please sign up or login with your details

Forgot password? Click here to reset