Temporal Self-Attention Network for Medical Concept Embedding

by   Xueping Peng, et al.

In longitudinal electronic health records (EHRs), the event records of a patient are distributed over a long period of time and the temporal relations between the events reflect sufficient domain knowledge to benefit prediction tasks such as the rate of inpatient mortality. Medical concept embedding as a feature extraction method that transforms a set of medical concepts with a specific time stamp into a vector, which will be fed into a supervised learning algorithm. The quality of the embedding significantly determines the learning performance over the medical data. In this paper, we propose a medical concept embedding method based on applying a self-attention mechanism to represent each medical concept. We propose a novel attention mechanism which captures the contextual information and temporal relationships between medical concepts. A light-weight neural net, "Temporal Self-Attention Network (TeSAN)", is then proposed to learn medical concept embedding based solely on the proposed attention mechanism. To test the effectiveness of our proposed methods, we have conducted clustering and prediction tasks on two public EHRs datasets comparing TeSAN against five state-of-the-art embedding methods. The experimental results demonstrate that the proposed TeSAN model is superior to all the compared methods. To the best of our knowledge, this work is the first to exploit temporal self-attentive relations between medical events.


BiteNet: Bidirectional Temporal Encoder Network to Predict Medical Outcomes

Electronic health records (EHRs) are longitudinal records of a patient's...

Medical Concept Embedding with Time-Aware Attention

Embeddings of medical concepts such as medication, procedure and diagnos...

Comparative Visual Analytics for Assessing Medical Records with Sequence Embedding

Machine learning for data-driven diagnosis has been actively studied in ...

Interpretable ICD Code Embeddings with Self- and Mutual-Attention Mechanisms

We propose a novel and interpretable embedding method to represent the i...

Contrastive Learning of Temporal Distinctiveness for Survival Analysis in Electronic Health Records

Survival analysis plays a crucial role in many healthcare decisions, whe...

Inpatient2Vec: Medical Representation Learning for Inpatients

Representation learning (RL) plays an important role in extracting prope...

Self-Attentive Hawkes Processes

Asynchronous events on the continuous time domain, e.g., social media ac...

Please sign up or login with your details

Forgot password? Click here to reset