Detecting Unknown Encrypted Malicious Traffic in Real Time via Flow Interaction Graph Analysis

by   Chuanpu Fu, et al.

In this paper, we propose HyperVision, a realtime unsupervised machine learning (ML) based malicious traffic detection system. Particularly, HyperVision is able to detect unknown patterns of encrypted malicious traffic by utilizing a compact inmemory graph built upon the traffic patterns. The graph captures flow interaction patterns represented by the graph structural features, instead of the features of specific known attacks. We develop an unsupervised graph learning method to detect abnormal interaction patterns by analyzing the connectivity, sparsity, and statistical features of the graph, which allows HyperVision to detect various encrypted attack traffic without requiring any labeled datasets of known attacks. Moreover, we establish an information theory model to demonstrate that the information preserved by the graph approaches the ideal theoretical bound. We show the performance of HyperVision by real-world experiments with 92 datasets including 48 attacks with encrypted malicious traffic. The experimental results illustrate that HyperVision achieves at least 0.92 AUC and 0.86 F1, which significantly outperform the state-of-the-art methods. In particular, more than 50 in our experiments can evade all these methods. Moreover, HyperVision achieves at least 80.6 Gb/s detection throughput with the average detection latency of 0.83s.


page 1

page 11

page 12


Realtime Robust Malicious Traffic Detection via Frequency Domain Analysis

Machine learning (ML) based malicious traffic detection is an emerging s...

Decrypting SSL/TLS traffic for hidden threats detection

The paper presents an analysis of the main mechanisms of decryption of S...

3D-IDS: Doubly Disentangled Dynamic Intrusion Detection

Network-based intrusion detection system (NIDS) monitors network traffic...

Unsupervised Abnormal Traffic Detection through Topological Flow Analysis

Cyberthreats are a permanent concern in our modern technological world. ...

Tweaking Metasploit to Evade Encrypted C2 Traffic Detection

Command and Control (C2) communication is a key component of any structu...

A Graph-Based Machine Learning Approach for Bot Detection

Bot detection using machine learning (ML), with network flow-level featu...

Please sign up or login with your details

Forgot password? Click here to reset