Heterogeneous Anomaly Detection for Software Systems via Attentive Multi-modal Learning

by   Baitong Li, et al.

Prompt and accurate detection of system anomalies is essential to ensure the reliability of software systems. Unlike manual efforts that exploit all available run-time information, existing approaches usually leverage only a single type of monitoring data (often logs or metrics) or fail to make effective use of the joint information among multi-source data. Consequently, many false predictions occur. To better understand the manifestations of system anomalies, we conduct a comprehensive empirical study based on a large amount of heterogeneous data, i.e., logs and metrics. Our study demonstrates that system anomalies could manifest distinctly in different data types. Thus, integrating heterogeneous data can help recover the complete picture of a system's health status. In this context, we propose HADES, the first work to effectively identify system anomalies based on heterogeneous data. Our approach employs a hierarchical architecture to learn a global representation of the system status by fusing log semantics and metric patterns. It captures discriminative features and meaningful interactions from multi-modal data via a novel cross-modal attention module, enabling accurate system anomaly detection. We evaluate HADES extensively on large-scale simulated and industrial datasets. The experimental results present the superiority of HADES in detecting system anomalies on heterogeneous data. We release the code and the annotated dataset for reproducibility and future research.


Heterogeneous Anomaly Detection for Software Systems via Semi-supervised Cross-modal Attention

Prompt and accurate detection of system anomalies is essential to ensure...

Log-based Anomaly Detection Without Log Parsing

Software systems often record important runtime information in system lo...

Deep Structured Cross-Modal Anomaly Detection

Anomaly detection is a fundamental problem in data mining field with man...

Anomaly Detection in DevOps Toolchain

The tools employed in the DevOps Toolchain generates a large quantity of...

Detecting anomalies in heterogeneous population-scale VAT networks

Anomaly detection in network science is the method to determine aberrant...

Eadro: An End-to-End Troubleshooting Framework for Microservices on Multi-source Data

The complexity and dynamism of microservices pose significant challenges...

A Hierarchical Approach to Conditional Random Fields for System Anomaly Detection

Anomaly detection to recognize unusual events in large scale systems in ...

Please sign up or login with your details

Forgot password? Click here to reset