ARM-Net: Adaptive Relation Modeling Network for Structured Data

by   Shaofeng Cai, et al.

Relational databases are the de facto standard for storing and querying structured data, and extracting insights from structured data requires advanced analytics. Deep neural networks (DNNs) have achieved super-human prediction performance in particular data types, e.g., images. However, existing DNNs may not produce meaningful results when applied to structured data. The reason is that there are correlations and dependencies across combinations of attribute values in a table, and these do not follow simple additive patterns that can be easily mimicked by a DNN. The number of possible such cross features is combinatorial, making them computationally prohibitive to model. Furthermore, the deployment of learning models in real-world applications has also highlighted the need for interpretability, especially for high-stakes applications, which remains another issue of concern to DNNs. In this paper, we present ARM-Net, an adaptive relation modeling network tailored for structured data, and a lightweight framework ARMOR based on ARM-Net for relational data analytics. The key idea is to model feature interactions with cross features selectively and dynamically, by first transforming the input features into exponential space, and then determining the interaction order and interaction weights adaptively for each cross feature. We propose a novel sparse attention mechanism to dynamically generate the interaction weights given the input tuple, so that we can explicitly model cross features of arbitrary orders with noisy features filtered selectively. Then during model inference, ARM-Net can specify the cross features being used for each prediction for higher accuracy and better interpretability. Our extensive experiments on real-world datasets demonstrate that ARM-Net consistently outperforms existing models and provides more interpretable predictions for data-driven decision making.


page 1

page 2

page 3

page 4


DNN2LR: Interpretation-inspired Feature Crossing for Real-world Tabular Data

For sake of reliability, it is necessary for models in real-world applic...

T2G-Former: Organizing Tabular Features into Relation Graphs Promotes Heterogeneous Feature Interaction

Recent development of deep neural networks (DNNs) for tabular learning h...

Detecting Statistical Interactions from Neural Network Weights

Interpreting deep neural networks can enable new applications for predic...

GAMI-Net: An Explainable Neural Network based on Generalized Additive Models with Structured Interactions

The lack of interpretability is an inevitable problem when using neural ...

Neural Generators of Sparse Local Linear Models for Achieving both Accuracy and Interpretability

For reliability, it is important that the predictions made by machine le...

AdaEnsemble: Learning Adaptively Sparse Structured Ensemble Network for Click-Through Rate Prediction

Learning feature interactions is crucial to success for large-scale CTR ...

DIDER: Discovering Interpretable Dynamically Evolving Relations

Effective understanding of dynamically evolving multiagent interactions ...

Please sign up or login with your details

Forgot password? Click here to reset