Learning High-Order Interactions via Targeted Pattern Search

by   Michela C. Massi, et al.

Logistic Regression (LR) is a widely used statistical method in empirical binary classification studies. However, real-life scenarios oftentimes share complexities that prevent from the use of the as-is LR model, and instead highlight the need to include high-order interactions to capture data variability. This becomes even more challenging because of: (i) datasets growing wider, with more and more variables; (ii) studies being typically conducted in strongly imbalanced settings; (iii) samples going from very large to extremely small; (iv) the need of providing both predictive models and interpretable results. In this paper we present a novel algorithm, Learning high-order Interactions via targeted Pattern Search (LIPS), to select interaction terms of varying order to include in a LR model for an imbalanced binary classification task when input data are categorical. LIPS's rationale stems from the duality between item sets and categorical interactions. The algorithm relies on an interaction learning step based on a well-known frequent item set mining algorithm, and a novel dissimilarity-based interaction selection step that allows the user to specify the number of interactions to be included in the LR model. In addition, we particularize two variants (Scores LIPS and Clusters LIPS), that can address even more specific needs. Through a set of experiments we validate our algorithm and prove its wide applicability to real-life research scenarios, showing that it outperforms a benchmark state-of-the-art algorithm.


page 1

page 2

page 3

page 4


Fast and More Powerful Selective Inference for Sparse High-order Interaction Model

Automated high-stake decision-making such as medical diagnosis requires ...

An Efficient Post-Selection Inference on High-Order Interaction Models

Finding statistically significant high-order interaction features in pre...

On Dropout, Overfitting, and Interaction Effects in Deep Neural Networks

We examine Dropout through the perspective of interactions: learned effe...

Safe Feature Pruning for Sparse High-Order Interaction Models

Taking into account high-order interactions among covariates is valuable...

Disentangled Self-Attentive Neural Networks for Click-Through Rate Prediction

Click-through rate (CTR) prediction, which aims to predict the probabili...

Random Intersection Trees

Finding interactions between variables in large and high-dimensional dat...

A Guided FP-growth algorithm for fast mining of frequent itemsets from big data

In this paper we present the GFP-growth (Guided FP-growth) algorithm, a ...

Please sign up or login with your details

Forgot password? Click here to reset