PED-ANOVA: Efficiently Quantifying Hyperparameter Importance in Arbitrary Subspaces

04/20/2023
by   Shuhei Watanabe, et al.
University of Freiburg
0

The recent rise in popularity of Hyperparameter Optimization (HPO) for deep learning has highlighted the role that good hyperparameter (HP) space design can play in training strong models. In turn, designing a good HP space is critically dependent on understanding the role of different HPs. This motivates research on HP Importance (HPI), e.g., with the popular method of functional ANOVA (f-ANOVA). However, the original f-ANOVA formulation is inapplicable to the subspaces most relevant to algorithm designers, such as those defined by top performance. To overcome this issue, we derive a novel formulation of f-ANOVA for arbitrary subspaces and propose an algorithm that uses Pearson divergence (PED) to enable a closed-form calculation of HPI. We demonstrate that this new algorithm, dubbed PED-ANOVA, is able to successfully identify important HPs in different subspaces while also being extremely computationally efficient.

READ FULL TEXT

page 1

page 2

page 11

page 18

04/24/2019

Reducing The Search Space For Hyperparameter Optimization Using Group Sparsity

We propose a new algorithm for hyperparameter selection in machine learn...
06/20/2015

Filtrated Algebraic Subspace Clustering

Subspace clustering is the problem of clustering data that lie close to ...
09/08/2020

Hyperparameter Optimization via Sequential Uniform Designs

Hyperparameter tuning or optimization plays a central role in the automa...
01/13/2022

Hyperparameter Importance for Machine Learning Algorithms

Hyperparameter plays an essential role in the fitting of supervised mach...
02/26/2020

PHS: A Toolbox for Parellel Hyperparameter Search

We introduce an open source python framework named PHS - Parallel Hyperp...
11/29/2021

Automated Benchmark-Driven Design and Explanation of Hyperparameter Optimizers

Automated hyperparameter optimization (HPO) has gained great popularity ...
07/20/2018

Fast transforms over finite fields of characteristic two

An additive fast Fourier transform over a finite field of characteristic...

Please sign up or login with your details

Forgot password? Click here to reset