Efficient Hyperparameter Tuning with Dynamic Accuracy Derivative-Free Optimization

by   Matthias J. Ehrhardt, et al.
University of Bath
Australian National University

Many machine learning solutions are framed as optimization problems which rely on good hyperparameters. Algorithms for tuning these hyperparameters usually assume access to exact solutions to the underlying learning problem, which is typically not practical. Here, we apply a recent dynamic accuracy derivative-free optimization method to hyperparameter tuning, which allows inexact evaluations of the learning problem while retaining convergence guarantees. We test the method on the problem of learning elastic net weights for a logistic classifier, and demonstrate its robustness and efficiency compared to a fixed accuracy approach. This demonstrates a promising approach for hyperparameter tuning, with both convergence guarantees and practical performance.


page 1

page 2

page 3

page 4


Autotune: A Derivative-free Optimization Framework for Hyperparameter Tuning

Machine learning applications often require hyperparameter tuning. The h...

A Region-Shrinking-Based Acceleration for Classification-Based Derivative-Free Optimization

Derivative-free optimization algorithms play an important role in scient...

Relax and penalize: a new bilevel approach to mixed-binary hyperparameter optimization

In recent years, bilevel approaches have become very popular to efficien...

Fast Hyperparameter Tuning for Ising Machines

In this paper, we propose a novel technique to accelerate Ising machines...

The Statistical Cost of Robust Kernel Hyperparameter Tuning

This paper studies the statistical complexity of kernel hyperparameter t...

MindOpt Tuner: Boost the Performance of Numerical Software by Automatic Parameter Tuning

Numerical software is usually shipped with built-in hyperparameters. By ...

Enhanced Self-Organizing Map Solution for the Traveling Salesman Problem

Using an enhanced Self-Organizing Map method, we provided suboptimal sol...

Please sign up or login with your details

Forgot password? Click here to reset