Agent-based Collaborative Random Search for Hyper-parameter Tuning and Global Function Optimization

by   Ahmad Esmaeili, et al.

Hyper-parameter optimization is one of the most tedious yet crucial steps in training machine learning models. There are numerous methods for this vital model-building stage, ranging from domain-specific manual tuning guidelines suggested by the oracles to the utilization of general-purpose black-box optimization techniques. This paper proposes an agent-based collaborative technique for finding near-optimal values for any arbitrary set of hyper-parameters (or decision variables) in a machine learning model (or general function optimization problem). The developed method forms a hierarchical agent-based architecture for the distribution of the searching operations at different dimensions and employs a cooperative searching procedure based on an adaptive width-based random sampling technique to locate the optima. The behavior of the presented model, specifically against the changes in its design parameters, is investigated in both machine learning and global function optimization applications, and its performance is compared with that of two randomized tuning strategies that are commonly used in practice. According to the empirical results, the proposed model outperformed the compared methods in the experimented classification, regression, and multi-dimensional function optimization tasks, notably in a higher number of dimensions and in the presence of limited on-device computational resources.


page 1

page 2

page 3

page 4


Hierarchical Collaborative Hyper-parameter Tuning

Hyper-parameter Tuning is among the most critical stages in building mac...

Hybrid Algorithm Selection and Hyperparameter Tuning on Distributed Machine Learning Resources: A Hierarchical Agent-based Approach

Algorithm selection and hyperparameter tuning are critical steps in both...

A Comparative study of Hyper-Parameter Optimization Tools

Most of the machine learning models have associated hyper-parameters alo...

Behavior of Hyper-Parameters for Selected Machine Learning Algorithms: An Empirical Investigation

Hyper-parameters (HPs) are an important part of machine learning (ML) mo...

A Cooperative Enterprise Agent Based Control Architecture

The paper proposes a hierarchical, agent-based, DES supported, distribut...

Improved Bilevel Model: Fast and Optimal Algorithm with Theoretical Guarantee

Due to the hierarchical structure of many machine learning problems, bil...

Please sign up or login with your details

Forgot password? Click here to reset