Tune: A Research Platform for Distributed Model Selection and Training

by   Richard Liaw, et al.
berkeley college

Modern machine learning algorithms are increasingly computationally demanding, requiring specialized hardware and distributed computation to achieve high performance in a reasonable time frame. Many hyperparameter search algorithms have been proposed for improving the efficiency of model selection, however their adaptation to the distributed compute environment is often ad-hoc. We propose Tune, a unified framework for model selection and training that provides a narrow-waist interface between training scripts and search algorithms. We show that this interface meets the requirements for a broad range of hyperparameter search algorithms, allows straightforward scaling of search to large clusters, and simplifies algorithm implementation. We demonstrate the implementation of several state-of-the-art hyperparameter search algorithms in Tune. Tune is available at http://ray.readthedocs.io/en/latest/tune.html.


page 1

page 2

page 3

page 4


Learning Multiple Defaults for Machine Learning Algorithms

The performance of modern machine learning methods highly depends on the...

PyTorch Hyperparameter Tuning - A Tutorial for spotPython

The goal of hyperparameter tuning (or hyperparameter optimization) is to...

Orchestrate: Infrastructure for Enabling Parallelism during Hyperparameter Optimization

Two key factors dominate the development of effective production grade m...

AutoML-GPT: Large Language Model for AutoML

With the emerging trend of GPT models, we have established a framework c...

Tune As You Scale: Hyperparameter Optimization For Compute Efficient Training

Hyperparameter tuning of deep learning models can lead to order-of-magni...

DEHB: Evolutionary Hyberband for Scalable, Robust and Efficient Hyperparameter Optimization

Modern machine learning algorithms crucially rely on several design deci...

Towards Practical Learned Indexing

Latest research proposes to replace existing index structures with learn...

Please sign up or login with your details

Forgot password? Click here to reset