A Survey on Multi-Objective based Parameter Optimization for Deep Learning

by   Mrittika Chakraborty, et al.

Deep learning models form one of the most powerful machine learning models for the extraction of important features. Most of the designs of deep neural models, i.e., the initialization of parameters, are still manually tuned. Hence, obtaining a model with high performance is exceedingly time-consuming and occasionally impossible. Optimizing the parameters of the deep networks, therefore, requires improved optimization algorithms with high convergence rates. The single objective-based optimization methods generally used are mostly time-consuming and do not guarantee optimum performance in all cases. Mathematical optimization problems containing multiple objective functions that must be optimized simultaneously fall under the category of multi-objective optimization sometimes referred to as Pareto optimization. Multi-objective optimization problems form one of the alternatives yet useful options for parameter optimization. However, this domain is a bit less explored. In this survey, we focus on exploring the effectiveness of multi-objective optimization strategies for parameter optimization in conjunction with deep neural networks. The case studies used in this study focus on how the two methods are combined to provide valuable insights into the generation of predictions and analysis in multiple applications.


page 1

page 2

page 3

page 4


Explicitly Multi-Modal Benchmarks for Multi-Objective Optimization

We model multi-modality in multi-objective optimization problems and app...

Multi-Objective Hull Form Optimization with CAD Engine-based Deep Learning Physics for 3D Flow Prediction

In this work, we propose a built-in Deep Learning Physics Optimization (...

MEESO: A Multi-objective End-to-End Self-Optimized Approach for Automatically Building Deep Learning Models

Deep learning has been widely used in various applications from differen...

A survey on multi-objective hyperparameter optimization algorithms for Machine Learning

Hyperparameter optimization (HPO) is a necessary step to ensure the best...

Mitigating Gradient Bias in Multi-objective Learning: A Provably Convergent Stochastic Approach

Machine learning problems with multiple objective functions appear eithe...

Artificial Intelligence-Assisted Optimization and Multiphase Analysis of Polygon PEM Fuel Cells

This article presents new PEM fuel cell models with hexagonal and pentag...

Deep learning based Meta-modeling for Multi-objective Technology Optimization of Electrical Machines

Optimization of rotating electrical machines is both time- and computati...

Please sign up or login with your details

Forgot password? Click here to reset