On Using Retrained and Incremental Machine Learning for Modeling Performance of Adaptable Software: An Empirical Comparison

by   Tao Chen, et al.

Given the ever-increasing complexity of adaptable software systems and their commonly hidden internal information (e.g., software runs in the public cloud), machine learning based performance modeling has gained momentum for evaluating, understanding and predicting software performance, which facilitates better informed self-adaptations. As performance data accumulates during the run of the software, updating the performance models becomes necessary. To this end, there are two conventional modeling methods: the retrained modeling that always discard the old model and retrain a new one using all available data; or the incremental modeling that retains the existing model and tunes it using one newly arrival data sample. Generally, literature on machine learning based performance modeling for adaptable software chooses either of those methods according to a general belief, but they provide insufficient evidences or references to justify their choice. This paper is the first to report on a comprehensive empirical study that examines both modeling methods under distinct domains of adaptable software, 5 performance indicators, 8 learning algorithms and settings, covering a total of 1,360 different conditions. Our findings challenge the general belief, which is shown to be only partially correct, and reveal some of the important, statistically significant factors that are often overlooked in existing work, providing evidence-based insights on the choice.


page 1

page 8

page 9

page 10


Software Engineers vs. Machine Learning Algorithms: An Empirical Study Assessing Performance and Reuse Tasks

Several papers have recently contained reports on applying machine learn...

Deep Learning for Choice Modeling

Choice modeling has been a central topic in the study of individual pref...

A Validation and Quality Assessment Method with Metamorphic Relations for Unsupervised Machine Learning Software

Unsupervised machine learning is a task of modeling the underlying struc...

Security for Machine Learning-based Software Systems: a survey of threats, practices and challenges

The rapid development of Machine Learning (ML) has demonstrated superior...

On the choice of training data for machine learning of geostrophic mesoscale turbulence

'Data' plays a central role in data-driven methods, but is not often the...

A Review On Software Defects Prediction Methods

Software quality is one of the essential aspects of a software. With inc...

What Makes You Hold on to That Old Car? Joint Insights from Machine Learning and Multinomial Logit on Vehicle-level Transaction Decisions

What makes you hold on that old car? While the vast majority of the hous...

Please sign up or login with your details

Forgot password? Click here to reset