Incremental Calibration of Architectural Performance Models with Parametric Dependencies

by   Manar Mazkatli, et al.

Architecture-based Performance Prediction (AbPP) allows evaluation of the performance of systems and to answer what-if questions without measurements for all alternatives. A difficulty when creating models is that Performance Model Parameters (PMPs, such as resource demands, loop iteration numbers and branch probabilities) depend on various influencing factors like input data, used hardware and the applied workload. To enable a broad range of what-if questions, Performance Models (PMs) need to have predictive power beyond what has been measured to calibrate the models. Thus, PMPs need to be parametrized over the influencing factors that may vary. Existing approaches allow for the estimation of parametrized PMPs by measuring the complete system. Thus, they are too costly to be applied frequently, up to after each code change. They do not keep also manual changes to the model when recalibrating. In this work, we present the Continuous Integration of Performance Models (CIPM), which incrementally extracts and calibrates the performance model, including parametric dependencies. CIPM responds to source code changes by updating the PM and adaptively instrumenting the changed parts. To allow AbPP, CIPM estimates the parametrized PMPs using the measurements (generated by performance tests or executing the system in production) and statistical analysis, e.g., regression analysis and decision trees. Additionally, our approach responds to production changes (e.g., load or deployment changes) and calibrates the usage and deployment parts of PMs accordingly. For the evaluation, we used two case studies. Evaluation results show that we were able to calibrate the PM incrementally and accurately.


Automated Identification of Performance Changes at Code Level

To develop software with optimal performance, even small performance cha...

An approach for Test Impact Analysis on the Integration Level in Java programs

Test Impact Analysis is an approach to obtain a subset of tests impacted...

Learning to Learn to Predict Performance Regressions in Production at Meta

Catching and attributing code change-induced performance regressions in ...

How Do Code Changes Evolve in Different Platforms? A Mining-based Investigation

Code changes are performed differently in the mobile and non-mobile plat...

Adversarial Learning Networks: Source-free Unsupervised Domain Incremental Learning

This work presents an approach for incrementally updating deep neural ne...

Time-evolving psychological processes over repeated decisions

Many psychological experiments have participants repeat a simple task. T...

How is ChatGPT's behavior changing over time?

GPT-3.5 and GPT-4 are the two most widely used large language model (LLM...

Please sign up or login with your details

Forgot password? Click here to reset