Validation of a simulation model for FaaS performance benchmarking using predictive validation
share
In the paper Controlling Garbage Collection and Request Admission to Improve Performance of FaaS Applications, we verified and evaluated the impact of memory management mechanics of programming languages in the context of Functions as a Service (FaaS) via simulation experiments. The results of this study pointed to an impact of up to 11.68 when a garbage collector procedure was executed during the execution of a CPU-bound function. As future work, we listed a few threats to the validity of the results attained, and among them, we cited the validation of the simulation model used. The validation of the model is important because it validates the results generated in the simulation experiments, which ensures realistic results. In this work, we proposed and executed a validation to the simulation model used in the previous work. To do so, we run measurement experiments in a public FaaS platform and simulation experiments of the same scenarios using the same simulator of the previous paper. Then, we validate the simulator by comparing the results obtained in both experiments to ensure that the simulation result and the measurement one are equivalent.
READ FULL TEXT