The Diversity of Gamification Evaluation in the Software Engineering Education and Industry: Trends, Comparisons and Gaps

Gamification has been used to motivate and engage participants in software engineering education and practice activities. There is a significant demand for empirical studies for the understanding of the impacts and efficacy of gamification. However, the lack of standard procedures and models for the evaluation of gamification is a challenge for the design, comparison, and report of results related to the assessment of gamification approaches and its effects. The goal of this study is to identify models and strategies for the evaluation of gamification reported in the literature. To achieve this goal, we conducted a systematic mapping study to investigate strategies for the evaluation of gamification in the context of software engineering. We selected 100 primary studies on gamification in software engineering (from 2011 to 2020). We categorized the studies regarding the presence of evaluation procedures or models for the evaluation of gamification, the purpose of the evaluation, the criteria used, the type of data, instruments, and procedures for data analysis. Our results show that 64 studies report procedures for the evaluation of gamification. However, only three studies actually propose evaluation models for gamification. We observed that the evaluation of gamification focuses on two aspects: the evaluation of the gamification strategy itself, related to the user experience and perceptions; and the evaluation of the outcomes and effects of gamification on its users and context. The most recurring criteria for the evaluation are 'engagement', 'motivation', 'satisfaction', and 'performance'. Finally, the evaluation of gamification requires a mix of subjective and objective inputs, and qualitative and quantitative data analysis approaches. Depending of the focus of the evaluation (the strategy or the outcomes), there is a predominance of a type of data and analysis.

READ FULL TEXT
research
02/07/2018

Blueprint and Evaluation Instruments for a Course on Software Engineering for Sustainability

We report on a summer school course on Software Engineering for Sustaina...
research
05/15/2023

A Systematic Mapping Study and Practitioner Insights on the Use of Software Engineering Practices to Develop MVPs

[Background] The MVP concept has influenced the way in which development...
research
12/05/2018

Closing the gap between software engineering education and industrial needs

According to different reports, many recent software engineering graduat...
research
12/04/2017

On Using Network Science in Mining Developers Collaboration in Software Engineering: A Systematic Literature Review

The goal of this study is to identify, review, and analyze the published...
research
12/08/2019

A Flipped Classroom Approach to Teaching Empirical Software Engineering

Contribution: This study shows that a flipped classroom approach to teac...
research
12/29/2020

Research in Global Software Engineering: A Systematic Snapshot

This paper reports our extended analysis of the recent literature addres...
research
11/13/2018

Bayesian Data Analysis in Empirical Software Engineering Research

Statistics comes in two main flavors: frequentist and Bayesian. For hist...

Please sign up or login with your details

Forgot password? Click here to reset