Can GitHub Issues Help in the App Review Classifications?
share
App reviews reflect various user requirements that can aid in planning maintenance tasks. Recently, proposed approaches for automatically classifying user reviews rely on machine learning algorithms. Devine et al. demonstrated that models trained on existing labeled datasets exhibit poor performance when predicting new ones. Although integrating datasets improves the results to some extent, there is still a need for greater generalizability to be taken into consideration. Therefore, a comprehensive labeled dataset is essential to train a more precise model. This paper introduces an approach to train a more generalizable model by leveraging information from an additional source, such as the GitHub issue tracking system, that contains valuable information about user requirements. We propose an approach that assists in augmenting labeled datasets by utilizing information extracted from GitHub issues. First, we identify issues concerning review intentions (bug reports, feature requests, and others) by examining the issue labels. Then, we analyze issue bodies and define 19 language patterns for extracting targeted information. Finally, we augment the manually labeled review dataset with a subset of processed issues through the Within-App, Within-Context, and Between-App Analysis methods. The first two methods train the app-specific models, and the last suits the general-purpose models. We conducted several experiments to evaluate the proposed approach. Our results demonstrate that using labeled issues for data augmentation can improve the F1-score and recall to 13.9 and 29.9 in the bug reports, respectively, and to 7.5 and 13.5 for feature requests. Furthermore, we identify an effective volume range of 0.3 to 0.7, which provides better performance improvements.
READ FULL TEXT