Beyond Accuracy: ROI-driven Data Analytics of Empirical Data

by   Gouri Deshpande, et al.

This vision paper demonstrates that it is crucial to consider Return-on-Investment (ROI) when performing Data Analytics. Decisions on "How much analytics is needed"? are hard to answer. ROI could guide for decision support on the What?, How?, and How Much? analytics for a given problem. Method: The proposed conceptual framework is validated through two empirical studies that focus on requirements dependencies extraction in the Mozilla Firefox project. The two case studies are (i) Evaluation of fine-tuned BERT against Naive Bayes and Random Forest machine learners for binary dependency classification and (ii) Active Learning against passive Learning (random sampling) for REQUIRES dependency extraction. For both the cases, their analysis investment (cost) is estimated, and the achievable benefit from DA is predicted, to determine a break-even point of the investigation. Results: For the first study, fine-tuned BERT performed superior to the Random Forest, provided that more than 40 Active Learning achieved higher F1 accuracy within fewer iterations and higher ROI compared to Baseline (Random sampling based RF classifier). In both the studies, estimate on, How much analysis likely would pay off for the invested efforts?, was indicated by the break-even point. Conclusions: Decisions for the depth and breadth of DA of empirical data should not be made solely based on the accuracy measures. Since ROI-driven Data Analytics provides a simple yet effective direction to discover when to stop further investigation while considering the cost and value of the various types of analysis, it helps to avoid over-analyzing empirical data.


page 1

page 2

page 3

page 4


Smartpick: Workload Prediction for Serverless-enabled Scalable Data Analytics Systems

Many data analytic systems have adopted a newly emerging compute resourc...

Ensemble Classifier Design Tuned to Dataset Characteristics for Network Intrusion Detection

Machine Learning-based supervised approaches require highly customized a...

Software Engineering for IoT-Driven Data Analytics Applications

Internet of Things Driven Data Analytics (IoT-DA) has the potential to e...

Prediction of paddy cultivation using deep learning on land cover variation for sustainable agriculture

Geospatial analytics is a promising method of spatial data processing an...

Does Informativeness Matter? Active Learning for Educational Dialogue Act Classification

Dialogue Acts (DAs) can be used to explain what expert tutors do and wha...

Please sign up or login with your details

Forgot password? Click here to reset