A Large-Scale Comparative Study of Beta Testers and Standard Users

by   Vlasta Stavova, et al.

Beta testers are the first end users outside a software company to use its product. They have been used for decades and are rightly credited not only with finding and reporting bugs, but also with improving general product usability through their feedback and/or the ways they use the product. In this paper, we investigate whether beta testers represent standard users well enough to allow for the extrapolation of testing data to standard users. We have investigated records of beta testers and standard users of home security solution developed by the IT security software provider ESET. With more than 600 000 participants from more than 180 countries, we present what we believe to be the first large-scale comparison between standard users and beta testers. We compared several aspects of both populations, such as hardware, operating system, country of origin and EULA reading time, all taken from system data. Other attributes, such as age, gender, privacy perception and computer proficiency self-evaluation, were available thanks to a user questionnaire. We conclude that - at least in our study - beta users represent standard users well in terms of hardware and operating system in large scale beta testing. However, populations differ significantly in the distribution of users and testers between countries. This may cause a problem when a testing includes localization and usability issues that may be influenced by regional differences.


page 3

page 4

page 6


On ordered beta distribution and the generalized incomplete beta function

Motivated by applications in Bayesian analysis we introduce a multidimen...

Higher moments of the multivariate Beta distribution

Beta distribution has been shown a useful model in many statistical prob...

Beta Survival Models

This article analyzes the problem of estimating the time until an event ...

A World Full of Privacy and Security (Mis)conceptions? Findings of a Representative Survey in 12 Countries

Misconceptions about digital security and privacy topics in the general ...

The Beta-Bernoulli process and algebraic effects

In this paper we analyze the Beta-Bernoulli process from Bayesian nonpar...

The statistical properties of RCTs and a proposal for shrinkage

We abstract the concept of a randomized controlled trial (RCT) as a trip...

Paperclickers: Affordable Solution for Classroom Response Systems

We propose a low-cost classroom response system requiring a single mobil...

Please sign up or login with your details

Forgot password? Click here to reset