Efficient crowdsourcing of crowd-generated microtasks

by   Abigail Hotaling, et al.

Allowing members of the crowd to propose novel microtasks for one another is an effective way to combine the efficiencies of traditional microtask work with the inventiveness and hypothesis generation potential of human workers. However, microtask proposal leads to a growing set of tasks that may overwhelm limited crowdsourcer resources. Crowdsourcers can employ methods to utilize their resources efficiently, but algorithmic approaches to efficient crowdsourcing generally require a fixed task set of known size. In this paper, we introduce *cost forecasting* as a means for a crowdsourcer to use efficient crowdsourcing algorithms with a growing set of microtasks. Cost forecasting allows the crowdsourcer to decide between eliciting new tasks from the crowd or receiving responses to existing tasks based on whether or not new tasks will cost less to complete than existing tasks, efficiently balancing resources as crowdsourcing occurs. Experiments with real and synthetic crowdsourcing data show that cost forecasting leads to improved accuracy. Accuracy and efficiency gains for crowd-generated microtasks hold the promise to further leverage the creativity and wisdom of the crowd, with applications such as generating more informative and diverse training data for machine learning applications and improving the performance of user-generated content and question-answering platforms.


page 1

page 2

page 3

page 4


Contributors profile modelization in crowdsourcing platforms

The crowdsourcing consists in the externalisation of tasks to a crowd of...

Accurate inference of crowdsourcing properties when using efficient allocation strategies

Allocation strategies improve the efficiency of crowdsourcing by decreas...

Crowdsourcing Feature Discovery via Adaptively Chosen Comparisons

We introduce an unsupervised approach to efficiently discover the underl...

Using the Crowd to Generate Content for Scenario-Based Serious-Games

In the last decade, scenario-based serious-games have become a main tool...

C2A: Crowd Consensus Analytics for Virtual Colonoscopy

We present a medical crowdsourcing visual analytics platform called C^2A...

On Efficient and Scalable Time-Continuous Spatial Crowdsourcing – Full Version

The proliferation of advanced mobile terminals opened up a new crowdsour...

CrowdOS: A Ubiquitous Operating System for Crowdsourcing and Mobile Crowd Sensing

With the rise of crowdsourcing and mobile crowdsensing techniques, a lar...

Please sign up or login with your details

Forgot password? Click here to reset