It's about time: Online Macrotask Sequencing in Expert Crowdsourcing

by   Heinz Schmitz, et al.

We introduce the problem of Task Assignment and Sequencing (TAS), which adds the timeline perspective to expert crowdsourcing optimization. Expert crowdsourcing involves macrotasks, like document writing, product design, or web development, which take more time than typical binary microtasks, require expert skills, assume varying degrees of knowledge over a topic, and require crowd workers to build on each other's contributions. Current works usually assume offline optimization models, which consider worker and task arrivals known and do not take into account the element of time. Realistically however, time is critical: tasks have deadlines, expert workers are available only at specific time slots, and worker/task arrivals are not known a-priori. Our work is the first to address the problem of optimal task sequencing for online, heterogeneous, time-constrained macrotasks. We propose tas-online, an online algorithm that aims to complete as many tasks as possible within budget, required quality and a given timeline, without future input information regarding job release dates or worker availabilities. Results, comparing tas-online to four typical benchmarks, show that it achieves more completed jobs, lower flow times and higher job quality. This work has practical implications for improving the Quality of Service of current crowdsourcing platforms, allowing them to offer cost, quality and time improvements for expert tasks.


page 1

page 2

page 3

page 4


A Truthful Mechanism with Biparameter Learning for Online Crowdsourcing

We study a problem of allocating divisible jobs, arriving online, to wor...

Comparing Generic and Community-Situated Crowdsourcing for Data Validation in the Context of Recovery from Substance Use Disorders

Targeting the right group of workers for crowdsourcing often achieves be...

Tuning Crowdsourced Human Computation

As the use of crowdsourcing increases, it is important to think about pe...

Online Policies for Efficient Volunteer Crowdsourcing

Nonprofit crowdsourcing platforms such as food recovery organizations re...

Exploration vs. Exploitation in Team Formation

An online labor platform faces an online learning problem in matching wo...

CrowdOS: A Ubiquitous Operating System for Crowdsourcing and Mobile Crowd Sensing

With the rise of crowdsourcing and mobile crowdsensing techniques, a lar...

Octopus: A Framework for Cost-Quality-Time Optimization in Crowdsourcing

We present Octopus, an AI agent to jointly balance three conflicting tas...

Please sign up or login with your details

Forgot password? Click here to reset