Efficient Online Crowdsourcing

Efficient online crowdsourcing aims to leverage human intelligence for complex tasks while minimizing cost and maximizing accuracy. Current research focuses on improving the cost-quality trade-off through advanced algorithms that dynamically allocate tasks and assess worker reliability, often employing techniques like combinatorial multi-armed bandits for worker selection and sophisticated truth discovery methods for aggregating diverse annotations, even for complex data types like bounding boxes and sequences. These advancements are impacting various fields, from public policy development to natural language processing, by enabling more efficient and accurate data collection and analysis for large-scale projects.