Crowdsourcing platforms like Amazon’s Mechanical Turk (AMT) make it possible to assign human workers small ‘microtasks’, such as labeling images or detecting duplicate products, in order to apply the power of human intellect at scale to problems that cannot be fully automated. These platforms often provide programmatic interfaces for microtask management upon which those of us researching the ‘P’ in ‘AMP’ rely heavily. Unfortunately, using those APIs to manage the lifecycle of human computation tasks can be a real hassle. For example, a standard workflow when using a crowd platform like AMT to detect duplicate products involves:

• Designing a task interface in HTML, Javascript and CSS to allow users to select pairs of products that are identical.
• Implementing and hosting a web service (with ssl support) to present the task interface to workers on the AMT website.
• Using the AMT API to create a batch of new tasks and to send each task to multiple workers to ensure high quality results.
• Using the AMT API to fetch results when the batch has been processed by the workers.
• Writing custom code to merge the responses of multiple workers for the same task (either simple majority voting, or more sophisticated quality control).
• Storing the results in a database for future access.

Though much of the code supporting this workflow can theoretically be reused in subsequent crowd-based projects, it seldom turns out that way in practice. In response, we’re releasing AMPCrowd: an extensible web service with a simple RESTful API for managing the crowd task lifecycle. AMPCrowd makes it easy to run crowd tasks on existing platforms, is extensible to new types of microtasks and new crowd platforms, and provides state-of-the-art quality control techniques with no user effort.

Using AMPCrowd, the workflow described above can be accomplished with a single declarative API call to create a new batch of tasks. The user specifies the type of the tasks (e.g. ‘duplicate detection’), the number of distinct worker responses to get for each task, the crowd platform on which to process the tasks (current options are AMT and a local interface for in-house workers), and a JSON dictionary with the data needed to render each task (e.g. the pairs of products to consider). AMPCrowd transparently handles the work of sending tasks to the selected crowd platform, fetching results as workers complete them, and running quality control algorithms to improve the resulting output. Those looking to get results in real-time can pass in a callback URL to receive push notifications as workers finish each individual task. Otherwise, the results are available in AMPCrowd’s PostgreSQL database.

AMPCrowd’s modular design makes it easy to add both new task types and support for new crowd platforms. Adding a new task type is as simple as writing the HTML to display a single task. Adding support for a new crowd platform is a bit more involved, but can be done by implementing a common interface for creating and managing tasks in a standalone Django app–no need to modify existing code.

AMPCrowd is implemented in python using the Django web framework, and a recent contribution from one of our collaborators (thanks @EntilZhaPR at Trulia!) provides Docker support, so the code can be deployed with a single command. Check out the repository at https://github.com/amplab/ampcrowd, or read our (sparse but growing) documentation at http://amplab.github.io/ampcrowd/.