Amazon Mechanical Turk: Digital sweatshop or second-job safety net?

 

Amazon Mechanical Turk (AMT) is an online marketplace that allows companies and developers (“Requesters”) to outsource “microtasks” to humans in the form of “Human Intelligence Tasks” (HITs). These tasks pay fractions of a dollar and can be done in a very short time, such that workers will complete tens or hundreds of these tasks at a time. These workers, known as “Turkers”, are not well understood.

In their 2014 paper “Being a Turker”, David Martin et al convey their findings from an ethnomethodological analysis of Turker Nation, a discussion forum frequented by Turkers. The general approach was simply to read and explore the threads on the forum to uncover attitudes, motivations and issues (and to follow the debate off-site through links where appropriate).

Findings from the research

Several interesting insights were discovered:

• Contrary to prevailing rumours and expectations, Turkers are primarily motivated by money – though they will occasionally choose a more interesting task for lower pay to break the tedium. However, Turkers find portrayals of Turking as a leisurely pursuit highly damaging to perceptions, as it encourages poor pay. Turkers want to be fairly paid and fairly treated, like any employees.
• Many Turkers work on AMT as a secondary source of income in their spare time:

I usually plop on the couch, put the TV on and zone out with my laptop. The pay rate kinda sucks but […] it’s not a bad option when there is nothing going on and you don’t want to spend an hour on unpaid searching.

• This doesn’t mean it’s not an important income, though. For some, the money they earn from AMT is critical to their survival, to the extent that they can feel somewhat trapped :

It is a full part-time job for me. I can turk during slow periods at my day job. […] Because of pay cuts I would have to work a part-time job if I did not turk. […] My boss is aware […] but I would much rather have my salary back and drop turking.

• The system is very unbalanced against workers. Requesters have little visibility of the humans behind their tasks and have a lot of power to block Turkers or reject their work, sometimes at great financial loss to the workers, without explanation being required, while the Turkers are not given the mechanisms to discuss these decisions or challenge them. As a result, there is much discussion on TurkNation and other forums on “good” and “bad” Requester behaviour – but little opportunity to share this information with Requesters to educate them.
• There are many ethical issues created by this imbalance – Turkers being falsely accused of cheating or scamming, even when trying to help.
• As a community, Turkers (or at least those on Turk Nation) are highly moral and self-critical. They take responsibility for their actions, share good and bad practices, and analyse situations that have gone wrong to uncover technical or process issues.
• Turkers are in favour of enforcement of Amazon’s terms of service on Requesters and Turkers, but they are against legislation and regulation. They believe their power to influence and manage the market comes most fruitfully from collective individual actions, such as rejecting bad payers, boycotting poor Requesters etc.

If laws pass regulating hour wages on mTurk, requesters will flee for the hills and we’ll be fucked. Journalists, LEAVE US ALONE! We don’t want your help.

Conclusions & My Thoughts

The authors conclude by underlining the importance of making the “invisible” work of Turkers visible. The design of AMT as an “API for human computation” means that Turkers are sometimes misunderstood and denigrated.  They call for more participatory design of tools that Turkers can use to cooperate and exert more control on the function of the market and make better decisions.

On the whole I found the paper insightful and interesting. Much of the research so far has been focussed on the Requester’s perspective, looking at AMT purely as a computational resource, and in my view the authors have achieved their aim of redressing this imbalance and bringing the workers’ perspective to the forefront.

If I was to criticise it, I would say that the paper is very impression-based, and as such it’s hard to be sure if the findings reflect a representative sample of all Turkers, given that we could reasonably expect that TurkNation posters are only a small subset of all Turkers, as on a site like TripAdvisor. It’s likely there are a large number of Turkers who do not collaborate with others or discuss their working practices at all. This is likely to be a limitation of almost any ethnomethodological study though.

Reading the paper, I drew strong parallels with the current debates and legal cases happening about whether Uber drivers should be considered as employees. In both cases, we have a workforce that is somewhat secondary or invisible, often working as a second job or to provide necessary additional income. The insights of this paper have helped me to realise that sometimes well-meaning efforts to protect workers’ rights can work against the common interest of workers, and indeed this is borne out in the Uber community as it is here – 76% of Uber drivers would prefer not to be employees, despite the recent ruling that says they should be. Perhaps Uber’s rejection of the ruling should be supported, contrary to my instinct prior to reading this paper. There is clearly a place in modern society for low-paid, casual work that can be done more informally than traditional employment arrangements, and Turking is a prime example of this. Turkers, like Uber drivers, face ethical issues and uncertainty due to their status as an invisible workforce, and as this “gig economy” grows, this issue will remain a hot topic for modern digital society.

 

---

The paper I have found which exemplifies a methodological issue in crowdsourcing is “I want to be a Captain! I want to be a Captain!”: Gamification in the Old Weather Citizen Science Project (2013) by Eveleigh et al.

This is an interesting paper because:

• It touches on from one question raised in the above paper, the question of whether human computation tasks should be seen (and presented) as work, or as a “fun” task.
• It explores the impacts of gamification in a crowdsourced citizen science project on the Zooniverse (where I previously worked as Web Science Architect) and raises the methodological question of whether, and to what extent, making micro-tasks more game-like can affect worker productivity.
• This is also interesting as it looks at a volunteer-based project, where motivation is perhaps an even more important question, in contrast to paid human computation platforms such as Amazon Mechanical Turk – though for both, work quality is equally important.