Do the Online Activities of Amazon Mechanical Turk Workers Mirror Those of the General Population? A Comparison of Two Survey Samples

Authors

  • Aaron Shaw Associate Professor in the Department of Communication Studies at Northwestern University and a Faculty Associate of the Berkman Klein Center for Internet and Society at Harvard University.
  • Eszter Hargittai University of Zurich

Keywords:

Amazon Mechanical Turk, survey methods, data bias, social media use, online participation

Abstract

Amazon Mechanical Turk (AMT) offers a relatively low-cost alternative to traditional expensive survey samples, which likely explains its popularity among survey researchers. An important question about using such samples is whether they are representative of the larger Internet user population. Though prior research has addressed this question about demographic characteristics, little work has examined how AMT workers compare with others regarding their online activities—namely, social media experiences and online active engagement. This article analyzes survey data administered concurrently on an AMT and a national sample of U.S. adults to show that AMT workers are significantly more likely to use numerous social media, from Twitter to Pinterest and Reddit, as well as have significantly more experiences contributing their own online content, from posting videos to participating in various online forums and signing online petitions. The article discusses the implications of these findings for research that uses AMT as a sampling frame when examining questions related to social media use and active online engagement.

Author Biographies

Aaron Shaw, Associate Professor in the Department of Communication Studies at Northwestern University and a Faculty Associate of the Berkman Klein Center for Internet and Society at Harvard University.

Aaron Shaw studies collective action, collaboration, and mobilization online. His current research projects address the the organizational factors that determine whether efforts to create public goods in peer production communities (like Wikipedia) are effective or not. He holds degrees from Stanford University and UC Berkeley and is a Faculty Associate of the Berkman Center for Internet & Society at Harvard University.

Eszter Hargittai, University of Zurich

Professor

Downloads

Published

2021-10-12

Issue

Vol. 15 (2021)

Section

Articles

License

The International Journal of Communication is an academic journal. As such, it is dedicated to the open exchange of information. For this reason, IJoC is freely available to individuals and institutions. Copies of this journal or articles in this journal may be distributed for research or educational purposes free of charge and without permission. However, commercial use of the IJoC website or the articles contained herein is expressly prohibited without the written consent of the editor. Authors who publish in The International Journal of Communication will release their articles under the Creative Commons Attribution Non-Commercial No Derivatives (by-nc-nd) license. This license allows anyone to copy and distribute the article for non-commercial purposes provided that appropriate attribution is given. For details of the rights authors grants users of their work, see the "human-readable summary" of the license, with a link to the full license. (Note that "you" refers to a user, not an author, in the summary.) This journal utilizes the LOCKSSsystem to create a distributed archiving system among participating libraries and permits those libraries to create permanent archives of the journal for purposes of preservation and restoration. The publisher perpetually authorizes participants in the LOCKSS system to archive and restore our publication through the LOCKSS System for the benefit of all LOCKSS System participants. Specifically participating libraries may:
  • Collect and preserve currently accessible materials;
  • Use material consistent with original license terms;
  • Provide copies to other LOCKSS appliances for purposes of audit and repair.
 

Fair Use

The U.S. Copyright Act of 1976 specifies, in Section 107, the terms of the Fair Use exception: Notwithstanding the provisions of sections 106 and 106A, the fair use of a copyrighted work, including such use by reproduction in copies or phonorecords or by any other means specified by that section, for purposes such as criticism, comment, news reporting, teaching (including multiple copies for classroom use), scholarship, or research, is not an infringement of copyright. In determining whether the use made of a work in any particular case is a fair use the factors to be considered shall include:
  1. the purpose and character of the use, including whether such use is of a commercial nature or is for nonprofit educational purposes;
  2. the nature of the copyrighted work;
  3. the amount and substantiality of the portion used in relation to the copyrighted work as a whole; &
  4. the effect of the use upon the potential market for or value of the copyrighted work.

The fact that a work is unpublished shall not itself bar a finding of fair use if such finding is made upon consideration of all the above factors. In accord with these provisions, the International Journal of Communication believes in the vigorous assertion and defense of Fair Use by scholars engaged in academic research, teaching and non-commercial publishing. Thus, we view the inclusion of “quotations” from existing print, visual, audio and audio-visual texts to be appropriate examples of Fair Use, as are reproductions of visual images for the purpose of scholarly analysis. We encourage authors to obtain appropriate permissions to use materials originally produced by others, but do not require such permissions as long as the usage of such materials falls within the boundaries of Fair Use.

The International Journal of Communication encourages authors to employ fair use in their scholarly publishing wherever appropriate. Fair use is the right to use unlicensed copyrighted material (whether it is text, images, audio-visual, or other) in your own work, in some circumstances. We consult the Code of Best Practices in Fair Use for Scholarly Research in Communication, created by the International Communication Association and endorsed by the National Communication Association, and you should too. If you have any questions about whether fair use applies to your uses of copyrighted material (whether it is text, images, audio-visual, or other) in your scholarship, simply include your rationale, grounded in the Best Practices, as a supplementary document with your submission.