Too Many Bots: A Lesson for Online Quantitative Data Collection
DOI:
https://doi.org/10.18666/JPRA-2023-12011Keywords:
bots, quantitative analysis, methodology, online surveys, incentivesAbstract
“Bots”, computer software capable of taking surveys for an operator, pose a serious threat to the integrity of research that relies on publicly available online surveys. This paper addresses the issue of bot responses to online surveys and suggests several strategies for reducing and addressing these fraudulent responses. To combat this threat, researchers should employ specific methods for building, distributing, and processing surveys that deter and eliminate bot responses from the data set. Methods for anti-bot survey design include building in bot detection software to the survey, creating trap questions, and writing questions that require specific free-form answers. Survey distribution methods that avoid or hide monetary incentives, use a password-protected link, or employ some other form of population targeting will also receive less bot responses. Finally, data should be screened for bots after collection using a set of reliable criteria to identify and remove bot responses.
References
References
Cheng, Y. (Daniel), Farmer, J. R., Dickinson, S. L., Robeson, S. M., Fischer, B. C., & Reynolds, H. L. (2021). Climate change impacts and urban green space adaptation efforts: Evidence from U.S. municipal parks and recreation departments. Urban Climate, 39, 100962. https://doi.org/10.1016/j.uclim.2021.100962
Das, M., Ester, P., & Kaczmirek, L. (Eds.). (2010). Advances in Applied Methods and Research Strategies. Routledge. https://doi.org/10.4324/9780203844922
Griffin, M., Martino, R. J., LoSchiavo, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of internet bots. Quality & Quantity, 56(4), 2841–2852. https://doi.org/10.1007/s11135-021-01252-1
Henderson, K., Grappendorf, H., Bruton, C., & Tomas, S. (2013). The status of women in the parks and recreation profession in the United States. World Leisure Journal, 55(1), 58–71. https://doi.org/10.1080/04419057.2012.759142
Lebeuf, C., Zagalsky, A., Foucault, M., & Storey, M.-A. (2019). Defining and Classifying Software Bots: A Faceted Taxonomy. 2019 IEEE/ACM 1st International Workshop on Bots in Software Engineering (BotSE), 1–6. https://doi.org/10.1109/BotSE.2019.00008
Martínez-Jauregui, M., Delibes-Mateos, M., Arroyo, B., & Soliño, M. (2020). Addressing social attitudes toward lethal control of wildlife in national parks. Conservation Biology, 34(4), 868–878. https://doi.org/10.1111/cobi.13468
McMaster, H. S., LeardMann, C. A., Speigle, S., Dillman, D. A., & Millennium Cohort Family Study Team. (2017). An experimental comparison of web-push vs. Paper-only survey procedures for conducting an in-depth health survey of military spouses. BMC Medical Research Methodology, 17(1), 73. https://doi.org/10.1186/s12874-017-0337-1
Nwana, H. S. (1996). Software agents: An overview. The Knowledge Engineering Review, 11(3), 205–244. https://doi.org/10.1017/S026988890000789X
Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gundersen, D. A., & Berry, D. L. (2020). Threats of Bots and Other Bad Actors to Data Quality Following Research Participant Recruitment Through Social Media: Cross-Sectional Questionnaire. Journal of Medical Internet Research, 22(10), e23021. https://doi.org/10.2196/23021
Prince, K. R., Litovsky, A. R., & Friedman-Wheeler, D. G. (2012). Internet-mediated research: Beware of bots. The Behavior Therapist, 35, 85–88.
Rooney, T. (2010). Introduction to IP Address Management. John Wiley & Sons.
Schonlau, M., & Couper, M. P. (2017). Options for Conducting Web Surveys. Statistical Science, 32(2), 279–292.
Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got Bots? Practical Recommendations to Protect Online Survey Data from Bot Attacks. The Quantitative Methods for Psychology, 16(5), 472–481. https://doi.org/10.20982/tqmp.16.5.p472
Teitcher, J. E. F., Bockting, W. O., Bauermeister, J. A., Hoefer, C. J., Miner, M. H., & Klitzman, R. L. (2015). Detecting, preventing, and responding to “fraudsters” in internet research: Ethics and tradeoffs. The Journal of Law, Medicine & Ethics: A Journal of the American Society of Law, Medicine & Ethics, 43(1), 116–133. https://doi.org/10.1111/jlme.12200
Van Selm, M., & Jankowski, N. W. (2006). Conducting Online Surveys. Quality and Quantity, 40(3), 435–456. https://doi.org/10.1007/s11135-005-8081-8
Wang, Z., Qin, M., Chen, M., & Jia, C. (2018). Hiding Fast Flux Botnet in Plain Email Sight (X. Lin, A. Ghorbani, K. Ren, S. Zhu, & A. Zhang, Eds.; Vol. 239, pp. 182–197). Springer International Publishing. https://doi.org/10.1007/978-3-319-78816-6_14
Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman-Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. The Behavior Therapist, 42, 235–242.
Downloads
Published
Issue
Section
License
Sagamore Publishing LLC (hereinafter the “Copyright Owner”)
Journal Publishing Copyright Agreement for Authors
PLEASE REVIEW OUR POLICIES AND THE PUBLISHING AGREEMENT, AND INDICATE YOUR ACCEPTANCE OF THE TERMS BY CHECKING THE ‘AGREE TO THE TERMS OF THIS COPYRIGHT NOTICE’ CHECKBOX BELOW.
I understand that by submitting an article to Journal of Park and Recreation Administration, I am granting the copyright to the article submitted for consideration for publication in Journal of Park and Recreation Administration to the Copyright Owner. If after consideration of the Editor of the Journal of Park and Recreation Administration, the article is not accepted for publication, all copyright covered under this agreement will be automatically returned to the Author(s).
THE PUBLISHING AGREEMENT
Assignment of Copyright
I hereby assign to the Copyright Owner the copyright in the manuscript I am submitting in this online procedure and any tables, illustrations or other material submitted for publication as part of the manuscript in all forms and media (whether now known or later developed), throughout the world, in all languages, for the full term of copyright, effective when the article is accepted for publication.
Reversion of Rights
Articles may sometimes be accepted for publication but later be rejected in the publication process, even in some cases after public posting in “Articles in Press” form, in which case all rights will revert to the Author.
Retention of Rights for Scholarly Purposes
I understand that I retain or am hereby granted the Retained Rights. The Retained Rights include the right to use the Preprint, Accepted Manuscript, and the Published Journal Article for Personal Use and Internal Institutional Use.
All journal material is under a 12 month embargo. Authors who would like to have their articles available as open access should contact gbates@sagamorepub.com for further information.
In the case of the Accepted Manuscript and the Published Journal Article, the Retained Rights exclude Commercial Use, other than use by the author in a subsequent compilation of the author’s works or to extend the Article to book length form or re-use by the author of portions or excerpts in other works.
Published Journal Article: the author may share a link to the formal publication through the relevant DOI.
Author Representations
- The Article I have submitted to the journal for review is original, has been written by the stated author(s) and has not been published elsewhere.
- The Article was not submitted for review to another journal while under review by this journal and will not be submitted to any other journal.
- The Article contains no libelous or other unlawful statements and does not contain any materials that violate any personal or proprietary rights of any other person or entity.
- I have obtained written permission from copyright owners for any excerpts from copyrighted works that are included and have credited the sources in the Article.
- If the Article was prepared jointly with other authors, I have informed the co-author(s) of the terms of this Journal Publishing Agreement and that I am signing on their behalf as their agent, and I am authorized to do so.