Bot Check Items1

  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Hitches, E., Dudley, D., Johnstone, M., & Woodcock, S. (2024). Bots and baddies: Supporting the integrity of online survey research in the face of a growing challenge. Quality & Quantity. https://doi.org/10.1007/s11135-024-02001-w
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Loebenberg, G., Oldham, M., Brown, J., Dinu, L., Michie, S., et al (2023). Bot or not? Detecting and managing participant deception when conducting digital research remotely: Case study of a randomized controlled trial. Journal of Medical Internet Research, 25. doi:10.2196/46523
  • Rodriguez, C., & Oppenheimer, D. M. (2024). Creating a Bot-tleneck for malicious AI: Psychological methods for bot detection. Behavior Research Methods, 56(6), 6258-6275.
  • Simone, M., Cascalheira, C. J., & Pierce, B. G. (2024). A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research. American Psychologist, 79(7), 956-969. doi:10.1037/amp0001183
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472
  • Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman-Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. the Behavior Therapist, 42(7), 235-242.

Ask for the Same Information in Multiple Ways2

  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Levi, R., Ridberg, R., Akers, M., & Seligman, H. (2021). Survey fraud and the integrity of web-based survey research. American Journal of Health Promotion, 36(1), 18–20. https://doi.org/10.1177/08901171211037531
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176
  • Pratt-Chapman, M., Moses, J., & Arem, H. (2021). Strategies for the identification and prevention of survey fraud: Data analysis of a web-based survey. JMIR Cancer, 7(3), e30730. https://doi.org/10.2196/30730
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.

CAPTCHA3

  • Brainard, J., Killett, A., Houghton, J., Bunn, D., Watts, L., Mumford, S., O’Brien, S.J., Lane, K. (2022). The wasps are clever: keeping out and finding bot answers in internet surveys used for health research. Preprints. org 2022030243. https://doi.org/10.20944/preprints202203.0243.v2
  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Hitches, E., Dudley, D., Johnstone, M., & Woodcock, S. (2024). Bots and baddies: Supporting the integrity of online survey research in the face of a growing challenge. Quality & Quantity. https://doi.org/10.1007/s11135-024-02001-w
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Levi, R., Ridberg, R., Akers, M., & Seligman, H. (2021). Survey fraud and the integrity of web-based survey research. American Journal of Health Promotion, 36(1), 18–20. https://doi.org/10.1177/08901171211037531
  • Loebenberg, G., Oldham, M., Brown, J., Dinu, L., Michie, S., et al (2023). Bot or not? Detecting and managing participant deception when conducting digital research remotely: Case study of a randomized controlled trial. Journal of Medical Internet Research, 25. doi:10.2196/46523
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176
  • Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gundersen, D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional questionnaire. Journal of Medical Internet Research, 22(10), E23021
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.
  • Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman-Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. the Behavior Therapist, 42(7), 235-242.

Honeypot questions4

  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Hitches, E., Dudley, D., Johnstone, M., & Woodcock, S. (2024). Bots and baddies: Supporting the integrity of online survey research in the face of a growing challenge. Quality & Quantity. https://doi.org/10.1007/s11135-024-02001-w
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gundersen, D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional questionnaire. Journal of Medical Internet Research, 22(10), E23021
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.

Present Text as a Picture5 

  • Hitches, E., Dudley, D., Johnstone, M., & Woodcock, S. (2024). Bots and baddies: Supporting the integrity of online survey research in the face of a growing challenge. Quality & Quantity. https://doi.org/10.1007/s11135-024-02001-w
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472

Open-Ended Questions6 

  • Brainard, J., Killett, A., Houghton, J., Bunn, D., Watts, L., Mumford, S., O’Brien, S.J., Lane, K. (2022). The wasps are clever: keeping out and finding bot answers in internet surveys used for health research. Preprints. org 2022030243. https://doi.org/10.20944/preprints202203.0243.v2
  • Pratt-Chapman, M., Moses, J., & Arem, H. (2021). Strategies for the identification and prevention of survey fraud: Data analysis of a web-based survey. JMIR Cancer, 7(3), e30730. https://doi.org/10.2196/30730
  • Simone, M., Cascalheira, C. J., & Pierce, B. G. (2024). A quasi-experimental study examining the efficacy of multimodal bot screening tools and recommendations to preserve data integrity in online psychological research. American Psychologist, 79(7), 956-969. doi:10.1037/amp0001183
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.
  • Yarrish, C., Groshon, L., Mitchell, J. D., Appelbaum, A., Klock, S., Winternitz, T., & Friedman-Wheeler, D. G. (2019). Finding the signal in the noise: Minimizing responses from bots and inattentive humans in online research. the Behavior Therapist, 42(7), 235-242.

Set Survey Participant Caps7 

  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Levi, R., Ridberg, R., Akers, M., & Seligman, H. (2021). Survey fraud and the integrity of web-based survey research. American Journal of Health Promotion, 36(1), 18–20. https://doi.org/10.1177/08901171211037531

If you are Compensating Participants, Consider the Best Incentive Structure8 

  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176

Collecting IP Addresses9 

  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Griffin, M., Martino, R. J., LoSchiao, C., Comer-Carruthers, C., Krause, K. D., Stults, C. B., & Halkitis, P. N. (2022). Ensuring survey research data integrity in the era of the internet. Quality & Quantity, 56, 2841-2852. https://doi.org/10.1007/s11135-021-01252-1
  • Hitches, E., Dudley, D., Johnstone, M., & Woodcock, S. (2024). Bots and baddies: Supporting the integrity of online survey research in the face of a growing challenge. Quality & Quantity. https://doi.org/10.1007/s11135-024-02001-w
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472

Avoid Recruiting via Social Media – (especially if you are financially compensating participants)10

  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Loebenberg, G., Oldham, M., Brown, J., Dinu, L., Michie, S., et al (2023). Bot or not? Detecting and managing participant deception when conducting digital research remotely: Case study of a randomized controlled trial. Journal of Medical Internet Research, 25. doi:10.2196/46523
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176
  • Storozuk, A., Ashley, M., Delage, V., & Maloney, E. A. (2020). Got bots? Practical recommendations to protect online survey data from bot attacks. The quantitative methods for psychology, 16(5), 472–481. https://doi.org/10.20982/TQMP.16.5.P472
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.

Multi-Step Screening Procedures11

  • Bybee, S., Cloyes, K., Baucom, B., Supiano, K., Mooney, K., & Ellington, L. (2022). Bots and nots: safeguarding online survey research with underrepresented and diverse populations. Psychology & Sexuality, 13(4), 901-911.
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Louis, R. J., & Thompson, L. M. (2024). Bots and fake participants: Ensuring valid and reliable data collection using online participant recruitment methods. International Journal of Social Research Methodology, 1464-5300. https://doi.org/10.1080/13645579.2024.2410176
  • Pozzar, R., Hammer, M. J., Underhill-Blazey, M., Wright, A. A., Tulsky, J. A., Hong, F., Gundersen, D. A., & Berry, D. L. (2020). Threats of bots and other bad actors to data quality following research participant recruitment through social media: Cross-sectional questionnaire. Journal of Medical Internet Research, 22(10), E23021
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.

How to Detect Bots

  • Brainard, J., Killett, A., Houghton, J., Bunn, D., Watts, L., Mumford, S., O’Brien, S.J., Lane, K. (2022). The wasps are clever: keeping out and finding bot answers in internet surveys used for health research. Preprints. org 2022030243. https://doi.org/10.20944/preprints202203.0243.v2
  • Goodrich, B., Fenton, M., Penn, J., Bovay, J., & Mountain, T. (2022). Battling bots: Experiences and strategies to mitigate fraudulent responses in online surveys. Applied Economic Perspective Policy, 45, 762-784. doi:10.1002/aepp.13353
  • Guy, A. A., Murphy, M. J., Zelaya, D. G., Kahler, C. W., & Sun, S. (2024). Data integrity in an online world: Demonstration of multimodal bot screening tools and considerations for preserving data integrity in two online social and behavioral research studies with marginalized populations. Psychological Methods. https://doi.org/10..1037/met0000696
  • Karumathil, A., & Tripathi, R. (2025). Combating Survey Bots in Online Research: An Integrative Literature Review of Insights and Strategies. AIS Transactions on Human-Computer Interaction, 17(1), 80-109. https://doi.org/10.17705/1thci.00218
  • Shaw, T. J., Cascalheira, C. J., Helminen, E. C., Brisbin, C. D., Jackson, S. D., Simone, M., Sullivan, T. P., Batchelder, A. W., & Scheer, J. R. (2025). Yes stormtrooper, these are the droids you are looking for: Identifying and preliminarily evaluating bot and fraud detection strategies in online psychological research. Psychological Methods. Advance online publication. https://doi.org/10.1037/met0000724
  • Xu, Y., Pace, S., Kim, J., Iachini, A., King, L. B., Harrison, T., … & Simone, M. (2022). Threats to online surveys: Recognizing, detecting, and preventing survey bots. Social Work Research, 46(4), 343-350.

Bot Detecting Algorithms

  • Falk, C. F., Huang, A., & Ilagan, M. J. (2025). Unsupervised randomly responding survey bot detection: In search of high classification accuracy. Psychological Methods, https://doi.org/10.1037/met0000746
  • Rauchfleisch, A., & Kaiser, J. (2020). The false positive problem of automatic bot detection in social science research. Plos One, 15(10), e0241045. https://doi.org/10.1371/journal.pone.0241045
  • Roman, Z. J., Brandt, H., & Miller, J. M. (2022). Automated bot detection using Bayesian latent class models in online surveys. Frontiers in Psychology, 13, doi: 10.3389/fpsyg.2022.789223.

Other Relevant Readings

  • Horan, K. A., Shoss, M. K., Simone, M., & DiStaso, M. J. (2023). Preventing and mitigating the influence of bots in survey research. In The SAGE Handbook of Survey Development and Application (pp. 306-316). SAGE Publications Ltd.
  • Montanari, S. (2023). Scammers threaten quality of research data. The Transmitter. Retrieved from https://www.thetransmitter.org/ethics/scammers-threaten-quality-of-research-survey-data/ https://doi.org/10.53053/JKHR6038
  • Simone, M. (2019). Bots started sabotaging my online research. I fought back – STAT. STAT News. https://www.statnews.com/2019/11/21/bots-started-sabotaging-my-online-research-ifought-back/