There is no perfect way to prevent a bot/scammer attack, so it is best to use multiple strategies. In addition, prevention methods are quickly changing along with technological advances and more sophisticated scamming techniques. Our IRB best described it as a “cat-and-mouse” game between researchers and scammers. Below you will see a list of common prevention methods, along with references (listed in “Further Readings”) to helpful information about implementation. 

Bot Check Items1

  • These items require participants to engage in tasks that bots are not able to do, such as perspective taking (e.g., what would the person in the picture see), completing tasks (e.g., basic math), or image processing and reasoning (e.g., “you see this picture, what would happen if…”).  

Ask for the Same Information in Multiple Ways2

  • This method can be used to check answer consistency. For instance, someone’s zip code should match their reported general geographic area. Inconsistent responses may indicate fraudulent data. 

CAPTCHA3

  • CAPTCHAs (Completely Automated Public Turing test to tell Computers and Humans Apart) are simple tools used to detect bots and are often available on survey creation platforms. Although CAPTCHAs can be helpful, many scammers have found methods around these tools. 
  • Instructions for enabling CAPTCHAs on Qualtrics surveys can be found here. 

Honeypot questions4

  • A honeypot question is a hidden question that is (typically) not visible to a real participant, but may be completed by a bot. 
  • Honeypot questions can be implemented in different ways depending on the survey creation platform. Some methods for creating honeypot questions on Qualtrics can be found here. 

Present Text as a Picture5 

  • Including a questions’ text as an image may prevent bots from properly answering that question. Screen readers cannot read the image content, making it harder for bots to complete survey questions.  

Open-Ended Questions6 

  • Open-ended questions are an effective way to assess response quality. Responses that do not make sense, or that show evidence of generative AI (e.g., overly polished) may be evidence of fraudulent responses. 

Set Survey Participant Caps7 

  • To avoid our nightmare of having to look through over 3000 responses, limit how many people can complete your survey. Going through 200 responses is MUCH easier than 3000 responses. Most survey creation platforms allow you to set a limit on the number of participant responses. Similarly, most survey programs allow you to prevent participants from completing the survey twice. There are ways around this (and it didn’t prevent fraudulent data in our situation), but it is an easy way to add protection against ballot stuffing by less experienced scammers. 
  •  Instructions for how to prevent multiple submissions on Qualtrics can be found here. 

If you are Compensating Participants, Consider the Best Incentive Structure8 

  • Financially compensating participants substantially increases your risk of a bot attack. Research compensating all participants may be at greater risk than research raffling gift cards. You definitely want to consider multiple strategies to prevent bots if you are compensating participants.

Collecting IP Addresses9 

  • If you are not collecting sensitive information, your IRB may allow you to collect IP addresses. IP addresses are a form of personally identifiable information (similar to a physical addresses) which should typically be unique for each participant. If you receive several responses from the same IP address, you should closely examine their data for potential fraud. You can also use geoip look-ups to determine general IP address location.

Avoid Recruiting via Social Media – (especially if you are financially compensating participants)10

  • Scammers search for terms such as “paid” and “research” on social media. Based on our data, we believe scammers found our survey within an hour after posting on Facebook.

Multi-Step Screening Procedures11

  • Screening procedures prior to sending potential participants survey links can reduce the likelihood of bots.

Check if your IRB has Recommendations for Preventing Bots 

  • Due to the increasing number of bot attacks, many IRBs are developing recommendations for their researchers. For example, Ball State University’s IRB recommendations can be viewed here.