Survey methods
Contents
Excerpt from Judith G. Kelley, Scorecard Diplomacy: Grading State to Influence Their Reputation and Behavior, (Cambridge University Press, 2017): 269–72.
From 2012 to 2014, with the help of research assistants, I assembled a database of over 1,000 NGOs working on TIP issues around the world. During the summer and fall of 2014 over 500 NGOs working in 133 countries responded to a survey designed to understand their engagement with the US and the TIP Report, as well as their assessments of the role of the US in their countries and their own governments’ performance.1
Survey Methodology
With the help of Andrew Heiss, then a doctoral student at Duke, I used Qualtrics to administer the survey through the Internet. Online surveys pose several methodological challenges. First, they introduce a technology bias. Small grassroots organizations without an online presence are missing from the database and those without easy Internet access or poor English faced barriers to participate. Finally, all surveys have response bias—willingness to participate is rarely random.
We took several steps to address these problems, based on the methods and recommendations of others.2 To encourage participation, we sent each NGO a set of three personalized email invitations, re-sent approximately every two weeks. Organizations without a working email address were contacted by phone. Each invitation included a link to the survey and an offer to complete the survey via phone, and respondents were allowed to remain anonymous. We translated the survey into Spanish and Russian and encouraged respondents to answer all free-response questions in their native language. We provided additional reminders and assistance to respondents who began the survey but did not complete it and sent links to allow organizations to resume their response. To minimize frustration that might lead respondents to quit prematurely, they were free to skip any question and could move back and forth in the survey. Additional efforts were made to reach non-responding NGOs by phone if we had very low participation from their countries.
Participation Rates and Demographics
We administered the survey to 1,103 NGOs and received responses from 480 unique organizations, yielding a participation rate of 43.5 percent. Because NGOs often work in multiple countries, we allowed respondents to answer a series of country-specific questions for up to five different countries, resulting in 561 country-organization responses. Most organizations (415, or 86.5 percent) chose to fill out the survey for just one country. Figure A1.1a shows the location of the NGO respondents’ headquarters and Figure A1.1b shows their work location.
Figure A1.1a. Country location of NGO survey respondent headquarters. Number of NGOs: 469. Number of countries: 106.
Figure A1.1b. Country location of NGO survey respondent work. Number of countries: 125.
The NGOs surveyed have a nearly global reach. The majority of organizations (60 percent) are based in either Asia or Europe, roughly a quarter are based in North or South America, and fewer than 20 percent work in Africa.
On average, anti-TIP NGOs spend a little over half of their time and resources focused specifically on fighting trafficking and assisting victims and an overwhelming majority (93 percent) has at least some knowledge about the TIP policies in the countries they work in. Most organizations focus on sex (85 percent) and labor (61 percent) trafficking issues; 50 percent focus on both simultaneously. A handful of organizations (30, or 6 percent) work with human organ trafficking, and dozens of others deal with other issues such as brokered marriages, domestic servitude, illegal adoptions, and forced begging. Approximately two-thirds of NGOs serve and advocate for children and/or adult trafficking victims, and many of those who work with adults specified working especially with women and young girls.
Most organizations (83 percent) advocate for prevention and improved education about TIP issues, and nearly three-fourths assist trafficking victims by running safe houses and shelters, operating special hotlines, helping start businesses, or providing physical and emotional health care.