Simple Survey Idea #1: Keep survey language simple

I am working on a web survey for a group of people in India.  Smart folks, many of them technology savvy.  And they speak English — but often not as their first language.

Some surveys should be translated or fielded in multiple languages.  For many surveys, though, English is sufficient.  But what kind of English?

My default mode is to use more complicated English than is needed.  The more I work with multilingual people around the world, the more I realize the value of keeping language simple, especially with surveys and interview questions.

The good news is that there are tools out there that can help.  Here is a site that lets you paste in text and compare it to one of many collections of simple English words.  It shows which words are not considered simple.

http://www.online-utility.org/english/simple_basic_helper.jsp 

With many international audiences it is a good idea to test your language before sending out a survey.

I put the above portion of this post into the site and found out that the following words were not included in a somewhat large collection of 15,000 simple words: web, savvy, and multilingual.  With smaller collections, many more words miss the cut, including realizecompare and survey.

Try it out — you have nothing to lose but complexity.

It’s Winter: Time to Make Snowballs!

A lot of mission researchers are interested in studying people who aren’t easy to get to.  They may be unknown in number, difficult to access, suspicious of outsiders, etc.

This makes random sampling virtually impossible.  Unfortunately, a random sample is an assumption or requirement of many statistical tests.

So, if you’re doing research with underground believers or people exploited in human trafficking, you can’t just go to SSI and rent a sample of 1500 people to call or email.

When you need a sample from a hard-to-reach population, make a snowball!

Snowball sampling, a more memorable name for the formal term, respondent-driven sampling, is a means of getting to a reasonably large sample through referrals.  You find some people who meet your criteria and who trust you enough to answer your questions, then ask them if there are other people like them that they could introduce you to.

In each interview, you ask for referrals – and pretty soon the snowball effect kicks in and you have a large sample.

For years this approach was avoided by “serious” researchers because, well, the sample it produces just isn’t random.  Your friends are probably more like you than the average person, so talking to you and your friends isn’t a great way to get a handle on your community.

But, like six-degrees of separation, the further you go from your original “seeds,” the broader the perspective.  And in recent years, formulas have been developed that virtually remove the bias inherent in snowball samples – opening up this method to “respectable” researchers.

How to do it?  Some researchers simply throw out the first two or three generations of data, then keep everything else, relying on three degrees of separation.  Not a bad rule of thumb.

For more serious researchers, there is free software available to help you weight the data and prevent you from having to discard the input of the nice people who got your snowball started.  Douglas Heckathorn is a Cornell professor who developed the algorithm (while doing research among drug users to help combat the spread of HIV) and helped bring snowball sampling back from the hinterlands of researcher scorn.  You can read more about his method here and download the software here.

Suddenly, you need not settle for a handful of isolated snowflakes, nor for a skewed snowdrift of opinion (via an unscientific poll of your social media friends).  Instead, you can craft their referrals into a statistically representative snowman.

Meanwhile, if the sample you need is one of North American field missionaries or North Americans seriously considering long-term cross-cultural service, you should consider renting one of GMI’s mission research panels.  Email us for details.