Tag Archives: survey

Getting to know you: Future missionaries surf for agencies

 

This is the first in a three-post series.

Every day, people considering long-term cross-cultural service visit sending agency websites and social media pages.  What they experience in a few quick clicks can inspire them to bookmark a site, tweet about it to friends or complete an inquiry form.  Or it can lead them to a quick exit.

With mobilization events like Urbana and MissionsFest taking place in the next few weeks, agencies should be ready to put their best foot forward in assisting field-bound people to discover the next step in their journey.

Next month, GMI will field the 2013 Agency Web Review, in which hundreds of people considering long-term cross-cultural service will explore dozens of sending agency web sites, evaluating various website elements and providing helpful open-ended comments. Participating agencies will receive agency-specific reports with detailed feedback on their site, plus comparative data showing how the site’s ratings compare to a group of several dozen other agencies.

The study draws on GMI’s opt-in panel of more than 3,000 people who have confirmed that they are considering a career in cross-cultural mission. 

Eight years ago GMI fielded the first edition of the Agency Web Review, which provided actionable results for agency web designers and mobilization staff.  Kristi Crisp of World Gospel Mission had this to say about the study:

The Agency Web Review results helped us to set a better direction and convinced us of the need for changes.  …The World Gospel Mission website is a completely different site now.  We changed our focus to getting people actually going…whether with our organization or with somebody else.

In anticipation of the 2013 study, we are taking time to review a few of the highlights of the 2004 study.  The electronic communications landscape has changed dramatically since then, but many of the findings from 2004 continue to be useful. 

We asked which of 11 key activities people considering a missions career had done.  Visiting agency websites ranked fourth on the list (after attending conferences, reading mission books/newsletters and talking with missionaries).  Six out of 10 prospects had already visited the website of a sending agency.

The following chart reveals stated priorities for missionary prospects when visiting an agency site:

 

To us, the results suggest that the primary questions visitors of website visitors relate to identity: Who are you?  What do you do?  Where do you do it? What do you stand for?

Once those questions are answered, prospects feel free to consider, “OK, how would I fit in?” “What would it take for me to be a part?”

Stated priorities don’t reveal the degree to which elements of a website are linked to key response actions (more info in a few days on that), but they do express visitor expectations.  Therefore, we recommend that web designers make sure expectations are easily met without a lot of searching.  That means a well-placed “About Us” heading, opportunities that are dated and kept updated, and some explanation about what people can expect to happen after the inquiry form is submitted.  (That last item was the lowest-rated of 21 site elements tested across all organizations.)

In addition, we suggest providing some unexpected elements of “delight.”  A few of the unexpected pleasures encountered by site visitors include:

  • an opportunity to be prayed for by agency staff
  • engaging videos from field staff that give people a taste of daily life on the field
  • links to helpful resources for people considering service—even from other agencies

We noticed that the highest-rated agency websites tended not to minimize their service requirements, but they worked hard not to represent those requirements as barriers.  Their positioning was something like this:

“Becoming a missionary takes real commitment, knowledge and skills.  It’s not easy, but it is do-able, and we will walk alongside you to help you develop into an effective cross-cultural servant who enables others to realize all that God is calling them to.”

Do you know of an agency that incorporated user feedback into its website makeover? We’d love to hear about examples or standout experiences you’ve encountered.

Learn details about how to take part in the 2013 Agency Web Review here.

 

Simple Survey Idea #3: Give Something Away

When you do a survey, you are asking people for their time and their opinions.  People are increasingly aware of the value of both.

With that in mind, it is a good practice – even among those who already know and trust you – to give something away in appreciation for their input.  Doing so will bless people and build goodwill.  It will also improve your response rate (and therefore the quality of your data).  And it will make them more inclined to participate in future surveys.

“But we don’t have the budget to give anything away,” I sometimes hear people say.  I say, “If you can’t find something to give away, you’re not trying very hard.”  You don’t need budget – there are lots of ways to give survey responders something for free.

The first thing you should give them is a short survey.  That may be a topic for another Simple Survey Ideas post, but it’s so important that it always warrants mentioning.

You can give people access to the survey results, a good idea if your responders are peers/stakeholders and you know they will be interested in what you are learning.  Depending on your survey software, it may cost you some time to format and email results out to those who responded.  But if you use an online package like Survey Monkey, you can set up options to automatically show the survey results to date upon completion of the survey.  That “Instant Results” feature is even available on the free Survey Monkey package.

Of course, the first few people who respond won’t get a very complete picture, so you might also want to send people a link to the full set of responses once you complete and close the survey.  This option is available in all of Survey Monkey’s paid subscription plans.

Quick aside: the advantages that you get with the online services’ paid plans (unlimited responses, survey logic, ability to download data, HTTPS security) make them well worth the cost (vs. the free plan) for almost any survey.  Even if you are just doing a one-off survey, you should still sign up for a month and then cancel the subscription when you’re done.  Your survey is worth the $24 investment.

Another useful offering is free-information-of-interest-to-respondents.  I use this with virtually every survey I do.  You can almost always find an article or ebook or presentation or video related to your survey topic or to a common interest of the survey audience.  Even if you don’t produce content, you can always find something free on the Internet to direct people to.

In this way, you can say in your survey invitation, “Everyone who completes the survey will receive a free ‘Top 10’ list of resources about _____.”  It doesn’t matter that the list is out there on the Internet for anyone to find – linking people to it is delivering value.  With Survey Monkey, the option to redirect survey finishers to the website of your choice only comes with annual plans.  So, you may need the workaround of embedding your own link on the last page of the survey, so responders can get to your resource.  At the risk of going beyond “simple,” try something like this:

Thanks for completing our survey.  Before clicking “Done,” click <a href=”http://www.yoursite.org/” target=”_blank”>this link </a> to open a new window with the free resource we promised.

Be careful that the resource will be of interest to nearly everyone that you invite.  Giveaways that appeal only to a certain segment of your audience will lead to response bias.

Should you ask permission of the content provider in advance?  It’s a good idea but not required – groups that offer free content on the web typically want people to find that content.  You benefit them by linking to their site.  Groups that provide many free mission-related resources include the World Evangelical Alliance and the U.S. Center for World Mission.

A quick-response incentive promises resources to the first X number of responders.  This can be a good idea if you have a limited number of tangible resources to give away – and especially if you need responses quickly.

A related incentive is the sweepstakes prize offer, where respondents are randomly selected to receive a prize – usually something with significant value.  Many researchers use a combination of a free something-for-everyone resource with a high-value sweepstakes prize for a few randomly selected winners.

I like sweepstakes offers – they are fun and they work to generate response.  But you have to be responsible with them – some laws apply (see a quick overview here and know that this post does not constitute legal advice).  If you go this route, make sure that everyone who responds has an equal chance to win (even those who don’t meet the criteria for responding to your survey – nothing ruins a good survey like people lying to qualify for a prize), clearly communicate what and how many prizes you are giving away, eligibility, how and how often you can enter, when the giveaway will take place, how winners will be notified, approximate likelihood of winning, and any geographic or residency limitations.

That sounds like a lot, but consider that the following covers all of that without sounding too much like the legal disclaimer lingo in car dealer’s radio ad:

“You and up to 400 others who complete the survey by March 31 will qualify for a random prize drawing for one of 10 copies of the Operation World DVD.  One entry allowed per survey link.  In April GMI will inform the 10 winners by email – they will need a valid U.S. mailing address to receive their DVD.  Not valid where prohibited by law.”

How to manage a random drawing without hundreds of slips of paper and a huge hat?  Discover the RAND function in Excel.  Very handy – be sure to sort, save and print results for your records.

Also make sure to give away everything you promise.  If some people don’t claim their prize by a given date, move on to the next people on your randomized list.

Prize giveaways are appealing to most, but it is not unusual for those in ministry circles to steer clear of them because of their similarity to gambling games of chance.  Before launching a contest, be sure your organization’s leadership knows about it.  If you run into concerns, one alternative is to allow or encourage winners to donate their prize to charity.

Some survey sponsors use a charitable donation as the incentive itself, which carries real appeal for respondents.  One commercial firm I worked with leads off its surveys with a question like this:

In appreciation for your opinion, our firm will be distributing charitable donations totaling $1000.  From the following list, please select the charitable organization that you would like your portion of the donation to go toward:

__ Organization A

__ Organization B

etc.

If your group is a charitable organization, you can use a list of projects instead.  This works well if you can (truthfully) mention that an individual donor has put up the gift money to be distributed in this manner.

A final tip that applies to any gift or incentive that you offer: don’t position it as the primary reason to respond – especially in the subject line of an invitation email.  Not only do words like “prize” and “win” tend to trigger spam filters, but leading with the gift offer sends a message to invitees that you view the exercise as a transaction (or worse, that you think they are primarily motivated by greed).

Instead, keep the focus on the importance of the survey topic and the value of the person’s opinion – then mention the gift or prize.  As a survey sponsor, your identity should be that of a listener asking people for the favor of their input and offering them the opportunity for involvement – plus a gift as a token of your appreciation – rather than as a purchaser of people’s opinions.

 

Simple Survey Idea #2: Send a Reminder

I talk with lots of people who design and field their own web surveys.  It amazes me how many have never considered sending a reminder out to those they have invited — even to people who are known well by the person doing the survey.

People are often very willing to help, but they are busy and working through lots of messages, and survey invitations are easy to set aside until later.  One reminder is often helpful.  I almost always send at least one reminder out to survey invitees.  In some cases, I will send out a second reminder.  In rare cases, a third.

Why send a reminder at all?  Perhaps it goes without saying, but more data usually equals better-quality information.  Better statistical accuracy is part of that: most people understand that a sample of 300 yields a tighter margin of error than a sample of 100.

But in most cases, response bias will be a bigger threat to the quality of your data than statistical error from sample size.  Consider your sample of 300 responses.  Did you generate those from 400 invitations (a 75% response rate) or 4,000 invitations (a 7.5% response rate)?  The former would give you much greater confidence that those you heard from accurately reflect the larger group that you invited.

What is a “good” response rate?  It can vary widely depending on your relationship to the people invited (as well as how interesting and long the survey is, but that’s a topic for another post).  Domestic staff/employee surveys often generate a response of 85 percent or more.  However, for internationally distributed missionary staff, a response of 60 percent is healthy.  For audiences with an established interest in your work (event attenders, network members), a 35-percent response is decent.  For other audiences, expect something lower.  One online survey supplier’s analysis of nearly 200 surveys indicated a median respose rate of 26 percent.

So, do reminders substantially increase response to surveys?  Absolutely.  Online survey provider Vovici blogs, “Following up survey invitations with reminders is the most dramatic way to improve your response rate.”  They show results from one survey where the response rate rose from 14 percent to 23, 28 and 33 percent after subsequent reminders.

My experience has been similar.  I find that survey invitations and reminders have something like a “half-life” effect.  If your initial invitation generates X responses, you can expect a first reminder to produce an additional .50X responses, a second reminder .25X responses, and so on.

I disagree with survey provider Zoomerang’s suggestion of sending a series of three reminders — especially if the audience is people you know — but I do agree with their statement, “Think of your first email reminder as a favor, not an annoyance.”  I recommend sending at least one reminder for virtually any survey, with a second reminder only if you feel that your response rate is troublesome and you need that extra .25X of input.

At least Zoomerang provides a sample reminder template you can use.  I agree that you should keep reminders short — shorter than the original invitation.  With any invitation or reminder, you will do well to keep the survey link “above the fold” (to use a phrase from old-time print journalism), meaning that it should be visible to readers without their having to scroll down through your message.

I also find that it very helpful to use list managers in sending survey reminders.  Most online providers will have an option where you can send only to those members of your invitation list who haven’t responded.  Not only does this keep from annoying those who already did respond, but you can word the reminder much more directly (and personally, with customized name fields).  So, instead of saying:

“Dear friend — If you haven’t already responded to our survey, please do so today.”

You can say:

“Dear Zach — I notice that you haven’t responded to our survey yet.  No problem, I’m sure you’re busy.  But it would be great to get your input today.  Here’s the link.”

Take care in using the above approach — if you have promised anonymity (not just confidentiality), as in an employee survey, opt for the generic reminder.

When to send a reminder?  If your schedule is not pressing, send a reminder out 5-10 days after the previous contact.  I recommend varying the time of day and week in order to connect with different kinds of people.  If I sent the initial invitation on a Monday morning, I might send the reminder the following Wednesday afternoon.

 

New research: Field staff estimate that 1 in 3 interns convert to long-term missions

In 2004 I attended The Mission Exchange/CrossGlobal Link (then EFMA/IFMA) Personnel Conference.  One of the breakout sessions dealt with AIM’s TIMO program, one of the longest-running mission internship programs.

There was a bombshell moment when the presenter cited statistics about how many of the interns were still on the field. (I can’t recall exactly, but the numbers were staggering — something along the lines of 90 percent still serving, two-thirds of those with AIM.)  Mobilization directors’ jaws were dropping across the room, and you could sense that any agency there that didn’t have an internship program would soon be considering one.

Given that prospective cross-cultural workers today are less likely than those in previous generations to commit to a lifetime of service with a particular agency or among a particular people, and internships seem to make sense — an on-field experience that allows future workers to understand what it takes to live and work “out there,” and a low-risk opportunity for agencies to train and assess the fitness of candidates.

But creating an internship program requires time and money – plus substantial cooperation from long-term field staff, who will have to oversee the interns. Is it worth the effort?

In November and December 2011, we put this question — actually, several questions — to the GMI Research Panels — large groups of current and future cross-cultural field missionaries ready to give their opinion on mission-related issues.  We asked about perceptions of field internships (defined as a cross-cultural field experience lasting from six months to three years); satisfaction with internship programs, the likelihood of prospective missionaries to do an internship, and estimated conversion of interns to long-term field staff.

We heard from more than 300 cross-cultural missionaries (from more than 18 agencies) and from more than 300 people who are considering long-term cross-cultural service.  The following charts show some of the topline statistics:

A third of prospective missionaries said they are somewhat or very likely to do a mission internship. As you might expect, interest increases with one’s commitment to entering long-term cross-cultural service. More than half of those who intend to serve long term said that they are somewhat or very likely to do an internship in preparation.

Three quarters of the field missionaries surveyed had experience with interns. Of those, 6 in 10 agreed or strongly agreed that they are satisfied with their agency’s internship program, while acknowledging in open-ended comments that internships involve a lot of work for field staff.

They also estimate that 45 percent of interns stay active in full-time field service after their internship, with a third continuing to serve with the agency for which they interned.

How does these perceptions compare with your experience?  Are field internships worth the effort?

In the next post, we’ll explore current and future missionaries’ open-ended comments about mission internships.

Keep your eye on the GMI store for the detailed report, which will include more information about the key elements that lead to satisfaction with internships, as well as descriptions of three types of interns based on their motivation for service.