Tag Archives: open-ended questions

To retain missionaries, help them keep their CHIN UP

Retention of field staff is a key effectiveness issue for mission sending entities. In The Future of the Global Church (p227) Patrick Johnstone notes that “the period most likely to prove fruitful in cross-cultural church-planting ministry” is between the eighth and 17th years of field service.

Retention is also a key stewardship issue, as the costs of recruiting, qualifying, training, funding and sending a cross-cultural worker are vastly front loaded (incurred before sending and in the early stages of field ministry).  This is true regardless of the worker’s country of origin.  From a financial perspective, sending costs are “amortized” over a worker’s tenure on the field—the longer the tenure, the more cost-effective the sending process.

While it is true that when missionaries need to leave the field, allowing them to stay on can be damaging (personally, organizationally and ministerially), the general principle remains: encouraging and equipping workers toward longer tenures is a worthy goal.

A few weeks ago I spoke to prospective cross-cultural workers on “How to Become an Ex-Missionary…Or Not.” The research supporting the talk came from the qualitative module of the Engage study (fielded 2006 and 2007), which GMI did in partnership with Best Christian Workplaces Institute and Rob Hay, now principal of Redcliffe College.

A bit of backstory before getting to five key retention factors for North American cross-cultural workers.

The Engage research was initiated by the WEA Mission Commission as a North American follow up to the global ReMAP and ReMAP II studies on attrition and retention, which led to the publication of the useful books Too Valuable to Lose and Worth Keeping.

The ReMAP studies did a great job of outlining best practices in missionary retention on a global scale, as well as in drawing attention to the issue.  For example, the 60-plus agencies participating in the U.S. portion of the studies could be divided into virtually equal groups based on retention.  High-retention agencies averaged 97.4 percent retention annually, while the low-retention group averaged 90.4 percent.  When compounded over a decade, the high-retention rate projects to 77 percent of non-retired workers remaining on the field, while the low-retention rate projects to only 37 percent remaining.  You can read the source report here.

Still, there was one significant problem with the ReMAP studies: no current field workers were interviewed, only agency administrators.  We first got involved when the ReMAP II U.S. study coordinator asked GMI to review the U.S. data.  Certain variables appeared to yield counter-intuitive results—meaning agencies that rated themselves as high performers in certain dimensions actually had lower retention than those that rated themselves as lower performers.  How could this be?  Easy—self-evaluation often produces results such as these due to differences in standards.  Highly effective organizations typically have very high standards.  Therefore, they see more room for improvement than do their peer organizations.

The solution: don’t self-evaluate.  Allow others—in this case, current and former field missionaries—to rate how well a sending organization equips and supports them.

That’s what we did in the Engage study.  The quantitative module surveyed more than 1,700 current field staff from 17 organizations.  Results verified that an organization’s retention rate correlates positively with the attitudes of current field staff.  That finding refutes the hypothesis that ex-missionaries are merely can’t-hack-it, sour-grapes misfits who needed to be weeded out.  Rather, an agency’s missionaries reside along a likelihood-to-stay continuum.  The better an agency’s vision, leadership, training, policies and support, the less likely workers are to fall off of the attrition cliff.

The qualitative module compared the experience of more than 40 current field missionaries and more than 40 ex-missionaries who had left before retiring or completing a fixed assignment.  Questions asked of both groups included open-ended inquiries about factors that encourage or discouraged continuing service.

Group comparisons of coded responses yielded five key encouragement factors (as well as six key discouragement factors).

I remember those five encouragement factors through the acronym CHIN UP:

CH      A strong sense of personal CALLING and HOPE from God

I          A feeling of INCLUSION in team/agency decision making

N        A perception of great spiritual NEED among the people being served

U        A sense of personal USEFULNESS, regardless of visible ministry progress

P        A strong sense of God’s PROVISION via the prayer and generosity of others

Agencies and their member care departments will do well to regularly check the pulse of their field staff in those key areas.

Want to know more about applying insights from Engage?  Contact us at GMI about speaking to or consulting with agency leadership and/or member-care staff on retention issues.  We also love to speak with future missionaries about how they can prepare to avoid becoming ex-missionaries.

Also, agencies can sign up to do the Engage survey with your current field staff.  Best Christian Workplaces offers the survey at a very reasonable cost, using a sliding scale based on the number of field workers invited to take part.  It is a great investment that also adds to an ongoing database of learning about retention.  Please let them know that you heard about Engage from GMI.

 

 

From their keyboard to your web designer

(Second in a series of three posts)

As we introduce the 2013 Agency Web Review, we are reviewing highlights from the 2004 edition.

It is great for an agency to get numeric ratings on various aspects of its website—especially when comparable ratings for a group of other agency websites are available to show relative strengths and weaknesses.

But the ratings come to life when designers and mobilizers learn the specific reasons beneath the great ratings—or when they get suggestions to improve elements of their site.

Back in 2004, when we first gathered opinions of dozens of agency sites, social media was still in its infancy and dialogue with prospects was much less prevalent—so getting actionable feedback from the target audience was more difficult.

Today future missionaries have more channels to offer ideas—the challenge is getting them to take time to look closely and consider how their input can help.

Next month, the Agency Web Review will provide incentives to those considering cross-cultural service to spend several minutes observing agency websites—and then offering their opinions about them.  In addition to numeric ratings, respondents will provide open-ended feedback on their initial impressions, the website’s strengths, and elements that can be improved.

Here are some of the verbatim quotes from future missionaries.  Would any of these apply to your site?

It was attractive, but it wasn’t obvious what they did.

 

It was a little cluttered with things and I really didn’t know what I should click on. But it was VERY informative.

 

Really liked the home page and the different pictures that come up when you point at the options to enter.

 

Opportunities— list was hugely long—I had no idea which ones to select.  Perhaps narrow the categories a bit for a first selection.

 

They recommended books which would be helpful for someone looking for more resources and help in a particular area.

 

I liked the opportunity boxes that popped up over the world and then I could click on them.

 

Include more stories about past missionaries’ experiences.

 

I love the emphasis on prayer. I think that it probably points to a good orientation.

 

Things weren’t hidden in fine print from what I could tell in my brief study. Everything seemed very up-front.

 

Give more detail about service opportunities without requiring a person to receive mailings.

 

I could not find a belief statement.

 

It is impressive they have the site in a different language.

 

Increase the size of the font, I can barely read it.

 

Visually a bit sparse, but seemed spiritually grounded.

 

There is a lot going on, visually creative. I wanted to learn more.

 

Seemed to be more focused on their organization than on the people they were serving, kind of a turn off.

 

Mission and values were clearly stated and inspiring.

 

The global map links on the home page didn’t all work.  I couldn’t find a webmaster link to report it.

 

How do I actually work for (agency)!? I have no idea how to apply or what to do if I want to work with them!

 

Present more ‘in your face’ opportunities to serve or donate.

 

I really liked that one of the first things I saw invited me to pray with the group.  It gave me a way to get involved right now.

At the agency level, this kind of feedback is especially valuable for designers.  Collectively, comments about dozens of sites can be coded and analyzed for helpful trends.  Comments in the 2004 study most frequently related to the following:

  • Ease of Navigation
  • Quantity of Information
  • Organization of Site
  • Design / Layout / Color
  • Graphics / Photos

The most-frequently-mentioned opportunities for improvement fell into these categories:

  • Information about Opportunities
  • Visual Style / Layout
  • Information About the Agency
  • Text / Font
  • Verbal Style

What kinds of ideas has your agency implemented on its website based on the suggestions of visitors?  Add a comment and let us know!

 

Analyzing open-ended questions: A bright future for word clouds

Commercial survey research firms usually charge clients significantly extra to include “open-ended” questions in a survey.  They tend to be messy and time-consuming.  Traditionally, analysts would read through a selection of responses to create categories of frequent or typical responses, then read back through all of the responses to categorize them.

For publishing in a peer-reviewed journal, multiple people would categorize responses independently, then work together to create a synthesized coding scheme to limit bias.

Most qualitative text-analysis software still requires you to manually “code” responses.

With all that work, open-ended questions are still important in exploratory and qualitative research – and frequently satisfying for survey respondents looking for an opportunity to say what is on their mind, unhindered by structured response categories.

But the tag-cloud age has been a blessing to those without the time and money to do full, traditional analysis of text responses.   Graphics with words sized by frequency of use enables analysts to quickly get a sense of the nature of open-ended responses.

New editions of survey software – even budget packages like Survey Monkey – include cloud-creating tools to help users understand open-ended responses at a glance, without all the coding work.

Even those doing traditional coding enjoy working with clouds, which help analysts to quickly create an initial set of codes.

If your survey package doesn’t have cloud-generating capacity, no problem.  Worldle is a free site that lets you create art-like word clouds.  The clouds in the previous post were created using Worldle.  It’s a terrific, easy-to-use site that lets you paste in your text – our data came straight from a spreadsheet – and generate a word cloud with one click.  It automatically removes common words, allows you to choose the relative cloud shape, color scheme, font and orientation of the words.  We chose to illustrate the top 100 terms for each question.  Wordle lets you save and use your clouds however you want to.

I really like the tool’s artistic quality.  Wordle clouds almost beg to be shown to others.  Then they become motivated, too.  My daughter, upon first seeing Wordle, immediately had a vision about making a sign to promote a bake sale.  A few descriptive terms later, she had created a beautiful graphic to draw people’s attention.

This is where research moves from information to influence.  Imagine asking your constituents about their needs – or your organization’s impact – then printing a graphic of their responses to hang in your office as a reminder and motivator to staff.  Unlike a research report, which may or may not get read before being filed away (or worse!), word cloud art can keep research right in front of your team.  The graphic format makes the information more memorable as well.

Researchers, meanwhile, can compare and contrast different audience segments, as I did in the word cloud below.

What applications can you think of for word clouds?