Tag Archives: from info to influence

Analysis links website “personality” to follow-up actions

 
Third in a series of three posts looking forward to the 2013 Agency Web Review by reviewing highlights from a prior edition of the study.

As in 2004, the upcoming edition of the Agency Web Review will not only consider the rational elements of a missions website (clarity, functionality, information) but also the emotional side—its “personality”—and how that aligns with visitors’ responses and potentially influences follow-up activity.

From a list of 40 or so descriptive terms, respondents select the ones that they felt best describe each website.  Once data collection is complete, GMI analysts use factor analysis to boil those characteristics down to a smaller set of themes, or personality factors.  Then, they use regression analysis to measure how those themes correlate with the follow-through outcomes that agencies are looking for: revisiting the site, recommending it to peers and pursuing service opportunities.

In 2004 four out of the 10 personality factors correlated strongly with desired actions.  Web designers for the participating agencies learned about these factors and were able to draw on them in adapting their sites to encourage future missionaries to take the next step.  Those four factors are shown in this chart:

The degree of correlation varied depending on the follow-through action being studied.  For Intent to Recommend, the strongest link was to the “Creative” aspect of a site (a factor comprised of “Creative,” “Fresh,” “Visual” and “Non-Traditional”).  For overall website appeal, the same four factors emerged, but with “Energetic” (an absence of characteristics such as “Calm,” “Simple” and “Casual”) at the top of the list.  

How did designers use this information? Here are two ways: 

  1. Striking the right balance between identifying with who your audience is and who it aspires to be.  The presence of Energetic and Creative suggests that an agency should demonstrate that it understands the next generation of workers and is relevant to them.  This is the classic affinity principle—demonstrating that an agency is “for people like me.” Prospects are drawn to organizations that have and welcome high-energy people with a creative spirit.At the same time, the presence of Wise (“Humble,” “Experienced”) and Capable (including “Secure” and “Aware”)  suggests traits that future missionaries do not yet have—especially in terms of cross-cultural effectiveness.  Prospects desire to develop these qualities and hope that an agency will be able to draw them out.  Prospects look for signals that an agency knows what it is doing and can help new people get where they need to be.

    It is easy to miss the mark a bit one way or the other.  Too much creativity can be misinterpreted as an emphasis on style over substance.  Too much emphasis on experience and resources can be misread as close-mindedness or lack of need.  In short, prospects don’t want agencies that aspire to be “like them”—rather, they do want to be understood while being given something to aspire to become.

  2. Designers also need to understand what is not helpful.  In this case, six other website personality factors were not significantly linked to any of the key follow-through behaviors:  intention to revisit, refer or respondto opportunities.  Websites that were viewed as “Courageous,” “Concerned,” “Sharp” (incorporating “Confident” and “Smart”), “Thoughtful” or “Fun” likely did not generate the best possible response from visitors.(Note that positioning does not refer to the inauthentic donning of characteristics that are not part of an agency’s identity, but rather to expressing aspects of one’s true identity that are likely to resonate with candidates.)

Web designers usually know how to help make a website more usable or functional.  Some also have intuitive skills that enable agencies to strike the right notes in messaging and imagery. But for those who don’t—and even for those who do—message modeling can provide a well-defined target to shoot for and criteria to assess whether the messages are hitting their mark.  Such models are a standard feature of the Agency Web Review report.

Eight years after the first review, who knows how the personality characteristics may have changed?  New factors are likely to emerge in association with key outcomes.  To find out, be sure to register for the 2013 Agency Web Review.

 

Applied research helps donors, implementers to be better partners

Research provides a needed listening function for the mission community.  Listening well results in better understanding, and better understanding usually leads to better ministry.

A great example of the way that research increases understanding and leads to practical action in ministry is the Lausanne Standards project that fosters dialogue and collaboration among ministry implementers and funders about the giving and receiving of money in mission.

Check out this entertaining whiteboard video that illustrates (literally) how the Lausanne Standards were developed and the role that research played.

GMI is honored to have conducted the first round of research (mentioned in the presentation) that supported the development of the Lausanne Standards.   Rob Martin, Lausanne Senior Associate for Global Philanthropy, whose voice (and likeness) feature prominently in the video, graciously gave us permission to discuss some of that research here on this blog.

A survey of 147 mission leaders – divided roughly 55/45 between ministry implementers and ministry donors – revealed that both groups agreed that positive funding partnerships are almost always an important issue.  However, the leaders were divided on whether partnerships were problematic, and what the nature of the problems (if any) and solutions were.

Cluster analysis led to the identification and description of four “attitude segments” among ministry donors and implementers.  This enabled the research sponsors to understand the likely objections to developing a set of guidelines for philanthropic partnerships.

 

Each of these groups believes that funders and implementers want to partner well with one another.  However, each could pose a significant objection to the process of developing standards for effective funding partnerships.  Proceeding clockwise around the grid, from top left:

  1. Standards aren’t enough to fix the problems of dependence and power in philanthropy!  We need to overhaul the system and create new structures for working together.
  2. There isn’t a problem to address – the perceived conflicts in philanthropic partnerships are exaggerated.  Just because the work is hard doesn’t mean the system is broken.
  3. You can’t engineer a policy-based solution to a spiritual problem.  Partnership issues will dissolve when people focus more on the Lord and recognize their common dependence on God.
  4. Codes and policies are no substitute for deeper relationships with one another.  Making a greater effort to understand our neighbor will lead to more effective partnership, with or without a set of standards.

The Lausanne working group’s responses to these objections are:

  1. Yes, we can benefit from creating new forms.  Finding points of affirmation is a perfect starting point.
  2. Yes, the work is challenging, and good communication will help us to address challenges more effectively.
  3. Yes, human-centered solutions are insufficient.  Agreements must be developed and implemented in reliance on the Spirit.
  4. Yes, we must grow in understanding – and agreed-upon standards reflect an increasing level of understanding.

Watch the video again to see how some of these messages are communicated clearly and effectively.  That’s research in action!  Here, segmentation is not a tool to create or emphasize division but a means of addressing concerns to develop consensus and discover unity among varied perspectives.

Research for the sake of knowledge puffs up, but research for the sake of love builds up (variation on 1 Corinthians 8:1).  How are you are seeing research applied in your area of ministry?

Analyzing open-ended questions: A bright future for word clouds

Commercial survey research firms usually charge clients significantly extra to include “open-ended” questions in a survey.  They tend to be messy and time-consuming.  Traditionally, analysts would read through a selection of responses to create categories of frequent or typical responses, then read back through all of the responses to categorize them.

For publishing in a peer-reviewed journal, multiple people would categorize responses independently, then work together to create a synthesized coding scheme to limit bias.

Most qualitative text-analysis software still requires you to manually “code” responses.

With all that work, open-ended questions are still important in exploratory and qualitative research – and frequently satisfying for survey respondents looking for an opportunity to say what is on their mind, unhindered by structured response categories.

But the tag-cloud age has been a blessing to those without the time and money to do full, traditional analysis of text responses.   Graphics with words sized by frequency of use enables analysts to quickly get a sense of the nature of open-ended responses.

New editions of survey software – even budget packages like Survey Monkey – include cloud-creating tools to help users understand open-ended responses at a glance, without all the coding work.

Even those doing traditional coding enjoy working with clouds, which help analysts to quickly create an initial set of codes.

If your survey package doesn’t have cloud-generating capacity, no problem.  Worldle is a free site that lets you create art-like word clouds.  The clouds in the previous post were created using Worldle.  It’s a terrific, easy-to-use site that lets you paste in your text – our data came straight from a spreadsheet – and generate a word cloud with one click.  It automatically removes common words, allows you to choose the relative cloud shape, color scheme, font and orientation of the words.  We chose to illustrate the top 100 terms for each question.  Wordle lets you save and use your clouds however you want to.

I really like the tool’s artistic quality.  Wordle clouds almost beg to be shown to others.  Then they become motivated, too.  My daughter, upon first seeing Wordle, immediately had a vision about making a sign to promote a bake sale.  A few descriptive terms later, she had created a beautiful graphic to draw people’s attention.

This is where research moves from information to influence.  Imagine asking your constituents about their needs – or your organization’s impact – then printing a graphic of their responses to hang in your office as a reminder and motivator to staff.  Unlike a research report, which may or may not get read before being filed away (or worse!), word cloud art can keep research right in front of your team.  The graphic format makes the information more memorable as well.

Researchers, meanwhile, can compare and contrast different audience segments, as I did in the word cloud below.

What applications can you think of for word clouds?