Beginning Assessment and Evaluation for Church Communications

Why Assess?

We give intuition and guesswork a blank check in churches, especially when it comes to church administration and communications. But the reality is that our intuition and guesswork is often wrong—and maybe usually wrong—and we stink at checking our work. Pastors are overworked; volunteers are here one day, gone the other; and staff are often underpaid and lack continuing education funds.

Even still, you have access to a trove of data that is not only meaningful, it is also actionable. It doesn’t impact your budget, and it doesn’t take a data scientist to interpret. All it takes is time and energy, two of the greatest assets at our disposal.

Assessment affords your congregation a reality check, forges a path toward better stewardship of attention, and demystifies parts of human behavior online. You don’t have to submerge yourself in a sea of data; even dipping your toes into it will be more than enough.

If you haven’t yet, be sure to read about mission-driven, data-informed communications, which guides this approach.

Finally, full disclosure: this is primarily geared toward an initial, comprehensive assessment. This forms a scaffolding for an annual report, if you choose to assess regularly (and you should!).

Principles of Assessment

As strange as it might sound, the action-reflection method of learning of Clinical Pastoral Education (CPE) is a useful framework for assessment.

1. Assessment is inextricably linked with action. If you are asking for an assessment, you are asking for actionability (and if you’re not really looking to do anything afterward, then you need to ask hard, honest questions about your ministry and intentions). If you’re handed a report without recommendations, suggestions, or next steps, then it’s half complete. Even if you’re compiling analytics for an annual report, you should be selecting meaningful data demonstrating progress toward your ministry’s goals (i.e., it is situated in the context of ongoing action).

2. Assessment is backward- and forward-facing. Assessing church communications requires looking back for context and looking ahead for action. It requires understanding previous interventions, audience behavior, trends over time, and the history of the website or account. In looking ahead, assessment identifies mission-driven, data-informed goals and outline strategies to meet those goals.

3. Assessment is fundamentally concerned with interpretation. Data are complex and require an interpretive lens. There aren’t always clear-cut answers, and experimentation is required, so it’s important to be conscious of the hermeneutical process. Fortunately, there’s empirical evidence at your fingertips that can whittle down the interpretive demands of communications assessments.

Before you start

Depending on your communications team (if you have one!), you might need to double check a few basics.

Analytics

For websites, you need access to data, and depending on your site provider, it might not be available. Platforms like Wix, Squarespace, and Weebly collect analytics by default. It’s not always a given that WordPress sites collect analytics, even if you’ve contracted a developer or managed hosting provider.

If you don’t have analytic data—or know where to access them—you need to track it down through your host’s support center or by contacting your contractor. Depending on the granularity and your team’s experience, you might choose to install Google Analytics for free.

If a contractor or hosting provider tries to upsell you on “premium analytics,” or subscription-based services, politely decline and run like the wind.

Sample size

Make sure you have enough data to draw meaningful insight. For websites, 6-12 months of analytic data is generally needed, and for social media, consistent posting over a 90-day period is needed. If you’re evaluating email marketing (like MailChimp or Constant Contact), there should be at least 5-10 campaigns, or if it’s your weekly newsletter, a similar 90-day period will be more than enough.

Review your glossary

You’ll need to understand a few key terms for the assessment process. If you haven’t yet, check out my communications glossary for pastors and volunteers. 

What to ask for

“Data,” “analytics,” “email campaigns,” – it’s easy for the assessment process to become amorphous, dizzyingly complex, and a can of worms. Here’s what you should request from your communications team:

Survey

Conduct a congregational survey that quantifiably and quantitatively captures the following items:

• Overall sentiment regarding communications

• Channel preference and use

• Channel behavior

• Sentiment on channels

• Ease of use

• Perceived effectiveness

Be sure to capture respondent ages and gender if possible.

Data

Pull data from communication channels and prepare a readable reference document.

For the website, include:

1. Unique users

1. Sitewide 30-day average over the past 90 days.

2. Percent change in sitewide 30-day average of unique users (cf. preceding 30-day period).

3. List of top five pages, sorted by number of unique users.

4. Sources and relative share of traffic.

5. Location and geography (for churches with over 500 members).

2. Returning users

1. Sitewide 30-day average over the past 90 days.

2. List of top five pages, sorted by number of returning users.

3. Behavior

1. 30-day average bounce rate, including most common landing pages.

2. Homepage click rate and most common targets (what was clicked?).

3. Session time.

For social media, include:

1. Content

1. Total number of posts (past 90 days).

2. Percent change in number of posts (cf. preceding 90-day period).

3. Total number of stories (past 90 days).

4. Median post engagement (past 90 days).

5. Percent change in median post engagement (cf. preceding 90-day period).

6. Heatmap of posting frequency (i.e., show number of posts per day on the calendar).

2. Visibility

1. Total page reach (past 90 days).

2. Total page visits (past 90 days).

3. Percent change in page reach (cf. preceding 90-day period).

4. Percent change in page visits (cf. preceding 90-day period).

3. Benchmarking (Meta Business Suite only)

1. Published content percentile.

2. Followers percentile.

3. New followers percentile.

4. Interactions percentile.

For email marketing, include:

1. Campaigns (past 90 days)

1. Total number of campaigns.

2. Percent change of number of campaigns (cf. preceding 90-day period).

3. Open rate percentage.

4. Percent change in open rate (cf. preceding 90-day period).

5. Click rate percentage.

2. Subscribers (past 90 days)

1. Number of new subscribers.

2. Number of lost or removed subscribers.

3. Net gain or loss.

4. Comparison with previous 90-day period on each item.

3. Benchmarks

1. Average open rate for churches/nonprofits

2. Average click rate for churches/nonprofits

Report

Develop a written report for the Session (Church Council, Administrative Committee, etc.) and other stakeholders. 

• Report on the survey results.

• Summarize the church’s communication strategy as it stands, and identify whether it meets the needs of the congregation.

• If there are existing goals, evaluate progress on each goal.

• For each of the numbered areas in each section above, provide a summary and key takeaways.

• Make recommendations. If recommendations were made previously, report on the status (complete, incomplete, or in progress) and make any necessary notes.

Presentation

Provide a top-level presentation to relevant staff, followed by a presentation to the Session or the church’s governing body. Focus primarily on contextualizing action items and clarifying the necessary decision making.

---

Do your best

If you decide to narrow the scope of your assessment, use the core principles to guide your process. Focus on actionability and interpretation, listening to the data carefully while holding it loosely. At the end of the day, your church isn’t going to be data-driven. But it can be mission-driven, and data-informed.