How do you know if your work is working?
For those of us at mission-driven organizations, this is a critical question and one that is much harder to answer than it may appear. What we should measure and why is a question that leaders in the digital campaigning sector are currently struggling to answer. It turns out that understanding whether a strategy is actually moving you closer to your mission requires a lot more than counting the easiest-to-track numbers, or “vanity metrics” like pageviews or list size.
Let’s say you work at a large women’s rights group. An advocacy email goes out to hundreds of thousands of women’s rights supporters, and moments later, you can see how many people are opening it, clicking through to take action, or even opting out of membership. Sounds good! Not so fast.
Unfortunately, some of the most common measures online campaigning practitioners track tell us surprisingly little about whether we’re truly engaging the people we care most about over time. For organizations that are committed to regularly engaging large numbers of everyday people, that is a major problem.
“If what we measure grows, are we growing the right things?”
That is a question 18MR Executive Director C.M. Samala asks regularly. At an in-person convening of US online campaigning leaders co-hosted by Citizen Engagement Laboratory Education Fund (CELEF) last June, Samala asked questions that laid the foundation for a just released report entitled Beyond Vanity Metrics, a collaboration between CEL, a non-profit incubator for tech-fueled social change, and Mobilisation Lab at Greenpeace, a global learning hub for digital activism.
The new report explores how digital organizers can move #BeyondVanityMetrics to embrace more meaningful measures. It was shaped by convening conversations, surveying practitioners and organizations, and analyzing new metrics popping up in the sector. Trailblazers like corporate campaigning watchdog SumOfUs had recently shifted their team’s focus from vanity metrics like email list size to a more sophisticated monthly measure called “Members Returning for Action,” or MeRA. Just as importantly, they had established internal protocols to find and test new measures to ensure they were growing their source of power.
So how do we shift what we measure to be more meaningful?
The new report identifies and suggests concrete steps for digital campaigning organizations just starting to think about more meaningful metrics, as well as steps for those organizations already engaged in deepening their measures.
The study found that if a group really wants to measure what matters, they need to start talking internally and asking questions about what their current metrics incentivize. After thinking big about the mission and what’s needed, it’s important to drill down and get started. New metrics should be evaluated often, even quarterly.
Talking about metrics is half the battle.
Once you’ve started looking closely at a deeper set of metrics, however, it’s easy to get overwhelmed. The biggest immediate advice the report has for organizations with more advanced data programs is easy: simplify. By rallying around one or two priority, mission-focused metrics, the organization gains direction and focus. One example is a campaigning team that decides to foreground a metric like “member victory experiences per month,” which allows them to focus on what it would take to ensure every single member they are talking to participates in a winning campaign within a certain timeframe.
At mission-driven organizations, there is growing pressure to show quantitative data that points squarely to success.
Our funders and our members (and for some of us, our funder-members) want to know that we’re winning. That each of their dollars donated or moments spent clicking, marching, or calling are being wisely leveraged.
Vanity metrics become an easy proxy for showing partners that we are on the right track, in language they can grasp immediately, but they become dangerous when they distract us from the important work we are trying to accomplish — and the people upon whom we depend to exert influence and tip the scales toward justice on our issues.
More than ever, what we measure matters. To truly focus on what will supercharge our mission we have to take a deep look inside and be willing to challenge assumptions– again and again.
As individuals who wish to change the world for the better, however, we must rise to this challenge. Moving beyond vanity metrics will ensure that our hard work is really working.