Beyond the Numbers Evaluating Our Evaluations

Beyond the Numbers: Evaluating Our Evaluations

As a woman who tries to carefully investigate the effects of philanthropic efforts on a global scale, I have learned that when it comes to social and community efforts, impact often goes beyond the reaches of traditional metrics. 

In fact, I would go so far as to say ill-chosen measures can sabotage even the most well-intended efforts.  

Recently, a powerful article reminded me of the nuanced nature of the work. It highlighted a story where communities served by an NGO working to improve livelihoods and income identified the absence of fear as the true measure of change

 

We need to reflect on how we evaluate impact in the development sector.

Unlike manufacturing or industry, social change often occurs within multiple systems that require more than outputs as a measure of success. 

As interventions are implemented, there are very often unintended consequences, both positive and negative, that change the system—and therefore the outcomes—in unexpected ways. 

Traditional evaluation approaches can miss the importance of context, and fall short when focusing on isolated metrics and assumed outcomes for success.  

 

We need a new approach, where evaluators become co-learners and shape value in partnership with those closest to the problems being solved.

 

Evaluating Lasting Change from Contextual Intervention

Practically speaking, your metrics will drive your efforts. If the elements of a program are measuring the wrong thing, the program simply cannot fulfill its impact potential.

For example, we know affordable housing is a hard and fast measureable solution to homelessness, but it is only one of the multiple determinants of homelessness and poverty. If we only focus on housing and miss the varied needs and wraparound services of all those seeking housing, we will remain stagnant in solving the problem.

A mental-health based program in our hypothetical homeless community providing trauma support groups, psychiatric care, and employment services would not only create concrete change but also collaborate with other organizations and engage the community. This type of work leads to holistic intervention and advocacy, impacting homelessness at multiple levels.

If we want to see lasting change that affects a community, we must expect to enter into a long term partnership with humility and mutual accountability as a two-way street in our philanthropic work as well as our learning and measurement practices.  

It’s time to qualify and quantify a shared impact created in community, and buck the trends that prioritize fast fixes and disconnected interventions.  

By embracing a holistic view and redefining success, we can make a lasting difference in the lives of those we serve.