What A Decade Of Evaluation Practice Taught Me That Evaluation Couldn't
In 2017, I wrote my first piece for AEA365, which is the American Evaluation Association’s daily blog read by practitioners across the field. I was a new evaluator, figuring out the mechanics. The piece was titled Tricks for Getting Started and Unstuck. The title says everything about where I was intellectually. The question I was asking was simple: how do I do this well when the path forward isn’t clear?
That’s the right question for a new evaluator. It just isn’t the most important one.
The evolution looked like progress
Over the years, the questions got more sophisticated. A later AEA365 piece examined how to steer evaluation inside a new organization by connecting the formal and informal, building stakeholder relationships, and aligning evaluation design to strategic context. The question had shifted from how do I execute to how do I connect evaluation to what the organization actually needs.
Then came the work on collaboration and storytelling. A 2023 AEA365 contribution explored how evaluation findings become useful only when collaboration is built into the process and when storytelling bridges the gap between data and decision. Around the same time, I developed an Introduction to Focus Groups eLearning course for the American Evaluation Association. Because focus groups, done rigorously, are one of the most powerful tools for elevating voice and nuance that quantitative data alone cannot capture.
Each step felt like growth. Better methodology. Sharper tools. More intentional process. More compelling stories. And it was growth and I am not walking any of it back.
But something was missing from the frame. Serving on the boards of the American Evaluation Association and the Texas Evaluation Network gave me a front row seat to what the field was celebrating. It also gave me a clear view of what it was missing. And I couldn’t see it clearly until I looked at the same work through a different lens entirely.
The lens I wasn’t using
I came to evaluation post-PhD, with a research and evaluation methodology background, through organizational work, philanthropic engagement, and the kind of applied learning that happens when you’re already inside the rooms where decisions get made before you have the formal title to match. That entry point shaped everything I see in the work.
Alongside my evaluation career, I have spent two decades as a philanthropic practitioner, directing my own charitable resources, leading giving circles, serving on philanthropic steering committees, and sitting in the rooms where funders make allocation decisions.
That dual positioning created a tension I couldn’t resolve with evaluation tools alone. There were specific, uncomfortable moments when I looked at rigorous, well-designed evaluation work, and something didn’t sit right. The methodology was sound. The collaboration was genuine. The story was compelling. And yet I kept returning to a question the evaluation framework wasn’t asking:
Does what we’re evaluating actually track toward what the funder intended to produce?
Not whether the evaluation was rigorous. Not whether the story was compelling. Not whether the collaboration was genuine or the focus groups were well-designed. Those are questions about the quality of the evaluation practice.
The harder question is one order deeper: Does the program being evaluated actually create the conditions for the outcome the funder believes they’re funding? A beautifully executed evaluation of an incoherent investment is still an evaluation of an incoherent investment.
The difference is not in the quality of the measurement. It is in whether anyone was willing to interrogate the logic before the resources were committed and the story was already written. The rigor of the measurement doesn’t repair the gap in the theory of change.
What nearly a decade of practice actually taught me
The tools matter! Rigorous collaboration matters and compelling storytelling matters. Focus groups that elevate voice and nuance also matter. Every piece of craft I developed across those years of evaluation practice matters. But craft in service of the wrong question produces polished findings that do not move the pressing issues deeply.
The question I wasn’t asking; the one the evaluation field largely still isn’t asking at the level. Whether what gets funded actually does what it claims to do in a way that produces what the funder actually intends. That is not a methodology question and it requires standing simultaneously within the evaluation framework and within the philanthropic decision, holding both at once to see them clearly.
I could not write that AEA365 piece in 2017, the formation was not complete yet. The dual lens was not fully operational. The question was not yet visible to me with sufficient precision. It is now.
That is what a decade of practice taught me, that evaluation practice alone could not: that the most important question in philanthropic evaluation is not how well we measure what happened. It is whether what happened honestly tracks toward what the investment was designed to produce.
Everything I write here starts from that question.
Rhonda Williams, Ph.D. is an evaluator and philanthropic practitioner examining how capital deploys toward or away from the outcomes it intends to produce.