How Useful Are ‘Do-It-Yourself’ Charity Evaluations?

It’s one big challenge everyone in the nonprofit world wants to crack: finding a simple way that ordinary donors can evaluate charity effectiveness.

In a recent issue of The Chronicle, Sean Stannard-Stockton, an adviser to donors, listed five questions that people can ask all types of charities to determine if they’re worth supporting.

Now GiveWell, the charity-evaluation group, has come out with a “Do-it-yourself charity evaluation” that gives donors different sets of questions depending on the field in which the nonprofit works.

Questions to ask a charity that works in education, for example, include:

• What do you do to improve K-12 education? What is your relationship with the school? Do you work within it or outside it?
• Who is targeted by your activities? What are the requirements for participation? In the case of over-subscription, how do you determine who gets in?
• Have you done a randomized controlled trial of your program? If not, are you planning to? If not, why not?

Mr. Stannard-Stockton says on his blog that it’s less important for a charity to be able to “prove” its effectiveness from its answers than for it to show it is prepared to answer questions about evaluation and that it has a commitment to obtain that evidence over time.

The question about randomized control trials (which test the efficacy of programs), meanwhile, is generating a discussion.

Isaac Castillo, of the Washington charity Latin American Youth Center, says on Mr. Stannard-Stockton’s blog that it’s the cost of such trials—not a lack of willingness to do them—that prevents his $15-million organization from doing more.

His suggestion to donors who want to give but are concerned about the lack of such testing: “Write a check for $750,000 over 3-4 years to the nonprofit so they can do the [randomized control trials].”

“Put your money where your expectations are,” he says. “Otherwise, lower you expectations to match what you are willing to fund for evaluation.”

Holden Karnofsky, a co-founder of GiveWell, responds by saying that donors who are choosing among charities to which they have no connection may want to support a more established group that has done such testing of its programs.

But for donors who have honed in on Mr. Castillo’s group for another reason, an answer like Mr. Castillo’s ought to be enough to satisfy.

How useful do you think such questions might be for donors?