Last week I had the opportunity to speak to students in Dr. Christine Moe‘s water and sanitation class at Emory University. The theme was “10 un-FAQs”; or 10 questions that development professionals should ask themselves more frequently before embarking on water and sanitation projects. I chose this because after 13 years in international development, I have a lot of questions. I’m going to list them here and elaborate on each in future blogs.
un-FAQ 1: Why do people want a safe water project?
un FAQ 2: What happened before you got there?
un FAQ 3: Are you the best organization to help?
un FAQ 4: Is the technology you are planning to use appropriate?
un FAQ 5: How much will it really cost?
un FAQ 6: Who else is working in this area?
un FAQ 7: Who should pay for it?
un FAQ 8: Could your money be better spent on something besides a project?
By Susan Davis, Executive Director, Improve International
One of my hobbies is trying new restaurants. Since I’ve moved to New York I’ve had no limits of restaurants to choose from. Like most people I choose based on recommendations from friends, or strangers (Yelp), after reading a review, or by just walking by. But there’s an even more important criterion that I’m embarrassed as a public health degree-holder that I didn’t even think about: the sanitary inspection grade. A good foodie friend of mine pointed them out to me a few months ago. I’ve never even noticed them before, but according to its website, the New York Health Department has required restaurants to post sanitary inspection letter grades since July 2010. Maybe they have ramped up their inspections recently.
My foodie friend is obsessed with restaurant grades, and now I am too. She works in the food service industry and knows that even an “A” (the best) grade means the restaurant can have a few violations of the health codes. So I wondered, what’s behind those ratings? Well here’s the simplified version (underlining is mine): (from How we score)
In addition to the letter posted in the front window, you can see information on inspections online. You can click on links for historical ratings with specific violations described as well. I just looked up the ratings for a place in my neighborhood (see below). They have improved to score of 7 and a grade A from their score of 20 violation points in the fall. I trust the process; I ate there twice last weekend.
Besides the obvious relevance to my eating habits and health, part of the reason I find this model interesting is that there have been lots of discussions in the charity world about impact evaluations and ratings. Right now there are a few rating organizations Probably the best known is Charity Navigator, which gives stars based on financial data. They do not visit the charity’s work; rather they call themselves “an impartial evaluator of publicly reported financial information.” In the restaurant analogy, this would be providing the number of $$$$, but nothing about the menu or how the food tastes. This is useful information but doesn’t give the whole picture of the effectiveness of the work of the charity.
An aspect I’d like to investigate is whether the restaurant inspections help the restaurant improve its compliance. In an unscientific survey, I looked up some restaurants that I’ve been to in the past year and found that several (but not all) of them had violation points going down with each new inspection. I would expect more $$$$ to equal less violation points, but that doesn’t seem to be the case. It also would be interesting to find out whether customers know enough about restaurant grades to make decisions about what restaurants to visit. I know I am throwing away the menus and cards for the restaurants without A grades.
Of course restaurants are different from water projects, but the point is the third-party evaluation can be useful. I’ve established Improve International to help provide the third-party “restaurant visit” for water and sanitation projects in the developing world. The goal is to help organizations learn and innovate from what’s working and what’s not.
I know of a few organizations that do regular or occasional evaluation of their work in the field, and some who engage third party evaluators. See examples of how the evaluation information is presented from Water For People, Water1st, and water.org.
There are not any global metrics that would lead to letter grades for evaluating water and sanitation projects, but a few organizations are coming together in September 2011 to try to reach some consensus on common core indicators. Even if there were grades or rating standards, there isn’t yet a corps of “health inspectors” for water and sanitation (or any international development) projects like the NY Department of Health must have. But that’s part of my vision for what Improve International can help stimulate: the development of oversight organizations in each country where water and sanitation projects are being built.
 Freemans (A), Ofrenda (A), Bluebell (A), Mary Ann’s (A), BLT Prime (A), Vamos (grade pending), Baoguette (A), La Pizza Fresca Ristorante (A), Jack Bistro (A), Defonte’s (B!), Da Andrea (A), Ulysses (grade pending), Mike Due Pizza (C!), Haru (A), Pipa (A), Great Sichuan (grade pending), Soi 30 Thai (A), Jean George (A)
 Full disclosure: I recently worked for Water For People and also worked for WaterPartners International, which is now water.org.