Distributed, present, working, used

A shea programme we verify distributes improved cookstoves and plants shea trees each year. Both appear in reporting as single lines. Both are, on inspection, several different measurements stacked on one another, and the distance between the top and the bottom of that stack is where programme claims most often become hard to defend.

Several questions wearing one indicator

Take the stove. Behind the reported figure sit at least five separate facts: the stove was handed over, the stove is in the compound, the stove still works, the stove was used to cook the most recent main meal, and the open fire it was meant to replace is no longer in use. A programme can do well on the first and much less well on the last.

Trees work the same way. Planted is a procurement and labour fact. Surviving is a biological outcome that depends on rainfall and on the season in which the count is taken. Protected, which in parkland systems means fenced, watched or respected by herders, is a social outcome, and it is the one that predicts a tree reaching maturity. Three numbers, often reported as one.

Why the answers differ

They differ for reasons that are well documented and entirely ordinary.

• Stacking. Households rarely switch fuels outright. They add a stove and keep the open fire for the dishes it cooks better, for large pots, and for evenings when the new stove is already in use. Both statements, I use the improved stove and I use the open fire, can be true at once.
• Courtesy. The person asking often arrived with the programme, or is assumed to have. Reported use tends to run higher than observed use for the same reason a guest praises the food.
• Season and hour. A visit at ten in the morning in the dry season sees a different kitchen from a visit at six in the evening in the rains, and finds a different person at home.
• Who answers. The woman registered for the stove is not always the person who cooked the previous evening.

What tends to work better

Much of this is addressed by looking rather than asking, and by agreeing what counts as a yes before anyone goes to the field.

Observe, with a condition check. Enumerators record whether the stove is present, whether the combustion chamber is intact, and whether there is ash in it. Ash is a useful witness, and looking costs nothing.

Ask about the last meal rather than about habits. What did you cook on last night produces an answer that can be compared with the state of both fires. How often do you use the improved stove produces an estimate of how the respondent would like to be seen.

Compare the two fires in the same visit. The outcome of interest is not stove use, it is displacement, and that comparison has to happen in one visit.

Separate survival from protection. Count surviving trees at a fixed point in the agricultural calendar each year, and record protection as its own observed variable. Where only one tree number can be afforded, survival at the end of the dry season is usually the one worth paying for.

The aim is not a smaller reported figure. It is a figure that means something specific, which is the version a reviewer, an auditor or a certification body can work with.

What to take from this