I once saw a quantitative survey turn to dust, literally. I was a young graduate student in Tucson, Arizona when an older anthropologist asked me if I would like to write up the results of a survey she had conducted on the city’s largely Hispanic south side. She swung open the doors to her storage shed, revealing a large, cardboard box. When the anthropologist tried to lift some of the forms out of the box, they crumbled in her fingers. Tucson’s warm, dry climate is perfect for termites, which had carved their galleries throughout the sheets of paper.

For that anthropologist, going door-to-door with her questionnaire had been the fun part of the survey. Analyzing the results and writing up the conclusions were harder. In the end the termites benefited the most from the survey.

A few years later, I found myself in northern Portugal, on a questionnaire study of smallholder farmers. I was part of a team of anthropologists and economists who designed the survey form, a straightforward task – or so we thought at the time. But at 20 pages, the form took about two hours to fill out. To encourage farmers to take part, we said that their answers would make policy-makers more responsive to agriculture, which may not have believable.

After we surveyed six parishes in the Entre-Douro-e-Minho province I went to live in one of them, Pedralva. There I learned how much the survey had annoyed the farmers. One couple had missed their irrigation turn while answering questions. One prosperous farmer complained how long the survey took and said that: “They even counted the eyes of the chickens!� That was an exaggeration (we had asked how many rabbits and chickens people had) but a sign of how frustrating farmers found the lengthy, prying survey.

Even worse, the farmers mistrusted the survey’s intentions. The farmers assumed that the tax bureau would be informed of the results, so they claimed to have harvested a fraction of their real yields, inadvertently making their well-adapted farming systems appear unproductive.

Eventually I learned to write shorter, more focused surveys, and to enter the data every night on a spreadsheet. And prizes can help to take the sting out of lost time. Once in the Chapare, Bolivia my colleagues and I rewarded each farmer we interviewed with three kilos of mineral fertilizer, left over from an earlier project. They liked the gift so much that one of them took the survey twice.

Sometimes four or five questions are enough. In Bolivia I once worked with a project that gathered hundreds of farmers for three “technology fairs� to watch other farmers demonstrate new ideas such as metal plows or fertilized quinoa. At the end we simply asked the fair goers what ideas they liked and which ones they wanted to try. The questionnaire was so short that a dozen agronomists could administer it in a few minutes. We could get feedback from some 200 farmers before breaking for lunch.

Of course times have changed. Surveys in the city or in the villages can now be entered electronically on a tablet. The questionnaires being filled out today are immune to termites, and you can send them out on-line.

But one thing remains the same. People still don’t like to answer long questionnaires. When you fill out a questionnaire in person, the respondents may be too polite to break off the interview, but with an on-line version, fatigue sets in quickly. On-line surveys yield the best results when they are short. Some respondents are willing to share more during follow-up phone calls or emails (as we have seen in previous blog stories (Families, land and videos in Northern Uganda. Watching videos to become a dairy expert, and Drip irrigation saves water in South Sudan).

Whether on-line or in-person, a few simple questions may be as revealing as a long and tedious questionnaire that tries too hard to gather information. If do you need answers to lots of questions, consider rewarding people for the time they give you.

Further reading

The results of the first Portuguese survey eventually contributed to:

Pearson, S.R., F. Avillez, J.W. Bentley, T. J. Finan, R. Fox, T. Josling, M. Langworthy, E. Monke, & S. Tangermann 1987 Portuguese Agriculture in Transition. Ithaca: Cornell University Press.

My community study in Entre-Douro-e-Minho:

Bentley, Jeffery W. 1992 Today There Is No Misery: The Ethnography of Farming in Northwest Portugal. Tucson: University of Arizona Press.

The short survey in the Chapare (where people received a gift of fertilizer for answering our questions) contributed to:

Bentley, Jeffery W. 2003 Desarrollo Participativo de Tecnología en el Trópico de Cochabamba. Cochabamba: Development Alternatives, Inc.

The results from the questionnaire at the technology fairs:

Bentley, Jeffery W., Graham Thiele, Rolando Oros & Claudio Velasco 2011 “Cinderella’s Slipper: Sondeo Surveys and Technology Fairs for Gauging Demand,� pp. 276-301. In André Devaux, Miguel Ordinola & Douglas Horton (eds.) Innovation for Development: The Papa Andina Experience. 418 pp. Lima: International Potato Center. Originally published in 2004 as AgREN Network Paper No. 138.

Bentley, Jeffery W., Claudio Velasco, Félix Rodríguez, Rolando Oros, Rubén Botello, Morag Webb, André Devaux & Graham Thiele 2011 “Unspoken Demands for Farm Technology�. pp. 302-324. In André Devaux, Miguel Ordinola & Douglas Horton (eds.) Innovation for Development: The Papa Andina Experience. 418 pp. Lima: International Potato Center. Originally published in 2007 in International Journal of Agricultural Sustainability 5(1): 70-84.