Without development research, efforts to understand human problems and create
opportunities for the poor would be mired in faulty assumptions and a confusing flea
market of reinvented wheels. To make sure this research, whether in agriculture or any
other domain, is tightly geared to the central development challenges of our time, it must
systematically and regularly examine the anatomy of its own impact. The result is a
learning process that continuously sharpens the focus of research and improves its
performance.
Today, international agricultural research has a diverse agenda. Reducing
poverty and protecting natural resources demand a creative mix of solutions whose benefits
are often hard to gauge. At the same time, our investors are justifiably vociferous in
asking scientists to account carefully for the public funds they spend. This is the
context in which the 1999-2000 annual report of the International Center for Tropical
Agriculture (CIAT) examines the elusive concept of research impact.
Anatomy of Impact: Director General’s Message
At CIAT measuring research impact means determining how specific results contribute to
the Center’s mission of reducing "hunger and poverty in the tropics through
collaborative research that improves agricultural productivity and natural resource
management."
A defining characteristic of poverty is the lack of funds to invest in long-term goals
such as protecting or improving natural resources. Therefore, even as we pursue such ends,
we must also solve immediate problems and develop technologies with a short-term payoff.
This is both a moral obligation and a practical requirement for achieving impact.
Whether the solutions offered by research have impact obviously depends on users
adopting them. And this, in turn, requires that we provide people with technologies,
information, and services they need or want and can manage and afford. Only by generating
short-term benefits can our work keep users interested and involved in longer term
endeavors.
In recent years many development agencies have come to focus on "results-based
management" as a way to increase impact. Hidden just beneath the surface of the
current management mantra is this tough issue of balance—between activities for
solving immediate problems and those that will yield broad, deep, lasting changes.
An energetic research mix
There are at least two explanations for the current concern about results-based
management and what some have called the "projectizing" of development research.
One is the perception that, after four decades of trying to bridge the North-South
development gap, we still haven’t got it right. Yes, there have been major successes,
like the Green Revolution and the eradication of smallpox. But poverty, malnutrition, and
environmental degradation persist. Overall, the vast human potential of this planet is
still far from being fully tapped.
The second explanation has to do with competition for funding and the growing pressure
for public accountability. In attempting to justify giving grants to centers like CIAT,
donors need a steady stream of success stories, along with impact information that can be
packaged and directed to policy makers and voting publics.
Personally, I think this is a good thing—as long as it doesn’t exclude the
longer term perspective. After all, the mandates of CIAT and the other Future Harvest
centers financed through the Consultative Group on International Agricultural Research
(CGIAR) require them to conduct strategic research that will provide the scientific basis
for solving problems tomorrow. Ensuring widespread and sustainable benefits to people
demands an energetic mix of short- and long-term research. Contributions must come from a
range of social and natural sciences and from many partners, especially the beneficiaries
themselves.
We need to recognize, though, that win-win solutions aren’t always possible. At
times the users of research results will face difficult tradeoffs and will have to make
hard choices between short-term benefits and the longer term social or environmental good.
Some groups will undeniably experience negative short-term impacts.
A key challenge of our work, then, is to demonstrate substantial short-term gains as
concrete steps in the longer journey through which science creates new opportunities for
bringing far-reaching sustainable change to whole communities and regions. These
opportunities include improved environmental services and sustainability at the landscape
level as well as the accumulation of social capital. Both these benefits accrue over many
years, and both are difficult, though far from impossible, to measure.
Farmers as researchers
The emergence of committees for local agricultural research (called CIALs) in eight
Latin American countries illustrates well the profound benefits that can come from
long-term commitment to a good idea. CIAT has served as the scientific midwife to this
growing farmer movement for about a decade now. Through their own experiments, farmers are
bringing practical new technologies and knowledge to their communities. The result may be
a new variety of maize, a better way to grow blackberries, or an effective compost recipe
for fertilizing tomatoes. These are short-term outcomes with measurable impacts.
But that is only the beginning of the story. Doing agricultural research builds the
technical expertise and confidence of farmers. Along with experimental crops, they
cultivate organizational, scientific, and networking skills. They build community trust
and learn to articulate needs—and complaints—to government agencies. This social
capital is then reinvested in new enterprises or otherwise absorbed by other community
members. New participants develop a strong entrepreneurial sense, a nose for commercial
opportunity. Poverty and passivity are replaced by activism, optimism, self-reliance, and
the cash needed to send a daughter or son to school.
As a catalyst of the CIAL movement, CIAT has learned a lot about how to make the link
between R and D. While it’s fairly easy to measure the immediate benefits of
participatory research, like adoption of farmer-selected crop varieties, it’s much
harder to gauge the wider, longer term impact on a country or region’s social
capital. Yet, the benefits are real and need to be scaled up. This can happen only if our
research enterprise is well balanced and lessons from both successes and failures are
systematically fed back into the next round of investigation. There is no single equation
defining this process. It’s a matter of learning by doing: as much an art as a
science.
Strategies for impact
As CIAT’s new director general, I see the issue of impact—how to achieve it
and how to measure it—as central to producing relevant international public goods.
The ultimate litmus test, of course, is whether and how such products improve poor
people’s lives.
There are several things CIAT can do, and is doing, to ensure that maximum benefits are
delivered efficiently to the greatest number of people. First, we must build on our
comparative advantages. Fortunately, these span both the natural and social sciences. We
have a good track record in research on crop improvement and natural resource management,
with particular strength in specialized areas such as biotechnology, geographic
information systems, agricultural economics, and participatory research methods. This mix
of expertise, I believe, will allow CIAT to position itself in the coming years as a
socially and environmentally progressive research institution.
Second, where CIAT does not have a clear comparative advantage, we’ll secure the
necessary expertise through partnerships. The emergence of an international science park
on the Center’s campus in Colombia, where we now host 18 national and international
institutions, is just one strategy for expanding our global network of collaborators.
But partnering shouldn’t be limited to research. The experience of CIAT bean
research in Central Africa a few years ago offers valuable lessons here. Part of our
strategy was to work with major development projects in the region. By funneling results
to NGOs with good farmer contacts, we were able to achieve much more than if each group
had acted alone. That collaboration continues today with World Vision and the Rwandan
national program, creating substantial benefits for thousands of farmers.
Third, CIAT needs yardsticks of the actual and potential impact of R&D
interventions. One domain in which it is indeed difficult to establish such indicators of
sustainable development is research on natural resource management. Nonetheless, CIAT is
putting together a "research-to-development" framework for systematically
analyzing our impact in this area.
Finally, CIAT must continually scan the external environment. This means analyzing
sociopolitical conditions in client countries and tailoring interventions to fit.
Two key targets of environmental scanning are farmer preferences and the nature of
local poverty. If we don’t know precisely what farmers want, our research will tend
to go off on a wild goose chase. Recent findings suggest that participatory research
strategies can cut by as much as half the traditional time lag between the first cross of
promising germplasm lines and significant adoption of improved varieties by farmers.
Similarly, we must understand the dynamics of human deprivation, especially from the
poor’s own point of view. Otherwise, how can we hope to provide relevant solutions? A
conference on poverty, held at San José, Costa Rica, in September 1999 and organized by
my predecessor, Grant Scobie, clearly demonstrated the heterogeneous nature of poverty and
refocused attention on the very heart of the Future Harvest centers’ mandate. Events
like this are the runway lights to future impact, guidelines for a new director general.
Joachim Voss
Director General, CIAT
A Donor View of Impact
Interview with Andrew Bennett, DFID, UK
Andrew Bennett is director of rural livelihoods and environment and chief natural
resources adviser at the British Department for International Development (DFID). He also
serves as chair of the Oversight Committee of the Consultative Group on International
Agricultural Research (CGIAR). Earlier he spent over 15 years as a tropical agronomist in
the West Indies, Malawi, Nepal, and Sudan.
Perspective: What importance does DFID, as a major bilateral development donor, assign
to impact assessment?
Bennett: British government policy is to focus development assistance on poverty
reduction. So, we give increasing emphasis to the impact of our activities on poor people.
But this doesn’t mean all investment in research must directly reduce poverty through
the delivery of technologies to farmers. Research that improves policies, stimulates
economic growth, or keeps food prices at reasonable levels is equally important.
The British public, civil society organizations, and our parliament are interested in
the impact of our work and in the success of the partnerships we enter. But they’re
also interested in how funds are spent. In development work we must take calculated risks,
tell people the nature of those risks, and then do all we can to direct those actions into
a successful outcome.
Perspective: Research on agriculture and natural resources can take many years to
produce results and benefit communities. What can centers like CIAT do to measure impact
usefully?
Bennett: Gone are the days when research could take 10 years to solve a problem. In an
impatient world where communications are fast and easy, it’s inevitable that results
are expected quickly. This is human nature. Telling people that impact will take decades
will not warm hearts and loosen purse strings.
Engaging communities in research and agreeing on priorities can help retain support and
interest. It can also help identify and strengthen causal links and the pathways whereby
research products are taken up and applied. The CGIAR and CIAT cannot manage or resource
the entire process. This points to the need for partnerships with clear responsibilities
and, where possible, time scales. It’s also important to agree on rigorous
performance indicators with the end users, so that progress is monitored along the road to
impact.
CIAT is fully accountable for selecting topics relevant to the poor, defining expected
outputs, and delivering them on time. It’s also in the interest of researchers to
understand and engineer the pathways by which research results can be adopted. Researchers
should then find out whether results have in fact benefited poor communities.
Perspective: Many development agencies have, like DFID, set poverty reduction as their
overarching goal. How does this shift of emphasis affect the way that the impact of
research for development should be assessed?
Bennett: We’ve found a livelihoods approach to be useful and productive for
developing and implementing programs. It has helped us to better understand the causes of
poverty and the opportunities that might exist for poor people to develop their assets,
reduce their vulnerability, and improve their livelihoods. The livelihoods approach
provides an analytical framework; it uses tried and tested tools to identify points of
entry for programs aimed at helping poor people.
An important feature of a livelihoods approach in a rural environment is that it
doesn’t presuppose that the best way forward is solely through increasing production
or the productivity of natural resources. Roads and access to credit or markets may be the
most important first step or access to better education, health, and water. However, in
the end improving the productivity of natural resources and protecting the environment
will continue to play a key role in sustaining rural livelihoods.
A livelihoods approach increases the demand for social science research skills and
interdisciplinary working. Rural development and environmental protection are about people
and the ways in which they live. Hence research for development must be assessed for its
impacts on people. Increasing production and productivity are important, but the results
must be sustainable in ways that result in better livelihoods for poor people.
Perspective: It’s often said that the best impact assessments are participatory,
directly involving key stakeholders in project design and implementation. Yet,
there’s clearly some value to drawing on the expertise of outsiders who have no
personal stake in the project being evaluated. How does one balance the need for
stakeholder participation with the need for an arm’s length assessment of impact?
Bennett: I agree that participatory processes for agenda setting, performance, and
impact assessment are powerful and useful tools. But they’re susceptible to bias and
fashions. However, more dispassionate empirical systems of data collection and assessment
are often expensive and fail to capture public concerns and opinions.
Outsiders can bring valuable perspectives and views by challenging prejudices and
suggesting new approaches and ideas. They can bring lessons from experience in other parts
of the world or in different sectors. Given the need to improve the livelihoods of poor
people, I think it’s a reasonable rule of thumb to favor a participatory process
involving the institutions and people we seek to help as a matter of routine during the
project. We should use the external inputs for ex-post evaluations and impact assessments
after the project has ended. But there should also be consultations with the end users.
Perspective: Many impact evaluations focus on success stories. Yet analysis of failed
projects surely can provide valuable lessons for new initiatives. How can open dialog on
past weaknesses or failures contribute to future impact without jeopardizing a development
agency’s credibility and funding prospects?
Bennett: It’s perfectly reasonable to focus on success stories, but I agree that
there’s much to be learned from our mistakes. There’s nothing basically wrong in
not being fully successful. The crime is making the same mistake again or in not
identifying, explaining, or quantifying the nature of the risks.
I agree that the best way to handle these issues is through open dialog and discussion.
There’s a risk that being "too open" might have a short-term impact on
confidence. But in the longer term, it’s better to be honest and to share successes,
experience, and disappointments. A problem shared is a problem halved. This is where an
impact culture and systematic monitoring are useful tools.
Perspective: What key actions do you think recipients of donor funding can take to
enhance their impact on the world’s poor and the natural resource base and measure it
usefully and convincingly?
Bennett: I don’t pretend to have all the answers to that question, but I do have a
few suggestions:
- Involve the intended beneficiaries in problem identification, program design,
monitoring, testing results, and impact assessment.
- Build purposeful partnerships that bridge from center to field, with clear roles and
responsibilities.
- Share successes and disappointments.
- Encourage a learning culture focused on outputs not process or techniques; the interest
should be in impact and outcomes.
- Take risks, but tell people they are risks.
- Think big, but focus on fewer issues.
- Don’t forget the private sector.
- Don’t be afraid to identify development constraints outside the field of
agricultural and natural resources research.
|
