Conclusion
It was hoped that this study of long-term change in the Koshi hills would collect and compile a comprehensive
database, which would not only provide new information and new insights into the complex
process of development and change over a relatively long period of time but a coherent analysis
14 American Journal of Evaluation
Downloaded from aje.sagepub.com at Naresuan University on January 23, 2016
of cause and effect. The aim was to describe and explain the major features of long-term change, to
identify the main drivers of change, and to assess the role of development interventions. This would
thus enable policy makers and planners to have a clearer and more comprehensive idea regarding the
role of development interventions in the wider process of change and to be able to better evaluate
and assess what kinds of interventions had what kind of effect over time.
The uniqueness of the study however in its scale and ambition required pulling together scattered
and disparate information drawn from a wide variety of sources, using diverse methods, to build up a
coherent and comprehensive analysis of change within the Koshi hills. This inevitably brought many
challenges that are important to consider when undertaking similar evaluations in other contexts or
countries. Nonetheless, the study provided interesting findings useful for the development community
in Nepal and other researchers and donors interested in understanding long-term change.
Notably, conducting an evaluation that captures these various aspects of long-term development
requires a number of different approaches:
(a) A multidisciplinary (or better still an interdisciplinary) approach, which fully recognizes
the complexity of relationships between ‘‘variables’’ and of possible chains of causation,
particularly when considered over the medium and long term and when the ‘‘noise’’ of
external factors becomes greater;
(b) An exceptionally wide-ranging review—and careful analysis—of existing information
(objective and subjective; quantitative and qualitative) on development and change covering
an exceptionally long period of time;
(c) An effective integration and synthesis of the information collected, reviewed, and analyzed,
utilizing a strong common evaluation framework routed in systems thinking. This in order to
be able to generate preliminary hypotheses (narratives, suggested chains of causation) to be
tested for robustness and plausibility, and subjected to argument, debate, and the development
of counterhypotheses; and
(d) Independent ‘‘reality’’ checks of preliminary theories, including re-visiting the original
information reviewed, re-considering the review and analysis, and where appropriate
undertaking further research, including fieldwork. There is a need for qualitative evaluation
as interpretative lens and to humanize the statistics, recognizing the limits of quantitative
evaluations as adequate contribution to knowledge generation.
In addition, we found that long-termimpact is not programor sector-specific; but is produced by the
interactions of intended and unintended changes over time. This provides a crucial framing concept by
which to understand the development successes and failures in the Koshi hills over the study period and
other similar studies of long-term change in other contexts. By extending the time frame and focus of
study it becomes clear that sustainable, long-termchange is a result of the cumulative and interconnected
impact of multiple programs and factors. This complexity is difficult to capture and address in single
programs and long-term evaluation methods must take this into consideration. Any requests for underlying
research materials related to this paper can be accessed by contacting the corresponding author.
Declaration of Conflicting Interests
The authors declared no potential conflicts of interest with respect to the research, authorship, and/or publication
of this article.
Funding
The authors declared the following financial support for the research, authorship, and/or publication of this article.
We received financial support for this study through the UK Department for International Development
(DFID) in London, UK.