Use of mixed methods in long-term retrospective evaluation. We found the importance of a participatory
component essential to conducting analysis and framing narratives, though more in-depth qualitative
work could have strengthened findings. An important consideration in the original design of the
study was the need to go beyond the confines of program and intervention logic which had defined
development assistance throughout the 40-year period under review and understand from the outset
how people living in the area had themselves experienced and perceived change. The RCA findings
are important, in that they are the only source that provides an indication of the subjective experiences
and views of people living in the region, and provide often vivid accounts and impressions of
those experiences and views. But the fieldwork was limited in time, space, and scope and was undertaken
after the extensive documentary review. It was not possible therefore to feed the results of the
fieldwork into an ongoing process of data collation and quantitative analysis as part of the delivery
of other individual research components; RCA studies while insightful and relevant cannot provide
firm conclusions regarding the effects of development interventions.
Rather, RCA insights were used to frame causal narratives grounded in the experience and perceptions
of people living in the area rather than as a primary source of qualitative data for the analysis
as a whole. This approach provided insights into how people themselves viewed change,
removed from the usual externally imposed program or sectoral lenses. It was able to report on how
some people in the area thought change came about and their view of the interplay of outside interventions
and local knowledge, and the role of government, development partners and private sector,
as well as their perceptions of the varying significance of different factors. It was able to distinguish
differences between intended benefits and actual benefits and differences between stated change and
actual behavioral change.
A nonprescriptive method such as the RCA proved an effective tool for framing narratives and
illustrating the views and experiences of local people. It also provided an effective means of identifying
alternative avenues of investigation and inquiry, causing us, as researchers, to question previously
assumed truths or realities and return to the data with a more critical appraisal. However, it
does need to be combined more systematically with other sources of data to generate more rigorous
and generalizable findings.
Managing interdisciplinary teams. The management of the research process itself needs to be strong in
order to ensure integration (both coherence and continuity) and adherence to a clear program timetable.
The team involved must be able to function as a whole, with individual specialisms harnessed
to address a well-defined set of issues and produce a defined set of outputs.
Throughout the data collection and compilation process, several analytical ‘‘check-ins’’ were
conducted, particularly in sharing the results of the documentary review to frame the data collection
activities of the RCA study in an effort to encourage interdisciplinary exchange among researchers.
12 American Journal of Evaluation
Downloaded from aje.sagepub.com at Naresuan University on January 23, 2016
However, the extensive period of documentary review was carried out for the most part on a sectoral
or disciplinary basis, with individual members of the team tending to work on their own area of
expertise and sector rather than in a fully integrated manner. Furthermore, the description of the
major findings from the documentary review tended to precede the analysis rather than for the two
to proceed together. The analysis was, in effect, largely undertaken as a later phase in the project.
This led to a number of weaknesses in the study that should be considered by future researchers.
Effective working relationships, coordination, and opportunities to exchange and analyze data in
a sequential manner throughout the study are essential to success. Future researchers should consider
investigating organizational tools that can be used to improve this critical process, including the
right use of general and technical steering committees, or holding regular plenary sessions and/or
refocusing workshops at regular intervals with key stakeholder groups.
Use of mixed methods in long-term retrospective evaluation. We found the importance of a participatorycomponent essential to conducting analysis and framing narratives, though more in-depth qualitativework could have strengthened findings. An important consideration in the original design of thestudy was the need to go beyond the confines of program and intervention logic which had defineddevelopment assistance throughout the 40-year period under review and understand from the outsethow people living in the area had themselves experienced and perceived change. The RCA findingsare important, in that they are the only source that provides an indication of the subjective experiencesand views of people living in the region, and provide often vivid accounts and impressions ofthose experiences and views. But the fieldwork was limited in time, space, and scope and was undertakenafter the extensive documentary review. It was not possible therefore to feed the results of thefieldwork into an ongoing process of data collation and quantitative analysis as part of the deliveryof other individual research components; RCA studies while insightful and relevant cannot providefirm conclusions regarding the effects of development interventions.Rather, RCA insights were used to frame causal narratives grounded in the experience and perceptionsof people living in the area rather than as a primary source of qualitative data for the analysisas a whole. This approach provided insights into how people themselves viewed change,removed from the usual externally imposed program or sectoral lenses. It was able to report on howsome people in the area thought change came about and their view of the interplay of outside interventionsand local knowledge, and the role of government, development partners and private sector,as well as their perceptions of the varying significance of different factors. It was able to distinguishdifferences between intended benefits and actual benefits and differences between stated change andactual behavioral change.A nonprescriptive method such as the RCA proved an effective tool for framing narratives andillustrating the views and experiences of local people. It also provided an effective means of identifyingalternative avenues of investigation and inquiry, causing us, as researchers, to question previouslyassumed truths or realities and return to the data with a more critical appraisal. However, itdoes need to be combined more systematically with other sources of data to generate more rigorousand generalizable findings.Managing interdisciplinary teams. The management of the research process itself needs to be strong inorder to ensure integration (both coherence and continuity) and adherence to a clear program timetable.The team involved must be able to function as a whole, with individual specialisms harnessedto address a well-defined set of issues and produce a defined set of outputs.Throughout the data collection and compilation process, several analytical ‘‘check-ins’’ wereconducted, particularly in sharing the results of the documentary review to frame the data collectionactivities of the RCA study in an effort to encourage interdisciplinary exchange among researchers.12 American Journal of EvaluationDownloaded from aje.sagepub.com at Naresuan University on January 23, 2016However, the extensive period of documentary review was carried out for the most part on a sectoralor disciplinary basis, with individual members of the team tending to work on their own area ofexpertise and sector rather than in a fully integrated manner. Furthermore, the description of themajor findings from the documentary review tended to precede the analysis rather than for the twoto proceed together. The analysis was, in effect, largely undertaken as a later phase in the project.This led to a number of weaknesses in the study that should be considered by future researchers.Effective working relationships, coordination, and opportunities to exchange and analyze data ina sequential manner throughout the study are essential to success. Future researchers should considerinvestigating organizational tools that can be used to improve this critical process, including theright use of general and technical steering committees, or holding regular plenary sessions and/orrefocusing workshops at regular intervals with key stakeholder groups.
การแปล กรุณารอสักครู่..