1. Introduction
Large-scale, lengthy, multi-situated programmes with multiple
actors pose specific challenges for their evaluators. This is the case
of the National Reading Plan (NRP) – a public policy initiative
whose aim is to increase literacy levels and reading habits among
the Portuguese population. Since it began, the NRP has always
provided for its own external evaluation, which the authors of the
present article undertook throughout the five years of the first
phase of the programme (2006–2011). The evaluation system that
was designed for the NRP made it possible to continuously and
systematically monitor the programme’s development and implementation.
The chosen methodological strategies were essentially
based on a mixed-method approach. This article presents the
methodological operations that were conducted and the contributions
obtained by mixing methods. Particular emphasis is placed
on the potentials of mixed methods for evaluating complex and
wide-ranging programmes.
Current evaluation theory and practice tend to be pluralist,
encompassing a variety of perspectives, analytical dimensions,
methods, data and social actors. There is an understanding of the
need to apply multiple forms of knowledge in each evaluation,
inasmuch as social problems are becoming ever more complex.Various authors have felt that a considered integration of data
sources, methodological procedures and analytical strategies is
decisive to the ability to generate more sustained and insightful
evaluation (Greene, Benjamin, & Goodyear, 2001; Stern, 2005).
The mixed-method approach, which has been widely employed
and discussed in evaluation practice, forms part of this trend.
Mixed methods have progressively become generalised in the
evaluation domain, with a tendency to a more systematic use since
the late 1980s (Creswell & Clark, 2006). Large numbers of recent
evaluation studies on a variety of programmes have included the
use of mixed methods (e.g. Berry et al., 2009; Celik, Abma, Klinge, &
Widdershoven, 2011; Giannakaki, 2005; Grammatikopoulos,
Zachopoulou, Tsangaridou, Liukkonen, & Pickup, 2008; Luo &
Dappen, 2005; Protheroe, Bower, & Chew-Graham, 2007; Siraj-
Blatchford, Sammons, Taggart, Sylva, & Melhuish, 2006; Townsend,
Floersch, & Findling, 2010).
The mixed-method approach offers the evaluation practice
many forms of potential. Greene, Caracelli, and Graham (1989)
developed a typology of purposes for mixed-method evaluation
designs, which includes triangulation, complementarity, development,
initiation, and expansion. Subsequently, Greene et al. (2001)
specified that the better understanding of the social phenomena,
which is the overall purpose for mixing methods, means enhanced
validity and credibility of inferences, greater comprehensiveness of
findings, more insightful understandings, and increased value
consciousness and diversity.
A central idea shared by researchers and specialists in the
evaluation field is that combining methods provides a broader,
deeper knowledge than any one method on its own. Quantitativeresearch leads to a generalisation of results and supplies relatively
standardised information. Qualitative research is more likely to
capture the circumstances and settings in which people talk,
highlighting cultural and contextual dimensions. Mixed-method
research makes it possible to capitalise the best of each component
and take advantage of each method’s strong points (Creswell &
Clark, 2006; Protheroe et al., 2007; Stufflebeam, 2001).
Researchers have also emphasised that mixed-method research
incentivates a widening and confrontation of social actors’ points
of view and allows a verification of the convergence or discrepancy
between the data obtained by the different methods (Creswell &
Clark, 2006). The combination of quantitative and qualitative
methods in the same study helps to understand more comprehensively
the phenomenon under analysis and to increase the validity
of the research or evaluation process (Olsen, 2004; Teddlie &
Tashakkori, 2003).
Mixed methods have recently been the object of important
methodological reviews (Bryman, 2012; Small, 2011; Tashakkori
& Teddlie, 2010). These reviews have analysed the debates about
mixed methods, and the ways in which they have been used in
both social scientific research and evaluation studies. In mixed
data-collection studies, different methods are employed for
different reasons, but also in varied sequences within the overall
research process and with different levels of nesting of the
multiple data sources. In addition to mixed data-collection
studies, which combine different kinds of data, Small (2011)
distinguishes mixed data-analysis studies, which combine
different analytical strategies. This type of study also combines
methods in the analytical stage, applying quantitative techniques
to qualitative data, or vice versa, or merging different analytical
approaches or techniques in a single study (Rihoux & Ragin, 2009).
The ways of combining different methods are varied, as Bryman
(2006, 2012) has shown in his systematic research on mixed
methods. He identifies and analyses different ways of combining
quantitative and qualitative research: triangulation, offset,
completeness, process, different research questions, explanation,
unexpected results, instrument development, sampling, credibility,
context, illustration, utility, confirm and discover, diversity of
views, and enhancement.
Like empirical research in general, evaluation studies have been
combining methods in the most diverse and sometimes innovative
ways, as well as from multiple perspectives. In the evaluation of
complex programmes, the mixed-method approach has for
example been applied within the overall framework of theorydriven
evaluations. To Chen (1990, 2006), much of a programme’s
potential outcomes should be theory-based, extracted from
existing social-science theory and the programme models that
indicate how each programme is supposed to work. Chen proposes
that we resort to theory and prior knowledge in order to construct
models of the programme-implementation system, in such a way
as to produce evaluations that supply more information about how
to achieve the desired effects. He underlines the articulation of
theory – which should support and provide the grounds for the
design, implementation and evaluation of programmes – with the
use of mixed methods.
1. บทนำโครงการขนาดใหญ่ ยาว หลายแห่งหลายนักแสดงก่อให้เกิดความท้าทายเฉพาะสำหรับ evaluators ความ ในกรณีนี้ของชาติอ่านแผน (NRP) – การริเริ่มนโยบายสาธารณะจุดมุ่งหมายจะเพิ่มระดับการวัดและอ่านพฤติกรรมระหว่างประชากรภาษาโปรตุเกส ตั้งแต่เริ่มต้น NRP ได้เสมอสำหรับการประเมินภายนอกของตัวเอง ซึ่งผู้เขียนของการบทความนำเสนอ undertook ตลอด 3 ปีแรกระยะของโครงการ (2006-2011) ระบบการประเมินที่ถูกออกแบบมาสำหรับ NRP ที่ทำไปอย่างต่อเนื่อง และระบบตรวจสอบของโครงการพัฒนาและการใช้งานกลยุทธ์ methodological ท่านถูกหลักใช้วิธีผสมวิธีการ บทความนี้นำเสนอการการดำเนินงาน methodological ที่ได้ดำเนินการและการจัดสรรได้ โดยวิธีการผสม เน้นเฉพาะอยู่ในศักยภาพของวิธีการผสมการประเมินซับซ้อน และโปรแกรมเกมทฤษฎีการประเมินปัจจุบันมัก pluralistครอบคลุมความหลากหลายของมุมมอง มิติวิเคราะห์วิธี ข้อมูล และสังคมนักแสดง มีความเข้าใจเกี่ยวกับการต้องใช้ความรู้ในการประเมินแต่ละ หลายรูปแบบinasmuch เป็นปัญหาสังคมจะกลายเป็นซับซ้อนเคย ต่าง ๆ ผู้เขียนรู้สึกว่าที่รวมเป็นข้อมูลแหล่งที่มา ตอน methodological และวิเคราะห์กลยุทธ์เป็นdecisive to the ability to generate more sustained and insightfulevaluation (Greene, Benjamin, & Goodyear, 2001; Stern, 2005).The mixed-method approach, which has been widely employedand discussed in evaluation practice, forms part of this trend.Mixed methods have progressively become generalised in theevaluation domain, with a tendency to a more systematic use sincethe late 1980s (Creswell & Clark, 2006). Large numbers of recentevaluation studies on a variety of programmes have included theuse of mixed methods (e.g. Berry et al., 2009; Celik, Abma, Klinge, &Widdershoven, 2011; Giannakaki, 2005; Grammatikopoulos,Zachopoulou, Tsangaridou, Liukkonen, & Pickup, 2008; Luo &Dappen, 2005; Protheroe, Bower, & Chew-Graham, 2007; Siraj-Blatchford, Sammons, Taggart, Sylva, & Melhuish, 2006; Townsend,Floersch, & Findling, 2010).The mixed-method approach offers the evaluation practicemany forms of potential. Greene, Caracelli, and Graham (1989)developed a typology of purposes for mixed-method evaluationdesigns, which includes triangulation, complementarity, development,initiation, and expansion. Subsequently, Greene et al. (2001)specified that the better understanding of the social phenomena,which is the overall purpose for mixing methods, means enhancedvalidity and credibility of inferences, greater comprehensiveness offindings, more insightful understandings, and increased valueconsciousness and diversity.A central idea shared by researchers and specialists in theevaluation field is that combining methods provides a broader,deeper knowledge than any one method on its own. Quantitativeresearch leads to a generalisation of results and supplies relativelystandardised information. Qualitative research is more likely tocapture the circumstances and settings in which people talk,highlighting cultural and contextual dimensions. Mixed-methodresearch makes it possible to capitalise the best of each componentand take advantage of each method’s strong points (Creswell &Clark, 2006; Protheroe et al., 2007; Stufflebeam, 2001).Researchers have also emphasised that mixed-method researchincentivates a widening and confrontation of social actors’ pointsof view and allows a verification of the convergence or discrepancybetween the data obtained by the different methods (Creswell &Clark, 2006). The combination of quantitative and qualitativemethods in the same study helps to understand more comprehensivelythe phenomenon under analysis and to increase the validityof the research or evaluation process (Olsen, 2004; Teddlie &Tashakkori, 2003).Mixed methods have recently been the object of importantmethodological reviews (Bryman, 2012; Small, 2011; Tashakkori& Teddlie, 2010). These reviews have analysed the debates aboutmixed methods, and the ways in which they have been used inboth social scientific research and evaluation studies. In mixeddata-collection studies, different methods are employed fordifferent reasons, but also in varied sequences within the overallresearch process and with different levels of nesting of themultiple data sources. In addition to mixed data-collectionstudies, which combine different kinds of data, Small (2011)distinguishes mixed data-analysis studies, which combinedifferent analytical strategies. This type of study also combinesmethods in the analytical stage, applying quantitative techniquesto qualitative data, or vice versa, or merging different analyticalapproaches or techniques in a single study (Rihoux & Ragin, 2009).The ways of combining different methods are varied, as Bryman(2006, 2012) has shown in his systematic research on mixedmethods. He identifies and analyses different ways of combiningquantitative and qualitative research: triangulation, offset,completeness, process, different research questions, explanation,unexpected results, instrument development, sampling, credibility,context, illustration, utility, confirm and discover, diversity ofviews, and enhancement.Like empirical research in general, evaluation studies have beencombining methods in the most diverse and sometimes innovativeways, as well as from multiple perspectives. In the evaluation ofcomplex programmes, the mixed-method approach has forexample been applied within the overall framework of theorydrivenevaluations. To Chen (1990, 2006), much of a programme’spotential outcomes should be theory-based, extracted fromexisting social-science theory and the programme models thatindicate how each programme is supposed to work. Chen proposesthat we resort to theory and prior knowledge in order to constructmodels of the programme-implementation system, in such a wayas to produce evaluations that supply more information about howto achieve the desired effects. He underlines the articulation oftheory – which should support and provide the grounds for thedesign, implementation and evaluation of programmes – with theuse of mixed methods.
การแปล กรุณารอสักครู่..
