6. Lessons learned
The combination of quantitative and qualitative methods made
important contributions to the evaluation of the impacts of the
NRP. The integration of mixed methods approach is useful to
evaluators and researchers dealing with the evaluation of wideranging
and complex programmes. Such methodological strategy
can help them to achieve the requirements for a successful
evaluation.
One of the main advantages of combining methods is
information complementarity. Gathering standardised quantitative
information allows generalisation and leads to the analysis
of social patterns. Collecting qualitative information highlights
cultural and contextual dimensions and reveals meanings
that actors attribute to situations, actions and processes.
The combination of these different types of information
provides a better understanding and interpretation of programs’
impacts.
Mixing methods makes it easier to construct an evolving
perspective of programs’ impacts. In the case of the NRP, since the
programme’s duration was a long one, it was necessary to obtain
A.F. Costa et al. / Evaluation and Program Planning 39 (2013) 1–9 7
evaluation elements regarding the changes that gradually took
place over the years, thereby identifying and explaining trends.
Quantitative and qualitative operations help to identify and
interpret both the immediate impacts of projects and those which
only become evident after a number of years. The regular return of
evaluation results can be useful to the ongoing process of
improving the programmes.
The mixed methods approach favours the contextualised
analysis of the programs’ impacts. In evaluating impacts of the
NRP we considered both the national and the local scales. The
combination of quantitative and qualitative methodological
operations permits an integrated use of information on these
different scales. This in turn makes it possible to not only outline a
general framework of the programs’ impacts, but also identity and
interpret specificities in the light of the social and cultural context
in which they develop.
The process of mapping the diversity of projects, situations,
actors and points of view also benefits from the combination of
methods. In the evaluation of complex programmes, such as the
NRP, it is necessary to cover vast ranges of projects and initiatives,
which in some cases are gradually launched. It is also important to
gather information from and about different social actors, and to
look for different opinions and points of view. The integration of
information from both quantitative and qualitative methodological
operations ensures plurality on these levels.
The combination of methods, which produces the contributions
we have described, facilitates the corroboration of results. This was
particularly true in terms of the collection and analysis of
information on the NRP’s impacts on practices, attitudes,
representations, knowledge, skills and processes. Such methodological
approach enhances the actors’ reflexivity and enables the
identification of best practices.