1. What impact do different school inspection regimes in Europe intend to have on
schools?
2. How do school inspections in Europe intend to contribute to the improvement of
schools?
Educ Asse Eval Acc
2 A framework for describing Inspectorates of Education in Europe
We start our analysis by describing a number of characteristics to differentiate
between different types of school inspections. These characteristics help us identify
the specific aspects of school inspections that are expected to be effective in promoting
school improvement. Several authors propose categories that may be used to
describe relevant characteristics of school inspections.
Van Bruggen (2010) for example distinguishes 51 characteristics to summarize the
features of Inspectorates in 18 European countries. He describes the characteristics of
inspection processes, the characteristics of the inspection report and the follow-up, the
characteristics of the observation of teaching and learning as one element of the
inspection. He analyses the characteristics of the system of inspection in a more general
way, including specific types of inspections and inspection products such as the regime
for failing schools or the publication of good practices. Other studies (such as Eurydice
2004 and Whitby 2010) use a similar taxonomy of characteristics, but add the consequences
of these evaluations and the ways in which evaluation results are used (e.g. to
monitor the education system and/or recommendations are provided to schools).
As this study focuses on the impact of school inspections, the characteristics of and
differences between school inspections that have an impact on schools must be described
in our model. The literature review of Klerks (submitted) provides a systematic
summary of the inspection characteristics that have been found to be effective in
empirical studies. Her review shows that no specific characteristic of school inspections
in itself leads to improvement, but effects arise as a result of a complex interaction
between inspection characteristics and students, teachers and the school management.
Generally, the type and frequency of school inspections (e.g. full/thematic school
inspections), the standards and thresholds used to assess and provide feedback to
schools during inspection visits and the sanctions, rewards and interventions used to
motivate schools to improve (including the public report of the Inspectorate) seem to
be the dominant aspects of school inspections affecting change in schools (Ehren and
Visscher 2006; De Wolf and Janssens 2007). Standards are expected to influence
actions in schools, particularly when schools face consequences for failing to meet
these standards and thresholds. School officials can be expected to select the improvement
actions that they perceive to have the highest yield, given their planning
horizon, budget and appetite for risk. The feedback provided by school inspectors on
the school’s performance in relation to the standards, working in conjunction with
sanctions and/or rewards, should support and motivate improvement. This set of
relationships between inspection inputs and change in schools provides, in theory, an
explanation of the way in which inspectorates hope to influence school improvement.
2.1 Types and frequency of inspection visits
Inspectorates of Education choose different methods to collect information on
schools. These methods may be part of regular cycles of full inspections of all schools
or of differentiated/proportional inspections of schools. According to Whitby (2010),
in most countries the frequency of external inspections depends on an analysis of
documents (including self-evaluation documentation) that the school submits to the
external Inspectorate, and schools are then visited ‘proportional to the need’. Within
Educ Asse Eval Acc
these different models, a wide variety of evaluation methods are used to collect
information, including desk research, visits to schools, interviews, questionnaires,
classroom observation and analysis of documents produced by the school such as the
results of self-evaluations.
2.2 Standards and thresholds
Standards present the details of what is expected of schools; they create boundaries or
domains for attention with respect to educational quality. The standards and criteria
may, according to Eurydice (2004), include educational responsibilities such as the
teaching/learning of skills and knowledge, teaching/learning about appropriate patterns
of social behaviour and personal development, and other administrative responsibilities
such as the management of resources, external relations and partnerships.
According to Whitby (2010), standards may emphasize input expectations (such as
opportunity to learn, class size, teacher training, etc.) and/or output standards as
measured by the performance of schools. In addition, the “quality frameworks” which
have been recently produced by many German lander education systems to guide
school inspection also distinguish context and process standards (Kotthoff and
Böttcher 2010). Context standards are used to take into account the school-specific
location, history, identity and student population, whereas process standards emphasize
compliance with legislation or principles and practices of good education.
The type of standards and thresholds developed will invariably influence which
improvement actions schools take and how effective these are in improving student
achievement. From this perspective, Scheerens et al. (2005) describe the process
indicators in the (Dutch) inspection framework and use educational effectiveness
research to evaluate their likely positive association with learning outcomes. Indictors
on the quality of learning and instruction (such as learning time, and clear and
structured teaching) have received empirical support in the literature as being associated
with relatively high performance.
The thresholds used to identify schools that are to be judged as failing, overall, to
meet the standards should motivate schools to alter their behaviour. Hanushek and
Raymond (2002) for example describe how schools that have scores close to a performance
target change their behaviour more than schools further away from that target.
2.3 Sanctions, rewards and interventions
Schools that are evaluated as failing often face consequences, such as sanctions or
interventions (Van Bruggen 2010). Sanctions may include fines or closure of a school.
School inspectors may intervene in such schools by means of increased monitoring of
specific improvement plans which the schools are required to implement to address their
weak points. Consequences of school inspections can also include rewards for highperforming
schools which may receive awards or financial bonuses.
Several studies suggest that sanctions and rewards have a positive effect on
educational quality in schools. The operating assumption in these studies is that
schools work harder to perform well when something valuable is to be gained or
lost; information and feedback alone are seen as insufficient to motivate schools to
perform to high standards (Malen 1999; Elmore and Fuhrman 2001; Nichols et al.
Educ Asse Eval Acc
2006). Heubert and Hauser (1999) found a significant relationship between the level
of incentives for schools and students and the extent to which the curriculum and
teaching in schools improves. Responses to inspection tend to be most focused and
effective where funding is at stake or exposure is higher, according to Matthews and
Sammons (2004). Formal sanctions like forced reconstitution of consistently low
performing schools were more likely to promote responses than just “informal
embarrassment” arising from the grading of schools and the reporting of results
publicly. In summary, the desired responses by schools are likely to be governed
by, on the one hand, a greater awareness of the importance of the standards and on the
other, sanctions and rewards which force schools to comply with the standards.
Importantly, however, high stakes (test-based) accountability systems have also
been shown to produce harmful consequences (Heubert and Hauser 1999; Koretz
2003; Stecher 2002). Sanctions and rewards may discourage desirable behaviour or
may stimulate unintended and undesirable behaviour. Kerr (1975) describes how
organisms seek information concerning what activities are rewarded, and then seek
to do (or at least pretend to do) those things, often to the virtual exclusion of activities
not rewarded. The extent to which this occurs depends, according to Kerr (1975), on
the perceived attractiveness of the rewards offered. According to Elmore and Fuhrman
(2001), schools operating under severe sanctions such as reconstitution and
probation do not appear to be making fundamental changes in their core processes.
Instead, they may place considerable emphasis entirely on the elements in the
organization of education that are assessed as part of school inspections. These quick
fix solutions often lead to more rapid improvement on the measures of the Inspectorate
than to genuine long-term improvements. Some of these schools may incorporate
structural changes but few appear to be making extensive or deep efforts to
rethink their instructional programs.
2.4 Feedback during inspection visits
Inspectors normally assess schools with respect to standards, usually defined within a
wider “quality framework”, and give feedback on the strong and weak points of the
performance of schools based on these standards. Some Inspectorates also give
schools advice on how to improve, while others are required to limit themselves to
their evaluative role and to refrain from remedial action.
In this research, feedback is expected to emerge as an important inspection
characteristic for improvement of schools. Ehren and Visscher (2008) for example
found in their case study that all schools use the feedback received from the school
inspectors to improve the