1.3.
Why a system of indicators?
The use of a theoretical frame of reference or framework underpins the structure and selection of the ‘system of indicators’ inasmuch as it offers a justification of the choices made – an explanation, that is, of the connections between the features described by the indicators and the procedures and techniques of data collection.
A system of indicators is not, or not completely, able to give definite answers regarding the direction and strength of the relationships between different aspects, but it offers the opportunity to explore these relationships and work out a complete picture of the effects and possible causes. As Fitz-Gibbon and Tymms (2002) put it, “An indicator system is only a step along the way to trying to understand what works, and how schooling can be improved. Consequently, some of our indicator systems include process variables such as descriptions of methods of teaching and learning […]. Process indicators serve to generate hypotheses and most importantly, they stimulate discussion of teaching methods among staff in schools and as such are valuable. The important problems in trying to attribute cause and effect must, however, be continuously emphasised”.
Amongst the variety of the indicators analyzed in the several countries considered, three main key points were focused.
The first point is that in most of the more developed education systems the frameworks adopted are structured matching a system perspective with a school perspective, often integrating quantitative and qualitative techniques to collect data.
The second point is related to the evaluation’s objectives, which are findable on a continuum having his poles in the two different concepts of ‘develop’ and ‘control’: the ‘develop’ approach focuses on the role that evaluation can play in changing the education system, in a perspective of operators’ involvement and constituting a real organizational learning; while the ‘control’ approach refers to the bureaucratic controls, that is the conformity with procedures and laws.
The third point deals with the definitions of the aspects investigated. While there’s a wide convergence about dimensions and areas of research, more difficulties are to be faced up regarding the definition of those elements (indicators, aspects, variables, etc.) considered able to describe
those dimensions/areas. The most common strategy to avoid this obstacle is to publish the conceptual frameworks, in order to make clear aspects to be investigated and procedures chosen for measuring school characteristics.
2.
Reports on the education systems of some European and non-European countries
The education systems of several European and non-European countries were studied along with their respective reports. The aim was to understand how information is presented, which framework is defined and used, and what aspects were chosen to describe, present or report to the community the state of the education system.
In some cases the reports are lengthy (Germany’s, for example), while others present only brief summaries (as in the case of Finland). Many of the reports refer to a set of indicators, but only a few of these are organized on the basis of a clearly illustrated framework (for example the reports of New Zealand and Spain).
The data are almost always accompanied by graphs that are easy to read and understand, and are often supplemented with commentaries, international comparisons and information regarding more than just the education system.
The frequency of publication varies among the different countries, e.g., France publishes more annual reports, while in Spain they are biannual. Some countries have been systematically producing reports based on indicators for many years, for others this is a recent achievement, and still others (like Finland) have thus far produced only a single publication.
The following are brief descriptions of the reports examined up to now.
In Australia, in 2000 the Ministerial Council on Education, Employment, Training and Youth Affairs (MCEETYA) agreed to the development of a set of national key performance measures to ensure that the key indicators of the outcomes of schooling in Australia would be publicly available. Starting from 2000 the National report on schooling in Australia has been available yearly on the website http://www.mceecdya.edu.au/mceecdya/anr/. In 2008 the report provides informations about schools and student, teachers and teaching, resourcing, equity, students attendance, students national and international assessments.
In Denmark, an annual publication presents a series of key figures in tables and graphs showing development trends, and illustrates these using Danish figures complemented by equivalent figures from selected countries. Key figures for education in Denmark are updated continuously, and in some cases previously published data is adjusted. Facts and Figures 2007 gives an overview of the Danish education system and a quantitative description of
developments in most of the various fields of education. Most of the figures presented in the publication are also shown on the web site of the Danish Ministry of Education (www.uvm.dk).
In England, an annual report presents evidence from inspections and regulatory visits undertaken by the Office for Standards in Education, Children’s Services and Skills (Ofsted). Evidence is taken from inspection activity across the full range of Ofsted’s remit, including childcare, children’s social care, local authority services for children and provision for education and skills in schools, colleges and adult learning. The report draws upon the findings both of routine inspection visits and of the focused survey inspections through which inspectors collect more detailed evidence about subjects and aspects of provision in social care, education and skills.
In Finland, Statistics Finland, the Finnish public authority specifically established for statistics, produces the vast majority of Finnish official statistics, including the field of education. The publication Education in Finland 1999 provids basic information on the regular education system and other forms of Finnish education, such as enrolment, educational staff, educational expenditure, graduation, level of achievement, transition from school to work and social outcomes.
In France, the Ministère de l'Éducation Nationale every year publishes L’état de l’École de la maternelle à l’enseignement supérieur ( available also in English), a brief analysis of the main features of the French education system. This is a collection of 30 indicators concerning costs, activities and results of the school system, going from nursery school right up to university. In addition, the Indicateurs generaux. Aide au diagnostic, au pilotage des académies et à la contractualisation has been published since 2000, as a means of monitoring internal variations within the French school system. These indicators supply statistical information on the functions and services of the education system of each of the territorial areas that make up the French scholastic system (académie). Their use facilitates the analysis of situations in the académie and contributes to the development and monitoring of territorial action plans.
In Germany, the report Education in Germany 2008 presents the second comprehensive empirical account of the German education system. The volume analyses all stages of education ranging from early childhood education and care through the general education system, vocational education and training and higher education right up to the continuing education of adults. The report is based on a project which was funded by the Standing Conference of the Ministers of Education and Cultural Affairs of the Länder in the Federal Republic of Germany and the Federal Ministry of Education and Research. The report, the
In the Netherlands, the Dutch Ministry of Education, Culture and Science presents yearly the statistical publication Key figures regarding the results and status of its policy areas in education, culture and science in the last five years. The publication Key figures 2004-2008 contains information for each educational sector regarding participants, institutions, staff, outcomes and expenditures, pre-school and early childhood education, early school-leaving, Lifelong Learning, the labour market for teaching staff and Dutch education in the international perspective.
In New Zealand, the website Education Counts has been developed to simplify access to quantitative education information for Ministry of Education clients. Education Counts has published a set of education sector indicators (http://www.educationcounts.govt.nz/indicators). In order to be more than an ad hoc collection of statistics, an explicit framework for education indicators was built. There are six domains to the indicator framework: education and learning outcomes, effective teaching, student participation, family and community engagement in education, quality education providers, and resources.
In Spain, the Instituto de Evaluacion, directly subordinate to the Ministerio de Educación, has the task of putting together the “State system of education indicators”. Beginning in 2000, indicators have been updated every year. Since 2006, one group of indicators, called priority indicators, have been updated annually, while the rest are updated every two years. In the complete edition of 2006 and in the successive 2009 edition, 38 indicators were published, 15 of which were priority indicators. The dimensions according to which the indicators are grouped are: context, resources, schooling, processes and results. The reports and the map of indicators are available on the site of the Instituto de Evaluacion (http://www.institutodeevaluacion.mec.es/publicaciones/index.php).
In Sweden, the publication Descrip