4.3.6 Enhanced quality assurance of examination process
Currently the BAA Assessment Centre observation visits focus on examination system and procedure compliance rather than the validity of assessment decisions.
It is proposed that a quality enhancement to this process be that an external expert involved in the development of Occupational Standards or IOC and/or a member of BNPQS joins BAA Accreditation Officers on the Assessment Centre observation visit. The external expert and/or BNPQS representative will observe and report on the quality and validity of the assessment decisions being made. The report will use the same criteria as those being used by the Assessment Centre examination committee in verifying examiner decisions and will contribute towards the observation visit report provided to the Assessment Centres on the outcomes of an observation. The proposed modification is identified under the monitoring component in Diagram Two – Quality Assurance System v.1.0.
Assessment Centres are expected to use this feedback to make adjustment to their examination process as necessary and the information will also form part of their self-assessment evidence when their three yearly re-certification cycle is due.
The inclusion of the external assessment focus in the observation process across Assessment Centres provides TPQI with information about the level of national consistency across examiner decisions. The process also provides direct feedback to BNPQS on the quality of the assessment tools and any issues with the occupational standards, either for later reviews or to address immediately.
NZQA also supports the Assessment Centre renewal and audit of accreditation process modifications BAA are considering with a view to engendering a ‘high trust, high accountability’ philosophy by:
• identifying only those key areas which need compliance measures
• differentiating between quality Assessment Centres and others and only imposing ‘burdensome’ measures on those with complaints or accountability issues
• modifying observer visit reports so that there is a more rapid feedback loop to Assessment Centres on the outcomes of an observation.
4.3.7 Quality assurance of Assessment Centre developed assessment tools
BAA staff noted that some Assessment Centre examiners were using their own assessment tools rather than those centrally designed and quality assured by TPQI.
The use of non-quality assured assessment tools is of concern as the use of centrally designed tools, is one of the significant factors in assuring the consistency of examiner decisions within and across assessment centres.
However the long-term reliance on solely centrally designed materials is not sustainable by TPQI and, at some point, there will also be questions around the assurance of authenticity of candidate responses with the limited pool of available assessment tasks.
The proposed adjustment to the Assessment Centre observation visit process can also have a role in the external moderation (quality assurance) of such materials.
Two quality assurance processes are proposed. One if the materials are to be used solely by a specific assessment centre and the other if TPQI wishes to acquire and publish them.
The proposed processes are described in Diagram Five – Quality Assurance of Assessment Task developed by Assessment Centres v.1.0.