Part 1: Introduction

New Zealand Qualifications Authority: Assuring the consistency and quality of internal assessment for NCEA.

1.1
In this Part, we set out:

About the New Zealand Qualifications Authority

1.2
NZQA is a Crown entity. It is governed by an independent board appointed by the Minister of Education.

1.3
NZQA's revenue for 2010/11 was $79 million. This comprised $40 million of Crown revenue and $39 million of third-party revenue from fees and charges, such as examination fees. Of the Crown funding, $26 million was for the delivery of secondary school assessments for NCEA. Supporting internal assessment costs $13.4 million. NZQA employs about 400 staff, of whom 86 full-time equivalents work on supporting internal assessment.

1.4
Under section 246A of the Education Act 1989, the functions of NZQA include:

  • overseeing the setting of standards for qualifications in secondary schools;
  • monitoring, reviewing, and advising the Minister of Education on, the standards for qualifications in schools; and
  • ensuring that there are mechanisms to guarantee that schools have assessment and moderation procedures that are fair, equitable, and consistent and comply with the appropriate standards for qualifications.

1.5
For NCEA, NZQA is responsible for:

  • national administration of external assessments (examinations), including producing, marking, and returning examination booklets to secondary school students (students); and
  • external moderation of students' work that contributes to internal assessment for NCEA. This involves working with secondary school teachers (teachers) to continually improve the quality of teachers' assessments of students' work.

The Ministry of Education's role

1.6
The Ministry of Education (the Ministry) sets the policies for teaching and learning in schools (including the school curriculum). With NCEA, the Ministry prepares the achievement standards (see Appendix 1 for an example) and produces assessment tasks with sample answers.

About the National Certificate of Educational Achievement

1.7
Between 2002 and 2004, NCEA was introduced as the main secondary school qualification.1 It replaced School Certificate, Sixth Form Certificate, University Entrance, and University Bursary qualifications and examinations. By 2011, NCEA had been fully operational for eight years in New Zealand, Niue, and the Cook Islands.2

1.8
Students usually study towards NCEA from Years 11 to 13 (previously fifth form to seventh form). Most of these students are 15 to 18 years old, but some younger students and some adult students also study NCEA subjects. Students usually study five or six subjects each year. For each subject, students are assessed against a number of standards. Appendix 1 sets out an example of a Level 1 standard for English.

1.9
When a student achieves a standard, they receive a number of credits. When they have the required number of credits, they gain an NCEA certificate (see Figure 1). There are three levels of NCEA certificate. Students usually gain Level 1 in Year 11, Level 2 in Year 12, and Level 3 in Year 13.

Figure 1
Number of National Certificates of Educational Achievement awarded to Year 11 to Year 13 students in 2011

Figure 1 - Number of National Certificates of Educational Achievement awarded to Year 11 to Year 13 students in 2011.

Source: New Zealand Qualifications Authority.

Standards

1.10
There are two types of assessment standards that can count towards NCEA: unit standards and achievement standards.

1.11
Achievement standards are based on The New Zealand Curriculum (the Curriculum),3 and are created by the Ministry. Achievement standards can be either internally or externally assessed. Students who achieve a standard are awarded "achieved", ‘"merit", or "excellence" according to their performance. If a student does not achieve a standard, their work is graded as "not achieved".

1.12
Unit standards mostly count towards technical and vocational training, and are created by either NZQA or an Industry Training Organisation. Unit standards are always internally assessed. Students who achieve a unit standard get awarded "achieved", and some unit standards also include "merit" and "excellence". If a student does not achieve a unit standard, their work is graded as "not achieved".

1.13
The Ministry and NZQA are aligning achievement standards with the Curriculum, which was introduced in 2007. Aligned achievement standards have been developed specifically in line with the Curriculum. The alignment of the standards to the Curriculum is a phased process: assessment for these newly aligned standards started in 2011 and will be fully implemented by 2014. The old non-aligned standards are valid for internal assessment only, and for a limited time. The new achievement standards will replace any unit standards that are linked to the Curriculum.

Internal and external assessment

1.14
Students work towards their NCEA qualifications by having their work internally and externally assessed. Internal assessment refers to tasks that are set and marked by teachers within the secondary school. External assessment refers to national examinations that are set and marked by NZQA.

1.15
The proportion of internal and external assessment for each subject varies. For example, for a student mostly studying subjects such as maths and science, about one-third of their credits for a subject are likely to be internally assessed, with the remaining two-thirds gained through external examinations. For a student studying mainly English and the social sciences (geography, history, and economics), about half of the credits will be internally assessed and half assessed through external examinations.

1.16
The mix of internal and external assessment will vary for each student, depending on the subjects they are studying. Throughout the country in recent years, there has been an almost equal mix of internal and external assessment for achievement standards. When combining achievement standards and unit standards, roughly 70% of all credits are gained through internal assessment.

1.17
It is easier to ensure that external examinations are nationally consistent because everyone enrolled in a particular subject throughout the country is assessed at the same time against the same task under similar conditions.

1.18
There is a lot more flexibility in how internal assessment is carried out. Secondary schools can design assessment tasks (related to the Curriculum) to measure students' skills and knowledge.4 Examples of tasks for internal assessment include delivering a speech or conducting a science experiment.

1.19
Given this permitted flexibility, it is important that all the assessment tasks that secondary schools set are appropriate (that is, the tasks are in line with the achievement standards and allow students to demonstrate their ability). It is also important that there is consistency in how the quality of students' work is assessed between schools and between subjects.

The purpose of our audit

1.20
The work that NZQA does is important for the credibility of NCEA. It is vital that students, parents and care givers, employers, and tertiary education providers can be sure that an NCEA qualification gained at one school is equivalent to an NCEA qualification gained at another school.

1.21
We carried out a performance audit of NZQA. Our aim was to provide assurance to Parliament about whether NZQA is making sure that internal assessment for NCEA (that is, the tasks that are internally set and marked by teachers) is consistent.

1.22
The introduction of NCEA as the main secondary school qualification – in particular, the internal assessment component – has been controversial. Criticism has centred on the potential for schools to set easy assessment tasks and mark tasks leniently to ensure that students gain their qualifications, which would improve the school's reputation. One aim of our audit was to provide assurance to Parliament, students, parents and care givers, employers, and tertiary education providers about whether NZQA has procedures to ensure that there is no such manipulation of internal assessment.

How we did our audit

1.23
We examined how well NZQA:

  • ensures that teachers and schools know the requirements for assessing students against internally assessed standards;
  • checks the appropriateness and consistency of schools' internal assessment; and
  • works with schools to address any concerns about the quality of a school's internal assessment.

1.24
We did this by:

  • interviewing NZQA staff and its contracted part-time moderators (see paragraph 3.8);
  • reviewing NZQA documents, including 200 moderation reports (see paragraphs 3.16-3.23);
  • attending meetings between NZQA staff and teachers at their moderators' meetings;
  • visiting NZQA's national workshop, where NZQA staff were preparing exemplars of student work (see paragraphs 2.9-2.14);
  • attending two Best Practice Workshops run by NZQA staff for teachers;
  • visiting five schools, where we interviewed various people (see paragraph 1.26);
  • meeting with representatives of the Secondary Principals' Association of New Zealand and the New Zealand Post Primary Teachers' Association; and
  • inviting all secondary school teachers to complete an online survey.

1.25
Our survey, which was carried out in November 2011, asked teachers about the quality of NZQA's resources and moderation activities (such as moderation reports). We emailed all secondary schools a link to an electronic survey, asking that it be sent out to all of the school's teachers. We received 1780 responses (a response rate of around 11.5% of all the full-time-equivalent secondary school teachers funded by the Ministry in 2010). The teachers who responded to the survey were from large and small schools throughout the country, covering the full range of decile levels5 and all subject areas. Appendix 2 sets out our survey questions.

1.26
The five schools that we visited included a mix of decile levels and urban and rural locations. They included single-sex and co-educational schools, and state and integrated schools.6 We visited these schools to get first-hand examples of how NZQA interacts with and supports schools. At most of these schools, we met with the Chairperson of the Board of Trustees and the student representative on the Board of Trustees. At all five schools, we met with the school principal and the Principal's Nominee.7

What we did not cover

1.27
In carrying out our audit, we did not look at the setting, administration, or marking of external examinations, or compare NCEA with other qualifications.

1.28
Because they are not part of NZQA's role, we did not look at:

  • quality assurance for unit standards that are used in secondary schools and administered by Industry Training Organisations (in 2010, 17% of unit standards used in schools were administered by Industry Training Organisations); or
  • the quality of teaching.

1.29
We did not examine whether NCEA is an appropriate system for assessing students' achievement. The Auditor-General's mandate does not extend to policy matters.

1: Some secondary schools also make other qualifications available to their students. These include the International Baccalaureate and the Cambridge University International Examinations.

2: The Cook Islands and Niue have constitutional links with New Zealand and have arrangements that align their secondary school assessment systems with New Zealand's. In our report, data from NZQA include the Cook Islands (where seven schools use New Zealand's NCEA qualifications) and Niue (where one school uses New Zealand's NCEA qualifications). Our survey of teachers did not include teachers from the Cook Islands schools or Niue High School.

3: The New Zealand Curriculum is a statement of official policy on teaching and learning in English-medium schools (in New Zealand, the Cook Islands, and Niue). Its main function is to set the direction for student learning and to provide guidance for each school in designing and reviewing its own curriculum.

4: Schools can also buy ready-made assessment tasks (see paragraph 2.16).

5: A school's decile level (1 to 10) is derived from Census meshblock data on the socio-economic status of students attending the school. There are equal numbers of schools in each decile, and the ratings are reviewed after each five-yearly Census. For more information, see "Deciles" on the Ministry of Education's website, www.minedu.govt.nz.

6: Integrated schools are former private schools that are now "integrated" into the state system under the Private Schools Conditional Integration Act 1975 "on a basis which will preserve and safeguard the special character of the education provided by them".

7: A Principal's Nominee is appointed by the principal to administer NCEA within the school and to communicate NCEA-related information to other school staff and NZQA. This staff member is the main point of contact for NZQA at the school (see paragraphs 2.26-2.32).

page top