Part 2: Providing teachers and schools with resources and support

New Zealand Qualifications Authority: Assuring the consistency and quality of internal assessment for NCEA.

2.1
In this Part, we look at the resources and advice that NZQA provides for teachers and schools to ensure that internal assessment for NCEA is administered effectively. These include:

Summary

2.2
NZQA produces a range of resources to help teachers assess students' work. These resources are well regarded by teachers. Annotated exemplars of student work and Best Practice Workshops were among the most popular resources. However, teachers want greater access to Best Practice Workshops.

2.3
NZQA has made changes that should increase access to Best Practice Workshops and has improved the timely availability of annotated exemplars of student work. We encourage NZQA to continue improving the availability of these well-regarded resources.

2.4
Teachers are expected to access a wide range of documents and reports to keep up to date with changes to NCEA administration. We consider that NZQA could streamline how it communicates with, and makes important information available to, teachers to make it easier for teachers to access all the information they need.

2.5
Our survey found that 61% of the 1780 teachers who responded thought that NZQA could support teachers more to help them assess students against the applicable standards. We consider that the support for teachers has improved, and is continuing to improve. NZQA responds to feedback from teachers and uses that feedback to improve how it administers NCEA.

Resources for teachers

2.6
NZQA provides a range of resources and advice for schools and teachers on how to assess students' work. These include:

  • moderator newsletters – which contain subject-related information for teachers;
  • National Assessment Moderators' reports – an annual summary of issues about interpreting assessment tasks for each standard;8
  • Clarifications of Standards – formal statements about how to interpret and assess unit or achievement standards;
  • annotated exemplars of student work – guidance for teachers to help with assessing students' work;
  • Best Practice Workshops – face-to-face training for teachers; and
  • Optional Teacher Selected Evidence – teachers can submit students' work to NZQA for feedback on specific questions.

2.7
Our survey asked teachers to rate the usefulness of these resources in helping them to assess students' work against internally assessed standards. Figure 2 shows the results for that survey question.

Figure 2
Teachers' feedback to our survey question: "How useful are NZQA resources for helping you assess students against internally assessed NCEA standards?"

Figure 2 - Teachers' feedback to our survey question: "How useful are NZQA resources for helping you assess students against internally assessed NCEA standards?

Note: Percentages may not add to 100% because of rounding.

2.8
The survey results showed that, overall, most teachers in our sample considered that the resources were very useful or useful. We provide further information about, and discussion of, these survey results in the rest of this report.

Annotated exemplars of student work

2.9
Annotated exemplars of student work guide teachers when assessing students' work. The exemplars show teachers the complexity required in a student's work for the student to be awarded achieved, merit, or excellence (or not achieved) for each standard.

2.10
Our survey indicated that teachers consider that annotated exemplars of student work, produced by NZQA, were among the most useful resources available to them. However, our survey noted much frustration among teachers about the exemplars. Other audit work we did showed that some of the criticisms in survey responses about annotated exemplars of student work were not well-directed – some criticisms related to assessment tasks with sample answers, which are produced by the Ministry of Education, rather than exemplars.

2.11
Concerns were also raised about the quality of the annotated exemplars. A few of the comments from teachers indicated that the exemplars could be misleading, unhelpful, or unclear, and that they contained errors. Another frequent criticism was that assessment resources, including sample answers produced by the Ministry of Education, were provided too late to be useful for teachers.

2.12
Respondents to our survey said that they need a much greater range of annotated exemplars of student work for a wider range of subjects and grades. For the Level 1 and 2 standards that have recently been aligned with the Curriculum, exemplars are available to show the differentiation between grades. Annotated exemplars for some older standards are not always available, but these standards are being phased out.

2.13
The Ministry and NZQA (along with Learning Media Limited, which is a State-owned enterprise) produce these resources for teachers.9 NZQA told us that there have been problems with the process. Changes were made in 2010/11 to ensure that better annotated exemplars of student work were available in time for teachers to use in planning their assessments.

2.14
NZQA is also improving its website. From December 2011, NZQA's website has had a greater range of annotated exemplars of student work (for the standards introduced during 2012 for NCEA Level 2). The improved processes for writing annotated exemplars have ensured that these resources were available in time for teachers to plan their 2012 assessment tasks.

2.15
Teachers also expressed strong concern about the difficulty of finding the annotated exemplars online. This problem was not restricted to exemplars, and is discussed further in paragraphs 2.20-2.25.

Other help for teachers

2.16
To help ensure that teachers are using assessment tasks that meet the quality required for a standard, NZQA is now reviewing and certifying the quality of commercially available assessment materials. If these products meet required levels of quality, they are approved. To indicate approval, the assessment task is given a Quality Assured Assessment Materials trademark. As a result, teachers can be confident that these commercial assessment tasks meet the quality required of internal assessment tasks for students.

2.17
Teachers are keen to receive feedback from NZQA on how to improve the assessment tasks that teachers set. NZQA is now providing a service where teachers can obtain advice on assessment tasks through the Optional Teacher Selected Evidence process. Many teachers devise their own assessment tasks because they consider that these tasks will connect better with their students.

2.18
In our survey, some teachers commented that they wanted more advice on how to improve their assessment tasks. Given the current realigning of standards, teachers may want extra help to set assessment tasks in line with the Curriculum. In our view, NZQA needs to ensure that teachers have the right support for setting assessment tasks because teachers value this help.

2.19
NZQA considers that there is a fine line between giving advice on assessment tasks and doing work that is the responsibility of the Ministry. NZQA will need to keep under review how it supports teachers in developing assessment tasks to ensure that this valued support is delivered in the most effective and timely way.

Resources on the New Zealand Qualifications Authority website

2.20
NZQA's website has a number of important resources for, and communication with, teachers. These include moderators' newsletters, National Assessment Moderators' reports, assessment reports on external examination, Clarifications of Standards, and annotated exemplars of student work.

2.21
Figure 3 shows that half of the teachers responding to our survey found it moderately easy to find the information about internal assessment that they wanted on NZQA's website. A further 14% found it very easy to locate the information they wanted.

Figure 3
Teachers' feedback to our survey question: "How easy is it to find the information you want about internal assessment on the NZQA website?"

Figure 3 - Teachers' feedback to our survey question: "How easy is it to find the information you want about internal assessment on the NZQA website?

Note: Percentages may not add to 100% because of rounding.

2.22
The remaining 36% of teachers who responded considered that it was not easy to access information. The most common concerns about the website were that:

  • related information for each subject is not accessible in one place – some information is on the NZQA website (www.nzqa.govt.nz) and other information is on the Ministry's Te Kete Ipurangi (TKI) website (www.tki.org.nz); and
  • the search function on NZQA's website is not useful and it takes a long time to find the information sought.

2.23
We reviewed NZQA's website and found that it was possible to find all the relevant subject information in one place – for example, standards, assessment tasks, annotated exemplars, and additional information such as National Assessment Moderators' reports and Clarifications of Standards. However, there are many links between NZQA's website and TKI's website – and some take the user around in circles, which is confusing.

2.24
We told NZQA that its website could be more useful. NZQA agreed, and accepted that some information is difficult to find. NZQA has recently revised its subject-specific web pages, removed additional links and clutter, and reorganised the content. NZQA considers that it now provides clearer links to materials on other sites so that teachers should be less confused when following links between sites. NZQA also told us that the TKI website would be improved in April 2012.

2.25
At the five schools that we visited, the teachers we spoke to considered that the changes to NZQA's website were an improvement and that information was now easier to find.

Newsletters, email circulars, and resources for Principals' Nominees

2.26
NZQA communicates directly with principals, Principals' Nominees, and teachers.

2.27
The role of the Principal's Nominee is to:

  • monitor the quality assurance of internal assessment;
  • ensure that the school engages in the external moderation process and meets all requirements;
  • ensure that issues identified by the external moderation process are addressed;
  • respond to NZQA communications and requests for information, and communicate relevant information to other teachers in the school;
  • ensure that internal assessments are completed in time to meet deadlines; and
  • facilitate NZQA visits and reviews.

2.28
NZQA regularly sends out circulars for principals and Principals' Nominees during school terms. These provide clarification about, and information about important changes to, NCEA administration. They also offer reminders about deadlines.

2.29
NZQA also produces additional resources for Principals' Nominees that are intended to help them keep track of all the required NCEA administration work and to manage the school's internal quality control systems. NZQA's School Relationship Managers (SRMs) are available to answer telephone and email queries from Principals' Nominees. NZQA also holds annual briefings for Principals' Nominees. Principals' Nominees at the five schools we visited told us that these briefings were useful.

2.30
A good flow of information between NZQA and schools depends on the Principal's Nominee keeping up to date and forwarding relevant information to other teachers. Example 2 (see Part 3) shows how a Principal's Nominee can influence the quality of internal assessment.

2.31
Teachers can subscribe to emailed newsletters containing subject-related assessment information. NZQA plans to add new functions to its website so that subscribers will be sent an email to alert them when new material is available.

2.32
To keep up to date, teachers must regularly access documents online and read NZQA's emails and the information forwarded by the Principal's Nominee. We consider that NZQA could simplify and streamline its methods for communicating important information to teachers to make it easier for teachers to stay up to date.

Direct advice and feedback for teachers

2.33
NZQA runs face-to-face training for teachers on internal assessment and provides opportunities for teachers to ask for specific help.

Best Practice Workshops

2.34
NZQA moderators run Best Practice Workshops throughout the country. The workshops allow teachers to:

  • understand more clearly how to assess students' work;
  • work with assessment materials supplied by NZQA, including samples of students' work;
  • discuss internal assessment with colleagues; and
  • discuss new standards.

2.35
Teachers told us that they value the Best Practice Workshops. They used words such as "essential", "invaluable", "fantastic", and "outstanding" to describe the Best Practice Workshops. Teachers value the ability to discuss issues face to face with moderators at the workshops. One teacher commented:

The Moderation Best Practice Workshops have been hugely successful for our staff. They've come back from them enthused and more confident about internal assessment.

2.36
NZQA evaluates its Best Practice Workshops and has changed the format of the workshops in response to feedback. For example, it has made the workshops one hour longer to allow for more discussion.

2.37
NZQA has encouraged teachers who are new to NCEA assessment, beginning teachers, and teachers who need to improve their moderation agreement rates and who have not attended a workshop before to attend Best Practice Workshops. This indicates a targeted approach to using this valued resource. NZQA reports that one in three teachers has attended a Best Practice Workshop.

2.38
In 2011, 3762 teachers attended a Best Practice Workshop. These workshops were held in 30 locations around the country and covered 27 subjects. We would expect teachers who have attended the workshops to share their learning with other teachers who have not yet attended. However, we consider that access to the workshops could improve.

2.39
In 2011, NZQA ran 221 of a planned 370 workshops. NZQA cancelled the other 149 (40%) because not enough teachers had registered for the workshops. NZQA aims to have at least eight teachers at each workshop.

2.40
To make Best Practice Workshops more accessible to teachers, NZQA has:

  • stopped charging registration fees;10
  • scheduled workshops during school holidays, weekends, or teachers-only days, so that schools do not need to pay for relief teachers;
  • invited teachers to attend workshops in another location or rescheduled cancelled workshops;
  • for workshops with less demand, required fewer participants (five to seven teachers, instead of eight) before holding a workshop;
  • provided workshops on request if at least eight teachers want to attend;
  • held some workshops by video conference; and
  • scheduled workshops alongside national conferences, so that conference participants could attend a Best Practice Workshop during the conference.

2.41
Responses to our survey indicated that many teachers were frustrated by the number of cancelled workshops and that some teachers considered that not enough workshops were held in remote places. Of the workshops held in 2011, 46% were in smaller centres but NZQA was more likely to cancel workshops in these smaller centres because teacher registrations were low.

2.42
In some regions in 2011, the rate of cancellations meant that teachers had access to fewer workshops covering fewer subjects. For example, in Taranaki and Whanganui, 73% of the planned workshops (16 of 22) were cancelled. Teachers in these locations would have had to travel at least as far as Palmerston North or Hamilton (which might require an overnight stay) to attend a Maths workshop.

2.43
In our view, NZQA has done a lot to make Best Practice Workshops accessible and, from 2012, has reduced financial barriers to attendance. We consider that NZQA could regularly review the regional availability and possible delivery mechanisms for workshops, to ensure that teachers are able to attend.

Optional Teacher Selected Evidence

2.44
As well as the mandatory external moderation of teachers' assessments (see Part 3), teachers can also choose to submit additional examples of students' work to moderators, to get specific feedback. This is called Optional Teacher Selected Evidence (OTSE). The purpose of OTSE is to help teachers make difficult decisions about whether to assess a student's work as achieved, merit, excellence, or not achieved, or to help with understanding important aspects of a standard.

2.45
We consider that OTSE encourages discussion between NZQA and teachers. It supports teachers' ongoing learning and improvement.

2.46
Although survey responses indicated that some teachers did not know that they could submit OTSE, the number of OTSE increased markedly between 2010 (849) and 2011 (1759). The increase in OTSE can be linked to the introduction of new standards for NCEA Level 1 in 2011.

2.47
A response from our survey mentioned the use of OTSE:

Teachers from our school have utilised the optional selected evidence for moderation of new standards to check that their marking is on track for Year 11 new standards before the moderation process for the newly aligned standards begins in 2012 and found this to be extremely helpful.

2.48
Teachers raised concerns in our survey about how long it takes to get feedback from NZQA on OTSE. NZQA's turnaround time for OTSE is about three weeks. In our view, NZQA will need to continue to work with teachers to ensure that the timeliness of OTSE meets the needs of teachers.

8: From 2012, the information provided in National Assessment Moderators' reports will be included in the moderator newsletters and information clarifying the standards (Clarifications of Standards). NZQA has made this change in response to feedback from teachers.

9: The current process is that the Ministry produces the standards, Learning Media Limited prepares the assessment tasks, then NZQA trials the tasks in schools and creates annotated exemplars of the work that students did in the trial. NZQA provides quality assurance over the tasks for the Ministry.

10: NZQA told us that it charged fees for Best Practice Workshops to recover the costs of providing them.

page top