Education Review Office

Faster and cheaper can be better

Getting better results with fewer resources may seem counter-intuitive but that’s what the Education Review Office (ERO) has learned during the past three years.

The Building Capacity in Evaluation (BCIE) initiative began in 2009 to address a number of issues ERO was facing, including keeping pace with international research into how education is delivered and how student learning is assessed. “We could see the education environment changing and we needed to build capacity within the system to address this,” says Chief Executive, Dr Graham Stoop. And, like other government organisations, ERO also had a financial imperative to change the way in which it worked.

ERO started by defining its goal – raising student performance, particularly among Māori students. All decisions, from the way in which ERO assesses schools and early childhood services to the way the organisation is now structured, have flowed from “having a clear strategic direction”, says Dr Stoop. “We were very aware that the first thing many organisations do in times like this is restructure, but that won’t have the desired impact unless the underlying reason is completely clear and it’s communicated well,” says Dr Stoop.

Efficiency and effectiveness means improving the capacity of the sector at a reduced cost to the taxpayer.

ERO then reviewed its evaluation methodology against “embryonic” international research. It concluded that the best and most sustainable way of improving results for students was for schools to own their performance assessment by conducting “self-reviews”. ERO recognised that the idea of school “self-review” was a significant change in approach and would require ERO to help school leaders to develop the capacity for doing this. Dr Stoop summarises the change in approach as, “We now ask schools ‘what have you been doing to improve?’ rather than say ‘you must do this’.”

After a year of planning and extensive consultation with key players in the education sector, ERO changed its operating model and organisational structure so that there are now closer links between regional staff and national office. The restructure included removing a middle management tier, reducing staff numbers by 41, and adding national regional managers to the senior leadership team, which now meets fortnightly. ERO review officers have also changed the way that they work. For example, they now spend less time in schools and “they model evaluative thinking and behaviour” when they are working on site. ERO review reports are also now “shorter, punchier”. As a result, ERO can focus on putting resources where they are most needed while still doing the same number of reviews a year – about 800 schools and 1300 early childhood education services.

Ongoing monitoring shows that the changes are making a difference to student achievement levels and that schools and parents are happy. Research New Zealand has conducted focus groups with parents to find out what they think of the new ERO reports, and the results will be used to make further improvements. Dr Stoop says schools have welcomed the opportunity to take responsibility for their performance, and a recent survey indicates that 88% have found the process helpful.

For Māori students in particular, the result has been impressive. ERO only “signs off” a school as having met its performance objectives if it is addressing the learning needs of its Māori students. Schools that meet this criterion are placed in the new top band that requires reviewing only every four to five years. Before BCIE, schools were reviewed every year or every two to three years. Now, 15% of schools are reviewed only every four to five years, and almost 70% are reviewed every three years. The remainder are on a one-to-two-year review cycle. Dr Stoop says the Ministry of Education and the Minister of Education have found this new categorisation helpful in leveraging improved results for Māori students, a government priority.

The BCIE is now being quoted internationally as an example of best practice, and the State Services Commission has adopted some of its evaluation processes into its Performance Improvement Framework. In early 2012, the initiative was rolled out in the early childhood education sector.

Dr Stoop puts the success of the medium-term initiative down to getting the theory right at the outset and engaging major stakeholders early. That has allowed ERO to make significant, sustainable changes rather than “tinker at the edges”. If there is one thing he would do differently, it would be to provide ERO front-line staff with more, and ongoing, support, including a professional practice model.

Based on an interview with Dr Graham Stoop, Chief Executive, on 20 June 2012.

Disclaimer: This case study is the entity’s story – we have not audited the facts but have confirmed with the entity that its story is fairly represented.