Part 4: Looking at the effectiveness of investment in assets
4.1
In this Part, we use a simple measure to show a relationship between the money spent on assets and educational outcomes for students – in other words, a basic cost-effectiveness measure that we call "investment effectiveness". We used publicly available information about tertiary education institutions' assets and publicly available accountability information about education outcomes (educational performance indicators) to develop the measure.
4.2
We wanted to show how the relationship between the money spent on assets (as measured by the net assets of each tertiary education institution) and educational outcomes for students could change with changes in sector-wide factors. That is, we considered external factors that could influence the behaviour and decisions of tertiary education providers and how changes in those factors affect our investment-effectiveness measure.
4.3
There are different ways of measuring the effectiveness of asset investment in the tertiary education sector. Investment decisions are made not only for educational purposes but also for research purposes, or a combination of both. A range of measures would be needed to comprehensively assess such objectives. The business cases we looked at did not clearly include research objectives together with educational outcome measures as reasons for the investments.
4.4
The educational performance indicators (noted further below) are only a part of a fuller cost-effectiveness picture. Qualification completion rates, for example, do not necessarily attach greater value to completing degree courses in one subject area or another. Participation rates are also not strictly an educational outcome indicator but are used by the Tertiary Education Commission as a broader educational performance measure of the proportion of students from particular groups engaged in tertiary education. The Tertiary Education Commission publishes participation rates alongside the four main educational performance indicators.
4.5
Educational performance indicators do not provide a comprehensive cost-effectiveness picture. The challenge for the whole sector is to improve on them. Comprehensive, publicly available performance information is important for transparency and accountability. In a climate that includes a greater emphasis on investing for outcomes or an "investment approach", improvements in cost-effectiveness measures in the public sector become even more relevant.
4.6
We intentionally used a simple measure to start conversations in the tertiary education sector about what is the right information for developing a range of cost-effectiveness measures for the sector and individual tertiary education institutions.
Summary of our findings
4.7
Our investment effectiveness measure compares the net assets of each tertiary education institution with its educational outcomes. We tested how investment effectiveness is affected by sector-wide changes in:
- international student numbers;
- domestic student numbers; and
- the proportion of the students in different regions.
4.8
The results of our testing show that a change in these factors can significantly alter the effectiveness of investment in tertiary education institutions. The effects differed for individual tertiary education institutions and for tertiary education institutions as a group. Specifically, our testing showed:
- a wide range of investment effectiveness between tertiary education institutions, with little apparent relationship to size, location, or type of tertiary education institution;
- that tertiary education institutions offering capital-intensive courses did not necessarily have a poorer result for investment effectiveness;
- large changes in international student numbers made little difference to investment effectiveness;
- small changes in domestic student numbers made a comparatively large difference to investment effectiveness; and
- one scenario of an increase in the market share of students in one region reduced the average investment effectiveness for all tertiary education institutions.
4.9
Changes in all three sector-wide factors had different effects for individual tertiary education institutions, depending on their operating and capital structures.
Assets, net assets, and student numbers
4.10
The value of the assets that tertiary education institutions hold has increased in recent years. According to the Tertiary Education Commission's capital intentions plan in 2014, tertiary education institutions were planning to spend a further $8.2 billion on capital assets in the 10 years to 2024.
4.11
Figure 3 shows tertiary education institutions' total assets: property, plant, and equipment, net assets, and student numbers as at 31 December in 2014 and 2015. In accounting terms, assets are not just physical items of property, plant, and equipment, but include, for example, cash in the bank and any money owed to the institution. Net assets are total assets minus total liabilities.
Figure 3
Tertiary education institutions' assets: property, plant, and equipment, net assets, and student numbers in 2014 and 2015
| Year | Total assets | Property, plant, and equipment | Net assets | Total number of students |
|---|---|---|---|---|
| 2014 | $11.1 billion | $9.1 billion | $9.4 billion | 232,398 |
| 2015 | $11.9 billion | $9.7 billion | $10 billion | 232,245 |
Source: Annual financial statements of tertiary education institutions and the Tertiary Education Commision's consolidated financial data.
4.12
Figure 4 shows how the tertiary education institutions' student numbers, net assets, and property, plant, and equipment changed between 2005 and 2015.
Figure 4
Change in student numbers, net assets, and property, plant, and equipment, 2005 to 2014
4.13
Figure 4 shows that, from 2005 to 2014, there was little apparent correlation between the growth in student numbers and the growth in property, plant, and equipment or net assets. Although the annual change in student numbers did not vary a lot during the 10 years, net assets and property, plant, and equipment growth declined significantly from 2006 to 2011, and then increased significantly from 2011 to 2014.
4.14
All tertiary education institutions have operating and capital structures that reflect, for example, the mix of courses offered, fee structures, proportions of international students, and types of assets. These differences influence not only the effectiveness of the investment in each tertiary education institution, but also mean that each tertiary education institution will react differently to changes in the wider sector in which it operates.
How we assessed the effectiveness of investment in assets
4.15
There are different ways of measuring the effectiveness of an investment. Our focus was on the accumulated investment in each tertiary education institution (as measured by its net assets) and the possible relationship with educational outcomes. The financial data used are from 2014/15.
4.16
Our measure of investment effectiveness was calculated using the following formula:
4.17
Net assets are the total assets of tertiary education institutions less their total liabilities. This accumulated investment cost is affected by movements in the level of Government and other funding and the tertiary education institutions' operational and investment decisions and programmes over time.
4.18
Student achievement is calculated as the average of the Tertiary Education Commission's educational performance indicators (course completion, qualification completion, student progression, student retention, and participation) multiplied by the number of students attending the tertiary education institution (which, for consistency, we sourced from the Ministry of Education). The Tertiary Education Commission told us that course completion was the most relevant indicator. We ran our analysis using only course completion data and, although this changed the relative positions of some tertiary education institutions, it did not materially change the overall picture. We note that there have recently been changes to the educational performance indicators and this may change the results of our testing.
4.19
Our simple measure is similar to how the Social Investment Unit analysed the investment effectiveness of various school initiatives, which considered investment cost, number of students, and the number of Level 2 qualifications gained in the National Certificate in Educational Achievement. Our measure is not comprehensive and there are sometimes good reasons why individual tertiary education institutions may maintain a high level of assets to achieve the educational performance they intend. For example, our measure does not consider:
- the strategic value of asset intensive courses, such as engineering, science, dentistry, and medicine, or the tertiary education institutions' wider social value to regions;
- the value of assets in terms of research; and
- contextual or historical factors that influence the financial position of individual tertiary education institutions – such as those that decide to lease their buildings or purchase them over time. Some tertiary education institutions also accommodate students of other tertiary education institutions on their campuses.
4.20
For the following analysis, we excluded the three wānanga. They, in particular, show the difficulties in measuring educational performance between different types of tertiary education institutions.9
4.21
To explain how the measure works, Figure 5 shows that for two tertiary education institutions with the same student achievement, the one with the highest net assets (Institution B) would be the least effective investment (that is, the most expensive provider). The lower the cost of student achievement, the better the investment is (at least in terms of how many net assets are needed to achieve the educational outcomes, as measured by the average of five education performance indicators).10
Figure 5
The calculations for measuring investment effectiveness
| Net assets | Average educational performance | Total students | Investment effectiveness calculation | Conclusion | |
|---|---|---|---|---|---|
| Institution A | $1.2 billion | 90% | 11,000 | $1.2 billion / [90% x 11,000] =$121,000 | More effective investment (a lower cost) |
| Institution B | $1.6 billion | 90% | 11,000 | $1.6 billion / [90% x 11,000] =$162,000 | Less effective investment (a higher cost) |
Tertiary education institutions' investment effectiveness
4.22
Figure 6 shows (based on our formula) our assessment of investment effectiveness of the 26 tertiary education institutions that we included.
Figure 6
Effectiveness of the investment made by 26 tertiary education institutions, using 2015 data
4.23
In dollar amounts, the average score of 74 in Figure 6 means that the cost of generating one unit of educational performance is about $74,000 for each student. The higher the cost, the less effective the investment in educational outcomes for each student.
4.24
At least in terms of how many net assets are needed to achieve the educational outcomes that are measured by the five educational performance indicators, the lower the cost in assets for each student, the better. Our measure is intended to illustrate one way of looking at asset effectiveness. We would expect the sector and each tertiary education institution to continue to develop the performance information that best captures comparative information as well as individual tertiary education institutions' points of difference and the breadth of their activities and purpose.
4.25
As Figure 6 shows, there is a wide range of investment effectiveness between tertiary education institutions. There is little apparent relationship with size, location, or type of tertiary education institution.
4.26
The cost of student achievement is highest (most expensive) for Lincoln University, Aoraki Polytechnic, and University of Canterbury. The cost of student achievement is lowest (less expensive) for Auckland University of Technology, Whitireia Polytechnic, and Otago Polytechnic.
4.27
One possible reason for some tertiary education institutions having relatively low investment effectiveness (see Figure 6) is the capital-intensive nature of some courses. However, using the Tertiary Education Commission's funding data for 2014, three of the five universities offering the most capital-intensive courses have relatively high investment effectiveness (the University of Auckland, Auckland University of Technology, and Victoria University of Wellington).11 This suggests the capital-intensive nature of some courses is only one determinant of a tertiary education institution's investment effectiveness (see paragraph 4.19).
4.28
Having set a base of investment effectiveness, we tested how it changed when we adjusted the number of international students, number of domestic students, and market share of students between regions. The first two tests show an unsurprising relationship between increasing student numbers and increasing investment effectiveness – but the relationship is much more significant for domestic students than for international students.
The effect of changes in the number of international students
4.29
To do this assessment, we prepared summary financial statements (using 2015 data) to show how changing international student numbers could affect the net assets, and therefore investment effectiveness, of each tertiary education institution.
4.30
Appendix 2 shows how we put together the summary financial statements and how changing student numbers affected two tertiary education institutions in different ways.
4.31
To work out a sensible range for how much international student numbers could change, we reviewed historical annual international student data from 1998 to 2015 and other forecast data from the Ministry of Education.
4.32
Figure 7 shows the annual changes in international student numbers from 1998 to 2015.
Figure 7
Annual changes in international student numbers, 1998 to 2015
4.33
Figure 7 shows that annual changes in international student numbers between 1998 and 2015 ranged between a drop of 15% and an increase of 54%.
4.34
Although international student numbers have been relatively stable in recent years, looking at the range of reasons for past variability suggests that this stability cannot be assumed.
4.35
For example, Ministry of Education documents point to (mostly unexpected) causes for the larger peaks and troughs in Figure 7. The Ministry of Education notes the rapid growth of international students between 2000 and 2002 was because of a more open immigration sector, a heightened interest by Chinese students in gaining overseas tertiary qualifications, and perceptions of New Zealand being a relatively safe and low-cost country in which to study. Recent shifts in the number of international students coming from India also shows volatility.
4.36
Using the average movement in the historical and forecast data, it would be reasonable to expect that the 2015 international student number of 31,165 could have increased by as much as 18% (to 36,775) or decreased by as much as 10% (to 28,049) when looking at a one-year period. These changes are within the maximum and minimum movements shown in Figure 7.
4.37
Figure 8 shows how the 2015 average investment effectiveness would change given an 18% increase and 10% decrease in the number of international students. An 18% increase in the number of international students improves the average investment effectiveness of tertiary education institutions by 1.5%, whereas a 10% decrease reduces the average investment effectiveness by 0.9%.
Figure 8
How tertiary education institutions' 2015 average investment effectiveness could change after changes in the number of international students
| Change in number of international students | Change in investment effectiveness |
|---|---|
| +18% | 1.5% |
| -10% | -0.9% |
4.38
Based on the way international student numbers have changed in the past, the main points to note are:
- A large change in international students (+18% or -10%) means a relatively small change in the overall investment effectiveness of tertiary education institutions.
- This relatively low sensitivity will increase if the proportion of international students increases.
- Nelson Marlborough Institute of Technology and Lincoln University have the highest sensitivity to changing international student numbers. This is because their international students are a high proportion of their total student population (about 28% and 25%, respectively).
The effect of changes in the number of domestic students
4.39
We repeated this assessment to show how changing domestic student numbers could have affected the investment effectiveness of each tertiary education institution.
4.40
To work out a sensible range for how much domestic student numbers could change, we reviewed historical annual domestic student data from 1998 to 2015 and other forecast data from the Ministry of Education.
4.41
Figure 9 shows the annual changes in domestic student numbers from 1998 to 2015.
Figure 9
Annual changes in domestic student numbers, 1998 to 2015
4.42
Figure 9 shows that changes in domestic student numbers between 1998 and 2015 ranged between a decrease of 4.4% and an increase of 10.4%. Compared to international student numbers, domestic student numbers have been relatively stable.
4.43
Using the average movement in the historical and forecast data, it would be reasonable to expect that the 2015 domestic student number of 177,940 could have increased by as much as 6% (to 188,616) or decreased by as much as 4% (to 170,822) when looking at a one-year period. These changes are within the maximum and minimum movements shown in Figure 9.
4.44
Figure 10 shows the change in the sector's 2015 average investment effectiveness arising from an increase of 6% to a decrease of 4% in domestic students. A 6% increase in the number of domestic students improves the average investment effectiveness of tertiary education institutions by 3.1%. A 4% decrease reduces the average investment effectiveness by 2.1%.
Figure 10
How tertiary education institutions' 2015 average investment effectiveness could change after changes in the number of domestic students
| Change in domestic student numbers | Change in investment effectiveness |
|---|---|
| +6% | 3.1% |
| -4% | -2.1% |
4.45
Based on the way domestic student numbers have changed in the past, the main points to note are:
- A small change in domestic student numbers leads to a comparatively large change in the average investment effectiveness of tertiary education institutions.
- The investment effectiveness of tertiary education institutions is more than two times as sensitive to changing domestic student numbers than it is to changing international student numbers.
- Aoraki Polytechnic and University of Canterbury have the highest sensitivity to changing domestic student numbers. This is due to various factors including the proportion of domestic students, revenue, and net assets.
The effect of changes in market share of students between regions
4.46
Gradual changes in the market share of students throughout New Zealand are apparent when we look at the Ministry of Education's data from 2007 to 2015. Those changes are particularly apparent in the Auckland region.
4.47
Figure 11 shows changes in the market share of students for the four tertiary education institutions in Auckland (Auckland University of Technology, University of Auckland, Manukau Institute of Technology, and Unitec New Zealand) compared with the rest of the tertiary education institutions, from 2007 to 2015.
Figure 11
Changes in the market share of students in tertiary education institutions, 2007 to 2015
4.48
Figure 11 shows the market share of the four Auckland tertiary education institutions increased by 3.2% (from 30.1% to 33.3% or about 0.4% each year) from 2007 to 2015. The market share of the rest of the tertiary education institutions decreased by 3.2% (from 69.9% to 66.7% or about -0.4% each year) in the same period.
4.49
The annual change in market share is consistent but small. As an example of how market share changes could affect net assets and therefore investment effectiveness, we used the total change in market share for the eight-year period and applied it to our summary financial statements for 2015. In other words, in this scenario we assumed the four Auckland tertiary education institutions' 2015 market share of 33.4% increased by 3.2% to 36.6%12 and assumed the market share for the 22 other tertiary education institutions fell from 66.6% to 63.4%.
4.50
When we applied these changes to our summary financial statements for 2015 we assumed no new international students, so the total 2015 domestic and international student base of 209,100 remained the same.
4.51
As expected, the additional students improved investment effectiveness of the four Auckland tertiary education institutions. However, the effectiveness gains in the Auckland tertiary education institutions were more than offset by larger decreases in the effectiveness of the 22 other tertiary education institutions. The average investment effectiveness deteriorated slightly from our average starting point of $74,000 (see Figure 6) to a higher investment cost of $76,000 per student.
Our concluding comments
4.52
Knowing how individual tertiary education institutions and the sector react to sector-level factors helps to provide a better understanding of the environment in which tertiary education institutions operate and the opportunities and challenges this creates.
4.53
Using our simple measure of investment effectiveness and 2015 data, we have shown that changes in the number, type, and distribution of students can have potentially significant implications for the cost of achieving educational outcomes.
4.54
When we combine this with what we found in our business case analysis in Part 3, it is clear that there is an opportunity for education agencies, tertiary education providers, and other stakeholders to do more work on measuring the effectiveness of investment in tertiary education sector assets and forecasting the effect of changes in the wider operating environment, such as changes in student numbers.
| Recommendation 2 |
|---|
| We recommend that the Ministry of Education, the Tertiary Education Commission, and other education agencies work with tertiary education institutions to improve the use of, and investment in, tertiary education assets by considering further types of analysis, measures, and forecasting that could improve the collective effectiveness of the investment in tertiary education assets. |
9: For two of the wānanga, the Tertiary Education Commission's educational performance measures reflect a greater focus on other educational objectives. As such, they had the lowest performance measures of all tertiary education institutions. Including them would have resulted in investment effectiveness ratios that were out of keeping with those of the other tertiary education institutions. The third wānanga has large international revenues and the Ministry of Education reports only five international students. This creates difficulties in carrying out any revenue-based analysis.
10: We discussed the inclusion of the five educational performance measures with the Tertiary Education Commission. They advised that all five indicators should be included but told us the most relevant was course completion, and that some of the other measures might be less relevant to some tertiary education institutions. Given this, we also ran our analysis using only course completion measures. Although this changed the relative positions of some tertiary education institutions, it did not materially change the overall picture.
11: The other two universities offering the most capital-intensive courses are the University of Otago and Massey University.
12: Equivalent to increasing each tertiary education institution's student numbers by about 9%. We also assumed that this increase could be accommodated using the existing land and buildings.