Part 4: The suitability of the 2016 Model
4.1
In this Part, we consider the 2016 Statement's projection model (the 2016 Model). We consider how the projection modelling has changed since 2013, and the reasonableness and robustness of the assumptions used in the 2016 Model.
Description of the 2016 Statement's projection model
4.2
The 2016 Model is designed to analyse the financial effects of an ageing population on the long-term financial position of the Government. There are two projections developed in the 2016 Model:
- historical spending patterns – a projection showing the financial implications of population ageing on government net debt; and
- stabilise net debt – a projection showing how the financial implications of population ageing on government net debt could be reduced by reducing government spending.
4.3
The 2016 Model is based on a set of spreadsheets created before 2006 to model medium-term fiscal strategies. The 2016 Model has been substantially updated and improved from the 2013 Model,43 but its basic structure remains the same. It has two parts:
- using the latest Economic and Fiscal Update forecasts as its base, it projects increased spending on healthcare, superannuation, and education44 associated with population ageing – all other revenue (mainly tax) and expenses (such as non-superannuation welfare) revert to a fixed historical-based percentage of GDP; and
- showing how the large increases in spending on healthcare and superannuation flow through and affect the operations, investments, and financing of the Government, in a standardised statement of financial performance and position, every year through to 2060.
4.4
The main measure of the Government's financial position is core Crown net debt without deducting the assets of the New Zealand Superannuation Fund.45 The assumptions behind this measure is one of four we consider more closely at the end of this Part.
Our high-level review of the 2016 Model
4.5
As in 2013, we carried out a high-level review of the 2016 Model's mechanics, internal logic, and presentation. We also examined independent reviews of the 2013 and 2016 Models.
Significantly improved presentation, mechanics, and quality control
4.6
During our 2013 review, we identified weaknesses in the robustness and usability of the 2013 Model. There were modelling problems, accounting inconsistencies, and limited ability to carry out "real world" analysis of uncertainty and shocks.
4.7
Based on this and other reviews, the Treasury has substantially improved the mechanics, accounting logic, and presentation of the 2016 Model. The 2016 Model's quality controls have been strengthened. We found no arithmetic errors, or errors in formulas.
4.8
However, the financial projection spreadsheets remain complex and at times difficult to follow. For example, understanding how the New Zealand Superannuation Fund will affect future core Crown borrowing is important but difficult to follow in the 2016 Model.
4.9
We expected to see that for the New Zealand Superannuation Fund, as the core Crown pays contributions to and receives contributions and taxes back from the Fund, these amounts would directly affect core Crown borrowing and interest costs. Instead, the 2016 Model assumes that any change in the Fund's net assets directly affects core Crown borrowing and interest costs. This modelling approach is problematic because changes in the Fund's net assets also include any market value gains and losses and movements in reserves, which would not normally affect core Crown borrowing.
4.10
The Treasury also found this approach difficult to explain and plans to improve the way the New Zealand Superannuation Fund is incorporated into the model.
4.11
Another example is how projected capital expenditure is modelled. The 2016 Model uses a theoretical accounting relationship rather than an assessment of likely capital needs or with any apparent reference back to the Treasury's 10-year capital intentions plan.
4.12
Although the 2016 Model has significantly improved from earlier versions, it remains our view that the 2016 Model would benefit from the Treasury obtaining expert financial modelling advice.
The modelling remains deterministic and linear
4.13
In 2013, the then Auditor-General was "concerned that the level of uncertainty implicit in the projection may not be readily understood by readers".46 We also commented on the lack of feedback loops in the 2013 Model. For example, we expected to see an increase in the interest rate on debt as the amount of debt increased – particularly to the levels that the financial projection anticipates.
4.14
Although much of the structure and the mechanics of the 2016 Model have improved, it remains limited in its ability to allow for real-world factors such as feedback loops and uncertainty. For example, it still does not include the above interest rate feedback effect. These limitations were highlighted in two recent working papers47 produced by the Treasury, which looked at how feedback loops and uncertainty could affect the Government's long-term financial position.
4.15
Both papers concluded that feedback loops and uncertainty were material to projecting the Government's long-term financial position. To show this, the authors prepared a simplified model that was consistent with the Treasury's long-term fiscal projection framework. One of the papers referred to the lack of analysis of feedback loops and uncertainty within the Treasury's long-term fiscal model as "serious" limitations.
Key person risk remains
4.16
During our work on the 2013 Statement, we noted that, for all practical purposes, the 2013 Model was maintained and operated by only one person in the Treasury. Although we were impressed with this person's commitment, competence, and knowledge of government, he appeared regularly overburdened and in need of some specialist support.
4.17
The Treasury has brought in staff to help from time to time, but the succession risk remains high. The Model's maintenance and operation is still, for all practical purposes, done by one person.
External reviews
4.18
There have been two external reviews of the Treasury's long-term financial modelling since 2013:
- As part of an external review of the Treasury's fiscal policy advice, Teresa Ter-Minassian, International Economic Consultant and former Director of the International Monetary Fund's Fiscal Affairs Department, looked at the Treasury's long-term analysis work in the 2013 Statement. Although noting that the scope of the analytical work in 2013 was impressive, Ter-Minassian suggested that improvements could be made to the 2013 Model in integrating feedback loops and incorporating sensitivity analysis of shocks to help demonstrate the need for fiscal buffers.48
- Professor Norman Gemmell, Chair in Public Finance at Victoria University of Wellington, reviewed the 2016 Model's assumptions and logic. He noted the basic approach and structure of the projection model is broadly sensible for making longterm financial projections and also noted the improvements that had been made since 2013. However, he observed an important internal tension or weakness because the future projections continue to use historical norms and unchanged policies. Professor Gemmell considered that this could cause some confusion about whether the Statement involves projections of the most likely scenarios or unchanged policy outcomes.
The reasonableness of the assumptions in the 2016 Model
4.19
The starting point for the long-term financial projections is the Treasury's Fiscal Strategy Model, which is used to populate the first five years of the projections (from 2016 to 2020). The assumptions underlying the Fiscal Strategy Model largely reflect current government policy settings.
4.20
From 2021 to 2060, the approach to projecting the long-term financial position moves away from the Fiscal Strategy Model. The move occurs in three main ways:
- All revenue and many expenses move to a constant, historical-based percentage of GDP.
- Economic and financial parameters, such as the Consumer Price Index, labour productivity, and interest rates move to a constant target.
- Healthcare, superannuation, and education costs are allowed to increase based on changes in New Zealand's population structure.
4.21
To ensure that its assumptions are reasonable and complete, the Treasury uses expert information and guidance, and tests these assumptions internally and externally. For most of the expenses and all the revenue groups, the Treasury assumes they revert to a constant historical percentage of GDP (usually the median or average for the last 20 years). The Treasury also assumes there will be no fluctuations in the economy or any natural or financial crises.
4.22
The Treasury is clear that many of the constant, historical-based assumptions are not its best estimate of what could happen in the future. These assumptions are part of a "what-if" analysis that is designed to show what would happen to government debt only if population changes increased the country's healthcare and superannuation costs.
4.23
In our view, using constant, historical-based assumptions in a financial projection would not normally be reasonable and supportable – unless there was an expectation of reverting back to a historical trend.
4.24
We noticed that there have been many changes to the mechanics of the 2016 Model and the assumptions underlying the projection. For example, there were 26 variables that capped revenue to GDP ratios in the 2013 Model, and these were reduced to seven in the 2016 Model. Although improvements are expected, substantial and ongoing changes can raise questions about the consistency and comparability of the projection process from one long-term fiscal statement to the next.
4.25
Many of these changes are also important for understanding the 2016 Statement's projections. However, it was generally difficult to locate the Treasury's discussion and analysis of these changes, some of which were included in background papers to the 2016 Statement. For example, there was no explanation in the 2016 Statement for why projected healthcare expenditure in 2060 fell from 10.8% in the 2013 Model to 9.7% of GDP in the 2016 Model.
Four examples of how the Treasury's assumptions are developed
4.26
As part of our review, we considered four of the main assumptions behind the 2016 Statement's financial projections.
Anticipating tax revenue
4.27
Projecting tax revenue is important in the 2016 Model. The projections assume:
- tax revenue is capped at a constant 28.6% of GDP (after an initial adjustment period). A small change in this assumption can be significant. For instance, if this assumption changed from 28.6% to 30.0%, net debt to GDP in 2060 would fall from 205.8% to 150.0%; and
- there is no allowance for personal income tax revenue to increase because of increasing wages and salaries.
4.28
The constant projected tax revenue is based on an estimated average tax revenue to GDP ratio over a 20-year historical period. The modelling approach was chosen to reflect a constant historical norm but recent history has shown substantial variation in this ratio, ranging from 34.9% in 1990 to 25.0% in 2011.
4.29
Although 28.6% is within a reasonable range, we do not consider that holding tax revenue constant at 28.6% of GDP is reasonable when projecting the long-term financial position of the Government. For example, the constant percentage assumes there can be no additional tax revenue earned from older people working longer or in response to the increasing expenses associated with an ageing population. Furthermore, no allowance is made for any additional revenue earned as personal incomes rise.
4.30
Personal income tax is a large part of total tax revenue (about 45% in 2015) and, without regular increases in tax bracket thresholds over time (to keep them in line with rising incomes), the income tax revenue to GDP ratio would rise. This is called fiscal drag. In the past, governments have increased tax brackets about every 10 years to reduce the tendency for personal income tax rates to rise as incomes rise.
4.31
Although there are legitimate reasons why fiscal drag should not be assumed to continue indefinitely, a more accurate assumption might have been that fiscal drag could continue for periods of time. Alternatively, a case could also be made where taxpayers, projected to have much higher average incomes by 2060, are willing to pay higher average tax rates than their counterparts in 2016.49
4.32
The 2016 Model includes the functionality to enable a fiscal drag assumption during the 40-year projection period.
Anticipating healthcare expenditure
4.33
The 2016 Statement projects an increase in public healthcare spending from 6.2% of GDP in 2016 to 9.7% in 2060 with an ageing population. We consider it reasonable to assume that the healthcare spending ratio will rise. For example, other OECD countries with an ageing population are anticipating increases in public healthcare spending relative to GDP.
4.34
The healthcare spending projection in the 2016 Model assumes that two factors push up public healthcare costs relative to GDP: rising healthcare prices relative to average consumer prices (the Consumer Price Index), and faster growth in the sector relative to the wider economy (that is, unrelated to demographic changes or price changes).
4.35
These non-demographic factors raise healthcare costs 35% faster than normal real output growth and 25% faster than normal consumer price growth. These two factors are behind most of the increase in healthcare costs during the 40-year projection period. If we removed them both, healthcare spending to GDP would increase only slightly during the projection period.
4.36
International evidence suggests that consumer spending on healthcare could rise faster than for other elements of GDP. This is through the higher price growth and proportionately greater demands for healthcare that come when incomes are higher. The Treasury's healthcare spending projections are based on some historical evidence for this pattern in New Zealand.
4.37
However, despite these increases, the 2060 projection of 9.7% of GDP is well below the projection of 10.8% in the 2013 Statement. This is because of a simplification of the method used to project healthcare spending, where the projected levels of health-related demographic, productivity, and price growth all decline compared to 2013.
4.38
The short period (the past 15 years) for which non-demographic healthcare spending trends are estimated, and relatively high volatility in the New Zealand data also suggest that the reliability of these trend estimates is limited. This, in turn, suggests a need for greater sensitivity testing of those projection assumptions. A similar argument applies to the comparable analysis of education spending.
4.39
Given the large future financial impact of these trends, we consider that more analysis and discussion of the uncertainty in the healthcare spending projection would be helpful for users of the 2016 Statement.
Anticipating outcomes from social investment opportunities
4.40
The 2016 Statement includes an analysis of the financial consequences that could happen if various outcomes from the Government's current social investment policies were achieved at different points in the future.
4.41
For example, the equitable Māori outcomes scenario, if successful, is expected to reduce future financial costs of social assistance and protection. Under this scenario, the projected net debt to GDP ratio in 2060 would reduce from 205.8% in the 2016 projection to 173.9%.
4.42
The equitable Māori outcomes opportunity projects the long-term financial effect of "bringing Māori outcomes to the same level as the rest of the population within two generations (35 years)".50 These outcomes include criminal justice, education, and welfare-related benefits. The scenario is projected to generate a net reduction in core Crown spending (tax revenue effects are ignored) of about one percentage point of GDP in 2060, mainly through lower welfare spending.
4.43
The various opportunities presented as part of the social investment analysis is a positive first step. However, significantly more work is needed to ensure that the assumptions the analysis and findings rely on are reasonable and complete. For example, except for the expert case studies scenario, all other opportunities presented assume dramatic changes in social conditions and outcomes at no additional financial cost. There is also no analysis of the effects these scenarios might have on other aspects of the projections individually or collectively. As such, whether the effects are material is unknown.
4.44
Although additional analysis and explanations are available in a background paper,51 these limitations affect the usefulness of the analysis and its findings. In our view, acknowledging them would have been helpful to users of the 2016 Statement.
Assessing net debt
4.45
The Treasury recently summarised the rationale for using net debt as a financial indicator and management tool:
Net debt can be seen as better reflecting the underlying strength of the Crown balance sheet because it incorporates gross debt … as well as liquid financial assets held by the government.52
4.46
In 2009, the Treasury prepared a new definition of net debt that excluded the New Zealand Superannuation Fund's assets. The Treasury's reason for excluding these assets was because the assets "are ring-fenced for long-term fiscal pressures and their time varying and volatile nature can complicate communication of a net debt target".53
4.47
In our commentary on the 2013 Statement, the then Auditor-General stated that the exclusion of New Zealand Superannuation Fund's assets from net debt was debatable and resulted in a higher projection of Crown net debt than if those assets were included.
4.48
The New Zealand Superannuation Fund was set up in 2001 to help "pre-fund" an expected increase in superannuation spending, associated with the relative size of the ageing population. The assets created from payments into the New Zealand Superannuation Fund were achieved by diverting current tax revenue from other uses, and the anticipated future superannuation payments would come from general core Crown spending.
4.49
In our view, if New Zealand Superannuation Fund assets are excluded from this financial sustainability indicator, then projected debt associated with future superannuation payment liabilities that the assets are intended to finance should also be excluded. The main core Crown net debt measure in the 2016 Statement, like previous versions, excludes the former, but implicitly includes the latter by projecting all future superannuation payments as part of core Crown expenses.
4.50
The 2016 Model currently assumes some contributions from the New Zealand Superannuation Fund to the Crown to offset superannuation spending. These contributions (excluding tax) occur from 2033 to 2060. The average annual contribution is about one half of 1% of the Fund's total financial assets.
4.51
This level of contribution means that by 2060 only 7.5% of the total financial assets of the Fund have been paid back to the Crown as contributions, despite projected net debt rising to unsustainable levels.
4.52
Figure 15 shows core Crown net debt to GDP ratios with and without New Zealand Superannuation Fund assets.
Figure 15
Core Crown net debt to GDP ratios with and without the New Zealand Superannuation Fund's assets
4.53
We note that the 2016 Statement also includes the measure of core Crown net debt after deducting the assets of the New Zealand Superannuation Fund. We remain of the view that the New Zealand Superannuation Fund's assets should be deducted in the calculation of net debt to GDP when used as the main financial sustainability indicator in the 2016 Statement.
43: Details of the changes are included in a background paper to the 2016 Statement: the Treasury (2016), Demographic, Economic and Fiscal Assumptions and Modelling Methods in the 2016 Long-Term Fiscal Model: Background Paper for He Tirohanga Mokopuna: 2016 Statement on the Long-Term Fiscal Position.
44: Education spending is also based on population projections, but the increase is small.
45: The 2016 Statement also includes other measures including net worth and net debt after deducting the assets of the New Zealand Superannuation Fund.
46: The Treasury (2013), Long-term Fiscal Projections: Reassessing Assumptions, Testing New Perspectives, Wellington, page 6.
47: Ball, C, Creedy, J, and Scobie, G (2015), Long-run Fiscal Projections under Uncertainty: The Case of New Zealand, Wellington, pages 1 and 20. Creedy, J and Scobie, G (2015), Debt Projections and Fiscal Sustainability with Feedback Effects, Wellington, page 27.
48: Ter-Minassian, T (2014), External Review of the Treasury's Fiscal Policy Advice, Washington DC, page 50.
49: This issue is examined in some detail by J Creedy and N Gemmell (2014), Can fiscal drag pay for the public spending effects of ageing?, New Zealand Economic Papers, Vol. 48, pages 183-195. This paper was originally prepared as a 2013 Statement expert panel presentation and background paper.
50: The Treasury (2016), The Benefits of Improved Social Sector Performance – Background Paper for He Tirohanga Mokopuna: 2016 Statement on the Long-Term Fiscal Position, Wellington, page i.
51: The Treasury (2016), The Benefits of Improved Social Sector Performance, Wellington.
52: The Treasury (2014), Fiscal Indicators and the Financial Statements. A Guide to How Fiscal Indicators Are Compiled from the Financial Statements, Wellington, page 7.
53: The Treasury (2009), Fiscal Strategy Report, Wellington, page 39.