Part 2: Were the projected savings realised?
- Our expectations
- How all-at-once dispensing makes savings
- Pharmac’s projection of savings from all-at-once dispensing
- Factors Pharmac did not include in the projections
- Pharmac’s calculation of the savings from all-at-once dispensing
- Factors Pharmac did not include in the calculations
- Comparing Pharmac’s projected and calculated savings
- Examining the savings projections over 5 years
- Our conclusions
2.1
In this part, we set out Pharmac’s:
- projection of savings to district health boards from all-at-once dispensing, and the factors Pharmac did not include in the projections; and
- calculation of the savings achieved under all-at-once dispensing, and the factors Pharmac did not include in the calculations.
2.2
We discuss the difficulties in comparing Pharmac’s projected and calculated savings figures in year one, and we discuss factors to be considered when making comparisons in future years.
Our expectations
2.3
We expected that:
- Pharmac would have identified the factors that could affect achieving the projected savings from all-at-once dispensing;
- the principles used by Pharmac to project, and calculate, the savings for all-at-once dispensing would have been reasonable;
- there would have been reduced spending on dispensing fees, as at 30 June 2004, consistent with Pharmac’s projected savings; and
- calculated savings would have been attributable to all-at-once dispensing.
How all-at-once dispensing makes savings
2.4
Under monthly dispensing, pharmacists earned 3 dispensing fees for dispensing a 90-day supply of medicine, in three 30-day quantities, to patients. With the partial reintroduction of all-at-once dispensing, patients can receive the full 90-day quantity of their medicine on their first visit to the pharmacy. The pharmacist earns only one dispensing fee.
2.5
All-at-once dispensing results in a clear saving, in dispensing fees, for district health boards, at the individual prescription level and nationally. Using a notional dispensing fee of $5, a 90-day quantity of medicine that under monthly dispensing would have cost $15 in dispensing fees, would cost only $5 under all-at-once dispensing.
2.6
In the rest of this Part, we discuss whether the savings were as Pharmac projected, given that all-at-once dispensing had associated ongoing and once-only costs.
Pharmac’s projection of savings from all-at-oncen dispensing
2.7
In July 2003, Pharmac’s Board approved a proposal to partially reintroduce all-at-once dispensing from 1 October 2003. The proposal set out the projected savings for district health boards that were expected because of the change.
2.8
Because of the cost of implementing all-at-once dispensing, the first 9 months of the regime were only projected to save district health boards $5.42 million. Annual savings from then on were expected to increase from $37.69 million in year 2, to $43.47 million in year 5. The savings were expected to increase because dispensing volumes were expected to grow by 4% each year.
2.9
To work out the total savings in present value terms, Pharmac applied a discount factor3 rate of 10% each year. It projected that all-at-once dispensing would save a total of $132.98 million for the years ending 30 June 2004 through to 2008, as shown in Figure 1.
Figure 1
Projected district health board savings from all-at-once dispensing ($m)
| 2004 | 2005 | 2006 | 2007 | 2008 | Total | |
|---|---|---|---|---|---|---|
| Net dollar savings | 5.42 | 37.69 | 39.36 | 41.37 | 43.47 | 167.31 |
| Savings in 2004 dollars | 5.42 | 34.26 | 32.53 | 31.08 | 29.69 | 132.98 |
Source: Pharmac data.
2.10
Our audit looked at the first 9 months since the partial reintroduction of all-at-once dispensing. We sought to provide assurance on whether Pharmac’s projected savings of $5.42 million for this period were achieved, and were attributable to all-at-once dispensing.
Factors Pharmac did not include in the projections
2.11
Where possible, Pharmac used conservative assumptions in reaching its savings projections. These assumptions, and the methods Pharmac used in reaching its projections, were reasonable.
2.12
Pharmac identified the most significant costs to district health boards that would occur under all-at-once dispensing. These were:
- implementing the partial reintroduction of all-at-once dispensing;
- the additional medicine provided to patients (because under monthly dispensing, patients would not, on average, collect all of the 90-day supply); and
- financial support for some remote pharmacies (those unable to remain open, following the reduction in their income from dispensing fees)4
2.13
However, Pharmac’s savings projections did not include the total cost of implementing all-at-once dispensing. Pharmacists had to increase their stock to manage the overlap between the phasing out of the monthly dispensing regime, and the partial reintroduction of all-at-once dispensing. Pharmac’s analysts estimated that this stock increase would cost district health boards $17.46 million, but this was rounded down to $17 million in the July 2003 proposal to the Pharmac Board.
2.14
Further, Pharmac’s savings projections did not include the $0.5 million that district health boards had allocated for the cost of the all-at-once dispensing communications campaign, information technology changes,5 and legal costs. We note that these costs were mentioned separately in the July 2003 proposal to the Board, and the Board was aware of them.
2.15
These 2 omissions are small compared to the projected total savings over 5 years, but they have a significant effect on the projected savings for the period 1 October 2003 to 30 June 2004. Instead of $5.42 million, Pharmac’s savings projection should have been $4.46 million for this period, and this is the figure we audited against. The breakdown of this amount is shown in Figure 2 below.
Figure 2
Projected savings, from 1 October 2003 to 30 June 2004, including all implementation costs
| Costs and savings for district health boards | Projected savings ($m) |
|---|---|
| Dispensing fee savings | 36.23 |
| Ongoing costs - additional medicine to patients - maintaining access to pharmacies* |
10.06 3.75 |
| Total ongoing costs | 13.82 |
| Savings minus ongoing costs | 22.42 |
| Once-only implementation costs - medicine supply - communication/other |
17.46 0.50 |
| Total implementation costs | 17.96 |
| Savings minus ongoing costs and implementation costs | 4.46 |
* Pharmac estimated the cost of maintaining access to pharmacies at $5 million. For the 9-month period to 30 June 2004, Pharmac estimated a cost of $3.75 million to allow for the shorter period.
Source: Our analysis of Pharmac’s data.
Pharmac’s calculation of the savings from all-at-once dispensing
2.16
Pharmac did not calculate the savings from all-at-once dispensing for 1 October to 31 December 2003. It was considered too difficult to isolate the savings because of the phased introduction of all-at-once dispensing.
2.17
Instead, Pharmac sought to calculate the savings achieved from 1 January 2004, when the all-at-once dispensing regime was fully established. To do this, Pharmac used information from the Pharmaceutical Information Database.
2.18
At the time of our audit, Pharmac had calculated the savings for the period from 1 January 2004 to 30 June 2004. The calculation was based on all pharmacy claims made during this period. Because of the verification undertaken by the Ministry of Health, complete data was not available to Pharmac until September 2004.6
2.19
Pharmac’s analysts constructed a model to calculate what the costs of the monthly regime would have been if the dispensing regime had not changed.
2.20
The model was based on dispensing fee spending. While it included the costs of additional medicine to patients, the model did not take into account all of the other factors that act to reduce savings.
Factors Pharmac did not include in the calculations
2.21
Pharmac prepared its model to monitor the ongoing savings to district health boards from 1 January 2004. It did not take into account the costs of implementation (medicine costs, communications, information technology, and legal costs) and maintaining access to pharmacies.
2.22
However, implementation costs should be taken into account, because Pharmac’s original projection of savings covers the 9 months to 30 June 2004, and district health boards incurred these costs because of the partial reintroduction of all-at-once dispensing.
2.23
In the case of the medicine costs of implementation, Pharmac’s best estimate of the cost is the original forecast of $17.46 million. Pharmac considered it too difficult, due to the effect of other policy changes, to isolate all-at-once dispensing implementation costs from other medicine cost information during the period 1 October 2003 to 31 December 2003.
2.24
Other implementation costs included the $704,417 that Pharmac spent from July 2003 on the all-at-once dispensing communications campaign, information technology changes, and legal costs.
2.25
In addition, Pharmac did not monitor the ongoing cost to district health boards of maintaining access to pharmacies. We surveyed district health boards to establish the amount spent from 1 October 2003. District health boards had allocated $779,666 to the costs of maintaining access to pharmacies in the year to 1 October 2004.
2.26
Figure 3 shows Pharmac’s calculated savings for the period from 1 January 2004 to 30 June 2004, alongside the other factors reducing savings (see paragraphs 2.21-2.25).
Figure 3
Calculated savings, from 1 January 2004 to 30 June 2004
| Costs and savings for district health boards | Calculated savings ($m) |
|---|---|
| Dispensing fee savings | 30.91 |
| Ongoing costs - additional medicine to patients - maintaining access to pharmacies* |
8.12 0.39 |
| Total ongoing costs | 8.51 |
| Savings minus ongoing costs | 22.40 |
| Once-only implementation costs - medicine supply (estimate)** - communication/other |
17.46 0.70 |
| Total implementation costs | 18.16 |
| Savings minus ongoing costs and implementation costs | 4.24 |
* As the table above relates to a 6-month period, 1 January 2004 to 30 June 2004, we have included half of the amount allocated for the 12-month period from 1 October 2003 to 1 October 2004.
** Pharmac’s best estimate of the cost.
Source: Our analysis of Pharmac’s data, and the results of our survey of district health boards.
Comparing Pharmac’s projected and calculated savings
2.27
There are factors that prevent any direct comparison between the projected savings and the calculated savings. These are the:
- different periods used;
- value of the dispensing fee used to project and calculate savings;
- estimated and actual rate of dispensing under close control;
- use of historical ratios; and
- sensitivity of Pharmac’s model for calculating savings.
Different timescales used
2.28
We are not able to provide assurance on the savings achieved in the first 3 months of all-at-once dispensing beyond our statement of in-principle savings made in paragraph 2.5 and illustrated in Figure 1. Therefore, no clear comparison can be made against Pharmac’s projections for this period.
Value of the dispensing fee
2.29
Pharmac’s model for projecting savings used the average dispensing fee, for January to March 2003, of $4.72. Pharmac’s model for calculating savings, however, uses the average dispensing fee that relates to the latest claims data. From 1 January to 30 June 2004, the average dispensing fee was $5.04.
2.30
If the original figure of $4.72 were used to calculate savings, the savings would drop by $1.56 million.
Rates of dispensing under close control
2.31
Pharmac’s projected savings assumed a close control rate of 5%. Pharmac expected the rate to be less than this, but to be conservative it decided to reduce projected dispensing fee savings to 95% of its original estimates. Savings are higher if close control rates are lower.
2.32
However, monthly reports to Pharmac’s Board between January and September 2004 showed that the rate of dispensing under close control from 1 January to 30 June 2004 was closer to 20%.
Use of historical ratios
2.33
Pharmac’s model for calculating savings uses historical data, known as pickup ratios, to generate its results. Pickup ratios are, for each medicine, the average amount of a medicine that patients collected under monthly dispensing. This data is used to model what the monthly dispensing regime would have cost, based on actual all-at-once dispensing data, and to calculate the cost of the additional medicine provided to patients under all-at-once dispensing.
2.34
This historical data cannot be updated, because the partial reintroduction of all-at-once dispensing has changed patterns of dispensing. Because these ratios are fixed, savings forecasts over 5 years have greater uncertainty the further out from 2003 that a projection is made.
Sensitivity of Pharmac’s model for calculating savings
2.35
We carried out statistical testing using a formally designed simulation study, to see how sensitive Pharmac’s model for calculating savings was to changes in certain variables, including:
- average number of dispensings each month;
- current dispensing fee7;
- pickup ratios; and
- cost of the additional medicine to patients8.
2.36
The results showed that the model is most sensitive to changes in the volume of dispensing, and changes in the dispensing fee. Simultaneous changes of plus or minus 5% across these variables gave savings results that varied by, at most, plus or minus 8.6% (95% confidence level).
2.37
Savings were still seen when the variables were all changed markedly, so we are satisfied that all-at-once dispensing results in savings for district health boards.
2.38
However, Pharmac’s statements on the calculated savings only sometimes reflect the level of sensitivity in the model. Further, any calculation of savings in the period 1 October 2003 to 30 June 2004 must take into account all the factors reducing savings.
Examining the savings projections over 5 years
2.39
Pharmac forecasts all of its savings and investment analyses over a 5-year period. In keeping with this, Pharmac projected that all-at-once dispensing would save district health boards $132.98 million over 5 years.
2.40
However, as shown in paragraphs 2.27 to 2.38, it is difficult to directly compare Pharmac’s initial projections and the calculated results – even in the first year.
2.41
As with any statistical model, Pharmac’s model for calculating savings is based on certain assumptions. One important assumption is that contractual relationships between district health boards and pharmacists remain as they were under monthly dispensing, with pharmacists earning a negotiated fee for dispensing medicine.
2.42
If, because of the partial return to all-at-once dispensing, the contractual relationship changes significantly, then a comparison with the monthly dispensing regime would not be valid. Such changes might include lump sum payments for services, or significant increases in dispensing fees in order to increase pharmacy revenue.
2.43
In future years, the dispensing fee used to calculate savings may fluctuate further from that used in the projections. The rate of dispensing under close control is similarly subject to change, and other health initiatives, such as the introduction of Primary Healthcare Organisations, could influence dispensing volumes.9
2.44
Although a certain amount of savings can be assumed, quantifying the amount of savings annually, or a net amount for 5 years, is difficult. Comparisons between any such figure and Pharmac’s original projections would be of questionable validity.
2.45
In partially reintroducing all-at-once dispensing, Pharmac was very clear that savings of $132 million over 5 years could be expected. In our opinion, this expected result should have been presented to Pharmac’s Board, and to the public, as being clearly qualified by the uncertainties inherent in reaching such a goal.
Our conclusions
2.46
There is no question that all-at-once dispensing results in a saving for district health boards, as pharmacists are paid fewer fees for dispensing each 90-day supply of medicine than was the case under monthly dispensing. This saving is partially offset by ongoing and once-only costs.
2.47
Pharmac took a reasonable approach to projecting the total value of savings to district health boards, despite some small omissions.
2.48
Pharmac’s calculation of savings to district health boards under all-at-once dispensing is reasonable. However, if any calculation of savings in the period 1 October 2003 to 30 June 2004 is made, it should take all the implementation costs into account.
2.49
Making a direct comparison between Pharmac’s projected and calculated savings figures for the first 9 months of the partial reintroduction of all-at-once dispensing would be of limited use. This is because Pharmac did not monitor savings or implementation costs between 1 October and 31 December 2003, and because of differences in the dispensing fee and the rates of dispensing under close control used to project and calculate savings.
2.50
Savings will occur in years 2 to 5 after the partial reintroduction of all-at-once dispensing, as long as the contractual relationships between district health boards and pharmacists do not change significantly. However, calculating a net 5-year value of these savings is open to much uncertainty.
2.51
We make 3 recommendations about presenting the expected effect of efficiency initiatives. These are relevant for Pharmac, although we acknowledge that it is likely that Pharmac’s Board was aware of the inherent uncertainties in the estimates of savings under the partial return of all-at-once dispensing. Recommendations 2 and 3 apply equally to other public entities, and other policy initiatives.
| Recommendation 1 |
|---|
| We recommend that the Pharmaceutical Management Agency ensure that reports on the savings under all-at-once dispensing reflect the sensitivity in the model for calculating savings. |
| Recommendation 2 |
|---|
| We recommend that when the forecast effects of any efficiency proposals are put to decision-makers, a breakdown of the important assumptions, uncertainties, and limitations around the estimates is included. |
| Recommendation 3 |
|---|
| We recommend that ranges of likely results be given when efficiency proposals are put to decision-makers, or released to the public, to reflect any uncertainty in the calculations. |
3: Discount factors translate expected benefits or costs in any given future year into present value terms. The discount factor is equal to 1/(1 + i)t where i is the interest rate, and t is the number of years from the date of initiation for the programme or policy until the given future year.
4: District health boards are required to maintain local access to pharmacy services.
5: The cost of updating the information technology used by pharmacies and doctors.
6: It takes approximately 3 months for verified data on pharmacy claims to become available from the Ministry of Health’s Pharmaceutical Information Database.
7: Pharmac’s model multiplies the number of dispensings that would have occurred under monthly dispensing by the current dispensing fee, in order to calculate dispensing fee savings.
8: This cost is subtracted from the dispensing fee savings to give the net savings as reported by Pharmac.
9: Primary Healthcare Organisations are expected to increase the volumes of medicine dispensed nationally. As noted earlier, in Pharmac’s model an increase in dispensing leads to an increased dollar value of savings.
page top