Part 5: Evaluating the proposals received

Inquiry into the Government’s decision to negotiate with SkyCity Entertainment Group Limited for an international convention centre.

5.1
In this Part, we set out:

Good practice expectations for evaluating EOI responses

5.2
As with the process for preparing an EOI document, the principles of fairness, integrity, and transparency have generated a set of well-established procedural expectations for the evaluation process. These are explained in the good practice material referred to in paragraph 3.35. In summary, those principles require the agency running the process to ensure that responses are evaluated objectively and in an even-handed way. In practice, that usually means that:

  • potential participants all have the same amount of time and information when preparing and submitting their responses to the EOI document;
  • the evaluation process has been explained to participants in advance, and any changes to it are communicated to all the participants;
  • there is objective application of the evaluation criteria by a defined group of people through a controlled and documented process; and
  • any follow-up interaction with participants during the formal evaluation process is controlled and documented so that no individual submitter has a greater opportunity than others to improve their proposal after the deadline for submissions or based on feedback from those evaluating the responses.

5.3
The level of formality and detailed control may not need to be as strong in an exploratory EOI process as in a tender that is evaluating fully developed proposals. However, the basic principle is still important. Public sector agencies need to be able to demonstrate that decisions are being made on the merits of the proposals and that nobody is given an unfair advantage. These basic steps help protect against allegations of favouritism.

The proposed assessment process

5.4
As noted, the EOI document was released on 19 May 2010, with a deadline of 18 June for submissions. On 2 June 2010, Ministry officials provided a written briefing to the Minister for Economic Development on the proposed process for assessing responses. The advice proposed a two-stage process for assessing responses to Part B of the EOI:

First officials will review each proposal against a set of criteria ... We will report to you how each one rates against these criteria and identify the top 3-4 proposals by 29 June 2010.

The second stage will involve taking the top 3-4 proposals and having these reviewed by a panel of experts. Three New Zealand-based and three internationally-based individuals will form the panel … We believe they have the right mix of knowledge, skills and expertise required to assist us in identifying which proposals might warrant more detailed scrutiny by government.

5.5
The written advice outlined the proposed criteria and the members of the panel of experts. Officials had devised evaluation criteria in five weighted categories:

  • city information: closeness to airport, hotels, shops (weighting 25);
  • fit for purpose: centre size and building attributes (weighting 35);
  • legal, regulatory, and ownership: clean site or restrictions, construction ready (weighting 10);
  • funding: costs, central government versus other funding sources (weighting 25); and
  • other: experience in operating a convention centre and positioning in national network (weighting 5).

5.6
The detailed evaluation criteria used by officials for four of the categories were essentially the same as the specific information sought in the EOI document. For example, for “City information”, the evaluation criteria were the six matters stated in the EOI document under that heading and one matter not explicitly included in the EOI document: “fit with urban development context and/or plans for city”. Of the 26 evaluation criteria, 21 were clearly signalled in the EOI document as matters that respondents needed to address.

5.7
For the funding category, five evaluation criteria were proposed, which together had a weighting of 25:

  • cost to develop;
  • central government funding versus other sources;
  • non-monetary central government support requirement;
  • investment readiness (partners identified, funding secured); and
  • cost-benefit ratio.

5.8
The EOI document simply asked for information on funding sources. Under the heading of “additional information”, submitters were also asked to provide a cost-benefit analysis if one had been undertaken.

5.9
Appendix 3 sets out the assessment table and weightings. The table also notes whether the evaluation criteria in the assessment table were included in the EOI document.

The actual process used for evaluation

The initial assessment by officials

5.10
When the request for expressions of interest closed at 5pm on 18 June 2010, eight respondents had addressed Part B of the request to some extent.

5.11 There were five responses that largely met the EOI requirements for Part B and were worth assessing.13 These proposals were from:

  • SkyCity – a site located between Hobson and Nelson Streets across the street from the SkyCity hotel and casino;
  • The Edge – a rebuild and expansion of the Aotea Centre and refurbishment of the St James Theatre in Queen Street;
  • Ngati Whatua – using land at Quay Street near the old Auckland railway station;
  • Infratil – a site at Halsey Street in the Wynyard Quarter (waterfront); and
  • ASB Showgrounds – development of its Greenlane site.

5.12
These proposals were assessed by officials (three Auckland-based Ministry officials, one Wellington-based Ministry official, and an Auckland-based official from the Ministry for the Environment). The four Auckland officials scored each of the proposals against the assessment criteria advised to the Minister for Economic Development in early June 2010. The Wellington official assessed the proposals but did not score them, because it was difficult to do so without local physical knowledge of the city and sites.

5.13
The individual scores given by officials to each of the proposals differed. On 30 June 2010, the officials had a teleconference to compare ratings and agree an initial assessment. When the scores were combined, the proposals were ranked from one to four. Two were deemed to be third equal, and the scores were all reasonably close.

5.14
However, none of the proposers had adequately addressed all of the funding elements that were part of the officials’ evaluation criteria. Only one proposal was clear on the extent of funding required from the Government. This meant that the proposals could not be scored for the funding category and this stage of the intended assessment process could not be completed.

5.15
The Ministry provided written advice to the Minister for Economic Development on 2 July 2010. The advice provided a brief analysis of the positive and negative aspects of the five proposals.

Decision not to proceed with the expert panel review

5.16
The 2 July 2010 advice noted that a panel of experts was due to meet (by teleconference) later that month to review the proposals. It also noted that officials would meet with the proposers to gather further detail about matters not addressed in the proposals and to help officials to decide on the next steps.

5.17
By Monday 26 July, officials had arranged contracts with all of the panel members and distributed an initial package of information on the proposals and the assessment work to date. Officials prepared notes on the proposals for the panel of experts and specific questions about each proposal for the panel’s consideration. These notes did not discuss how each proposal might be funded.

5.18
Officials told us that, when considering questions for the panel of experts to consider, it became apparent that the main distinguishing characteristics of the proposals were based on location. They thought that judgements about that could best be made by New Zealand-based people rather than international experts.

5.19
They discussed this matter with the Minister for Economic Development. The Minister did not think the panel would provide value for money at that point and asked officials to put the panel on hold. On 29 July 2010, the panel members were advised to stop their review of the proposals.

5.20
On 25 August 2010, officials told the panel of experts that work on a convention centre was still progressing and that the Ministry might call the panel together at some stage in the future to assess proposals and complete the evaluation process.

Obtaining more information from proposers

5.21
Ministry officials told us that, in considering the proposals, it became obvious that the information supplied by those who responded to Part B was incomplete or that the proposals did not address major contextual aspects. Some proposals did not include information on matters such as any consultation carried out with major partners (for example, hotel owners or builders) whose involvement would be needed for the proposals to succeed. Some did not address significant matters such as planning requirements or transport issues.

5.22
After the initial evaluation of the proposals, officials met with each of the proposers to seek the additional information needed to compare the proposals and to understand the wider context of the proposals. These meetings occurred after the officials’ evaluation on 30 June 2010.

5.23
Officials told us that each of the discussions was tailored to address the gaps in the proposal. However, each proposer was given an opportunity to:

  • talk officials through its proposal in detail;
  • talk about funding options it had considered; and
  • provide more detail in areas where Ministry officials thought its proposal
    had gaps.

5.24
Other information sought by officials included the views of conference professionals about the proposed designs and locations, alternative ways of funding the construction (such as a PPP), hotel occupancy rates and traffic flows within the CBD, and how Auckland City Council officers viewed the proposals. Ministry officials also sought advice and talked to experts about other technical and analytical issues, including planning requirements for some of the sites.

5.25
On 5 July 2010, one of the proposers sought a meeting with the Associate Minister of Tourism (Hon Dr Jonathan Coleman) to discuss its proposal. The Ministry suggested that this would be inappropriate because the Government, under Minister Brownlee’s leadership, was in the middle of an assessment process and it would be unwise for another Minister to become involved at that point.

5.26
Officials prepared a table summarising the proposals against their evaluation criteria and commenting on them. This went through several versions, and was later provided to the Acting Minister for Economic Development on 8 March 2011 to support the recommendation to choose SkyCity.

5.27
Officials told us that they considered that there were various issues with the other proposals that made them less suitable than SkyCity’s proposal. These included:

  • closeness to hotels and shops and public transport – one proposal was outside the CBD and two were on the CBD fringe;
  • one proposal required restoring another venue not related to the convention centre;
  • two of the proposals said hotels would be soon built in the area without providing enough support for this;
  • one proposal involved retaining an existing auditorium/theatre, which was not big enough for a large convention and could not be extended;
  • design aspects – one design involved long distances between parts of the convention centre;
  • challenging sites, including closeness to existing transport uses and to the sea; and
  • fit with urban development context or zoning requirements.

The progress of discussions with SkyCity

Initial discussions in July and August

5.28
On 21 July 2010, officials met with SkyCity to discuss its proposal. This was the first of a series of meetings throughout 2010 and in 2011, before the announcement in June 2011 that SkyCity was the Government’s preferred option. From August 2010, there was considerably more contact with SkyCity than with the other proposers.

5.29
Officials were concerned that SkyCity’s proposed site and design were not wholly suitable for the size of conventions that the centre was to attract. They were concerned that the proposed design was on too many levels and that the space for exhibitions was not big enough. The SkyCity proposal had been ranked last of the five proposals for this reason in the initial (June 2010) overall ranking by officials, despite scoring well in other areas (first for “City information”, second for “Legal, regulatory, ownership”, and first equal for “Other”).

5.30
After the 21 July 2010 meeting with SkyCity, officials sent SkyCity some comments on its proposed site and design. The officials noted that the comments were “impressions only” and “not part of any formal assessment process”. The comments were annotated on the images and plans that SkyCity had provided as part of its EOI response.

5.31
On 26 July 2010, SkyCity sent the Ministry information about the floor area of its existing convention centre and a proposed extension to that, and for an international convention centre.

5.32
SkyCity told us that, at the meeting with officials on 21 July 2010, it learned that the Government wanted a single-floor convention centre.

5.33
On 29 July 2010, the Ministry contacted SkyCity to set up a meeting with the Minister for Economic Development (Hon Gerry Brownlee) on 6 August 2010 to talk more about its convention centre proposal.

5.34
The Ministry prepared meeting notes for the Minister’s meeting with SkyCity on 6 August 2010. The notes of 5 August indicated, among other things, that:

I have already advised them that from a design and location perspective (alone) their proposal is running 4th in the competition. They need to do more to get over the line …

A useful outcome from the meeting would be agreement from Sky to work with a group of expert users of conference venues to help Sky come up with a workable and feasible design (recognising that this will not be easily achieved on the site they are currently promoting).

5.35
As part of the meeting notes, the Minister was provided with a diagram of the ownership of the land around the area of SkyCity’s proposed convention centre and the annotated comments sent to SkyCity in late July 2010. It was noted that the surrounding land ownership would affect any proposal to expand the size of the centre.

5.36
SkyCity told us that the Minister asked (at the 6 August 2010 meeting) whether SkyCity could secure more land next to its proposed site to provide for a larger building, with the convention centre on a single floor. SkyCity recalls the Minister indicating at the meeting that the Government’s willingness to consider regulatory reform depended on SkyCity meeting the requirements of a bigger convention centre on a larger site. (Later in the process, on 6 September 2010, SkyCity asked the Government to buy some of the land required – see paragraph 5.43.) On 20 August 2010, the Ministry followed up the 6 August meeting by asking SkyCity for a revised design and for details of SkyCity’s proposal for funding this.

5.37
In late August 2010, officials noted that, if the panel of experts were to meet, the officials would need to send the panel members more information, including any updated or new designs from SkyCity. However, because the panel did not meet at that time, this did not happen. Around this time, the official working on a table summarising each proposal was asked to complete the information for SkyCity last. This was because updated information had been sought.

5.38
The Minister for Economic Development did not hold any scheduled meetings with other proposers about the EOI responses during this period, and officials did not give detailed feedback on site and design issues to the other respondents as they did for SkyCity.

5.39
On 10 August 2010, a Ministry official raised a concern about process. The official was concerned that:

  • officials were not following the process that had been outlined in advice to the Minister for Economic Development in early July – that is, that the top three or four proposals would be referred to a panel for consideration; and
  • one proposer (SkyCity) had been given a time extension after its meeting with the Minister; and
  • the decision to progress SkyCity’s proposal could not be based solely on its EOI response.

5.40
Another Ministry official also expressed concern about the potential risk of changing the process. The lead Ministry official responded by noting that:

  • the Minister had been briefed on the process and had agreed that the panel of experts would not meet at the time that had been planned;
  • no proposer had been granted a time extension, and various degrees of additional information had been sought from all proposers; and
  • the deliberation by the panel of experts had been delayed rather than cancelled.

The shape of a possible deal emerges (August to December 2010)

5.41
SkyCity told us that officials told it, on 26 August 2010, that no government funding was available for a convention centre, to take account of this in preparing the revised proposal for a bigger convention centre, and to scale the proposal back if necessary.

5.42
SkyCity told us that, until that time, its preference was still for a PPP funding model, with the Government contributing some of the money required. As noted earlier (see paragraph 4.60), SkyCity told us that it considered the earlier comments from Ministers about the Government’s inability to contribute funds to be partly “positioning” statements.

5.43
On 6 September 2010, SkyCity wrote to the Ministry with a revised concept plan for a larger convention centre to be located at 101 Hobson Street and an “aide memoire” about casino regulatory reform. There were two main messages in the letter. First, that SkyCity would be prepared to consider developing a convention centre at its cost, subject to being freed from some of the regulatory restraints imposed by the Gambling Act 2003. The aide memoire outlined the specific legislative amendments sought. Secondly, because the plans were for a revised design that was larger than earlier proposed, this required buying some additional adjoining land owned by Television New Zealand (TVNZ). SkyCity wanted the Government to contribute:

  • the additional land required; and
  • an annual payment for marketing and promoting the convention centre and associated business events.

5.44
The correspondence included updated concept plans for a convention centre on an expanded site.

5.45
In acknowledging SkyCity’s 6 September 2010 letter, the Ministry noted the revised plans, the aide memoire, and the statements about the contribution sought from Government. The Ministry said that its task was now “to present that information to Ministers (along with an assessment of other National Convention Centre proposals) …”.

5.46
Officials explained to us that, by this stage, the revised SkyCity proposal was shaping up to be the one most likely to be recommended to Ministers. This was because of its combination of location, SkyCity’s convention centre management expertise, and the fact that no immediate capital funding for building the centre was required from the Government. SkyCity’s revised design for a larger convention centre better met the Government’s needs than SkyCity’s initial design.

5.47
However, before making a recommendation to Ministers to that effect, officials wanted to be sure that there would be no “show-stopping” barriers to the proposed convention centre being built on the Hobson Street site, and that the design met the international standards for a convention and exhibition centre that could take conferences of 3500 (and more) people. Therefore, officials began a range of detailed work to explore a full range of potential practical issues, including:

  • a “walk-through” of planning and consent aspects for the centre and site (carried out by Auckland Council in April 2011);
  • being clear (with the Office of Treaty Settlements) about any Treaty of Waitangi claims on the TVNZ land needed for an expanded centre;
  • valuation of the TVNZ land;
  • testing other regulatory issues that might arise (with external legal advice);
  • estimating the likely build cost of the revised design for the centre (with expert advice); and
  • using a panel of New Zealand and overseas experts to assess SkyCity’s revised design for suitability of hosting large conventions of 3500 (and more) people (in May 2011).

Considering the regulatory concessions sought by SkyCity

5.48
In late September 2010, the Ministry provided a written briefing to the Minister for Economic Development about SkyCity’s proposal, including the “quid pro quo” being sought. The “quid pro quo” covered the gambling regulation concessions that SkyCity wanted and other matters, such as the right to use the international convention centre brand, the need for the Government to provide three adjoining sites owned by TVNZ to allow for a larger convention centre, and the contribution sought for promotion and marketing.

5.49
The briefing had a more detailed section covering the regulatory concessions sought by SkyCity. This section included an assessment of whether the concessions would be controversial, and a comment on what would need to be done to make the change. The content was based on analysis by the Department of Internal Affairs. The briefing did not make recommendations.

5.50
On 10 December 2010, the Ministry responded to SkyCity’s 6 September “aide memoire” by identifying those aspects of the “quid pro quo” that the Government was prepared to consider. SkyCity and Ministry officials met again on 14 December to discuss this further. The Ministry stated that this meeting “cleared the way for a proper negotiation” but noted that a deal was still a long way off and not inevitable.

5.51
On 22 December 2010, SkyCity wrote to the Ministry with a redrafted “aide memoire”. The letter:

  • noted those items that SkyCity had sought and on which the Government was prepared to negotiate;
  • provided additional supporting comments on the items that SkyCity still wanted and thought the Government should negotiate; and
  • stated those items that SkyCity was now prepared to take “off the table” in terms of gambling reforms. Also, the letter noted that the cost to develop the convention centre was “significantly greater” than first envisaged.

5.52
The Ministry updated Auckland Council’s Chief Executive about developments with the convention centre project on 22 December 2010. Auckland Council’s Chief Executive noted that Auckland Council’s Mayor had discussed the need for an international convention centre with the Minister for Economic Development, and was keen for the project to proceed in 2011.

Further contact with the other proposers
(August – December 2010)

5.53
On 23 August 2010, the Ministry emailed those who had submitted proposals, excluding SkyCity, indicating that the Ministry would be delighted to hear about any new information that might be material to the selection decision. This was to seek any new information since officials had met with individual proposers during the first week of July 2010.

5.54
The email indicated that, since the meetings in early July, officials had been consulting widely within the Auckland community, as well as with those connected with the events and conventions sector. Officials were working through the variety of issues that needed to be considered in the selection of the best overall proposition. It said they were still on track to report to Ministers by the end of September 2010.

5.55
One submitter replied seeking more information from the Ministry on the evaluation criteria and process and likely timing of a decision, and asking for a meeting. Another asked for a meeting. Ministry officials met with each of these proposers in early September 2010.

5.56
Another proposer told us that, after the 4 September 2010 earthquake in Christchurch, it did not hold out any hope of success for its proposal because it knew there would be even more pressure on government funds.

5.57
On 30 October 2010, the New Zealand Herald reported that the Government liked the SkyCity proposal but wanted more. The article quoted SkyCity’s Chief Executive as saying that Ministry officials had asked SkyCity to expand its proposed design with a larger floor plate for the centre. The Chief Executive noted that SkyCity would need a property owned by TVNZ to meet the larger size requirements. The Chief Executive also noted that the Government had not yet decided on its preferred site, and speculated that the Government’s focus was on the aftermath of the Christchurch earthquakes.

5.58
As a result, Ministry officials met with staff from the Prime Minister’s Office on 9 November 2010 to update them on the EOI process for the international convention centre.

5.59
On 21 December 2010, the Ministry emailed all those who had submitted proposals for Parts B and C of the EOI document. The email noted:

As the year draws to a close I thought it important to send you this quick note just to let you know that the idea of a National Convention and Exhibition Centre for NZ (most likely located in Auckland) is still alive. However for a variety of reasons, including the need to give priority to responding to the Canterbury earthquake and the Pike River incident, progress has been slower than our early, optimistic, forecasts.

To those of you who responded to the EOI for the National convention centre … I want to reiterate that we do appreciate the time and effort it took to put your submissions together. As soon as we have something more concrete to convey about your proposals we will contact you.

Our comments on the evaluation process

5.60
In our view, there were a number of flaws with the way the evaluation process unfolded during 2010. The planning work that was done was too little and too late. The detail of how the evaluation process would work was put together only after the EOI had been issued, and it did not suit the situation very well. In particular, the proposed process was for a relatively standard and uniform technical evaluation of the responses. Yet the EOI was seeking creative and innovative ideas rather than fully developed proposals that would lend themselves to technical evaluation by experts.

5.61
It was also likely that the responses would be of quite different kinds: it was foreseeable that SkyCity would submit its concessions-based proposal, others might propose PPPs, and others would be for more standard construction options. The proposed process was unlikely to be able to accommodate the range of responses that the EOI was aiming to generate.

5.62
The result was that neither the process nor the detailed evaluation criteria really worked. The financial aspects of the responses could not be properly assessed based on what had been submitted, because the EOI document had not specifically asked for that information. The panel of experts proved not to be needed because the main questions were not technical, and officials needed to go back to all submitters for more information about the proposals.

5.63
We are satisfied that the initial assessment of the responses by officials was a careful and genuine evaluation of the information that had been provided. However, we are concerned that decisions about the process were made in a relatively informal way as issues arose. The contact with proposers to get more information about their proposals was also relatively informal. Very little was documented during this period. This meant that it would be harder to explain and defend the process if it was challenged.

5.64
Most importantly, the proposed evaluation process could not accommodate the situation that eventuated, where one response proposed quite a different approach that would require some difficult political decisions. Officials immediately adapted the process and began a separate stream of discussions with SkyCity to flesh out its proposal into a developed option for Ministers to consider – within the overall evaluation process.

5.65
Although we understand the challenge that officials were faced with, we do not consider that the approach they adopted was appropriate. The result was that, from the start of the evaluation process, the contact with one proposer was of a wholly different nature from the contact with others. In our view, officials effectively worked with SkyCity for some months, giving detailed feedback and engaging in some preliminary negotiations, while the other proposers were kept on hold and given very little information.

5.66
In our view, a properly planned and managed process would have guided officials and Ministers to a safer set of choices for next steps, that would have been consistent with the principles of fairness and openness. As explained in paragraph 4.46, the Government had two main options if it decided that one or more responses warranted more detailed exploration. It could either maintain competitive tension and move to a full request for proposals process, or it could decide that there was no effective competition and move immediately to negotiations for a direct procurement. Instead, the four other proposers were effectively kept in the dark for some months while the Government held detailed discussions with SkyCity.

5.67
Given the nature of the responses, it is likely that the SkyCity proposal was always going to be the most attractive from most perspectives. Indeed, in the course of this inquiry, we have not heard any comment to suggest that other proposers did not understand the reasons why the Government might prefer the SkyCity proposal. But, returning to the Government’s procedural obligations under the Mandatory Rules and good practice guidance on procurement, the Ministry still needed to run a good process to demonstrate that it had properly explored the market and was satisfied that it was justified in beginning direct discussions with SkyCity to explore a unique proposition.

5.68
Although we regard the process as flawed, we should also make clear that the records show that a great deal of careful work was carried out to understand the market and the different possibilities. We accept that officials were acting in good faith to support decision-making by Ministers on some difficult and controversial matters. The fact that the process was unsatisfactory does not automatically mean that the conclusions reached were unsound.

13: We refer to these five responses to the EOI process as “proposals” and to these five respondents as “proposers”. There were three other responses to Part B. One was purely on funding options, one was an evaluation by Auckland City Council of several sites, and one did not include enough detail to meet the EOI requirements.

page top