Part 6: After the procurement
6.1
In this Part, we describe:
- what happened after the procurement decision;
- how Callaghan Innovation dealt with concerns raised about the procurement;
- the information about the procurement that entered the public domain; and
- Callaghan Innovation's planned follow-up actions.
What happened after the procurement decision
6.2
Figure 6 sets out a timeline of what happened after Callaghan Innovation made its procurement decision.
Figure 6
Timeline of events after the procurement was complete, from June 2022 to November 2022
6.3
When carrying out a procurement, public organisations are required to act fairly, transparently, and reasonably. This obligation does not disappear after a procurement decision has been made.
How Callaghan Innovation dealt with concerns raised about the procurement
6.4
At a feedback meeting with Callaghan Innovation to close out the procurement on 1 July 2022, Manaaki became concerned about the reasons that had prompted Callaghan Innovation to exclude it from the procurement. At this point, Manaaki did not know that there was a second due diligence report, nor had Callaghan Innovation put the further concerns described in that report to it for comment.
6.5
After that meeting, Manaaki emailed Callaghan Innovation with concerns about the procurement process for the Start-up programme. Manaaki expressed concerns about the process's lack of transparency and lack of natural justice because it had only had a limited opportunity for engagement.
6.6
Manaaki also told Callaghan Innovation about claims that certain parties had made against it on social media. Manaaki was concerned that these claims included information that was similar to matters raised in the due diligence process.
6.7
Callaghan Innovation replied to the concerns that Manaaki raised by stating that it disagreed with the suggestion it did not carry out the procurement process fairly and consistently with the Procurement Rules. Callaghan Innovation told Manaaki that it did not intend to continue conversations about the procurement.
6.8
Throughout July and August 2022, Manaaki was in contact with Callaghan Innovation about Official Information Act requests made by the media and individuals, as well as Official Information Act requests made by Manaaki. Manaaki found out about the two due diligence reports when it was asked for its views on whether the information should be released under the Official Information Act.
6.9
On 12 August 2022, Manaaki wrote to the current Chief Executive expressing its concerns that the contractor had been conflicted in his role. Manaaki also raised concerns that the due diligence reports were unbalanced and unfair to Manaaki and that Callaghan Innovation had failed to give Manaaki a reasonable opportunity to respond to the findings.
6.10
Manaaki felt that Callaghan Innovation had not given it "the full picture" at the 25 May meeting and that the reasons for excluding Manaaki from the procurement had changed from the reasons given at an earlier meeting.
6.11
In response to this letter from Manaaki, Callaghan Innovation rejected the suggestion that there had been a potential conflict of interest or a failure in the procurement process. Callaghan Innovation asked Manaaki to provide evidence to support its allegations about the contractor's conflict of interest, which Manaaki supplied on 17 August 2022.
6.12
After receiving information about the conflict of interest from Manaaki and discussing the matter internally, Callaghan Innovation commissioned EY to review the due diligence process.
EY reviewed the due diligence process
6.13
Callaghan Innovation consulted its external probity advisor about the scope of the review of the due diligence process. It decided to limit the scope to questions that addressed whether the due diligence work needed to be re-done.
6.14
Depending on the review's outcome, Callaghan Innovation would need to consider whether it might need to repeat the due diligence using a different provider. As a result, the scope of the EY review did not specifically consider the impact of the perceived conflict of interest that Manaaki raised.
6.15
The current Chief Executive told us that the review was based on the assumption that there was a conflict of interest and that Callaghan Innovation was seeking to understand whether it could still rely on the due diligence reports to support the procurement decision.
6.16
Callaghan Innovation told us that this was evidenced by the contract with EY, which stated that Manaaki had raised concerns about an undisclosed potential conflict of interest and that Callaghan Innovation could provide the correspondence from Manaaki about the potential conflict and Manaaki's concerns about the process.
6.17
EY told us the review was not based on the assumption of a conflict of interest. EY was aware of the concerns about a potential conflict of interest but was not asked to assess this. EY also confirmed to us that it had talked to Manaaki as part of the review.
6.18
The scope of EY's review was limited to specific questions. Figure 7 sets out a summary of EY's responses to those questions.
Figure 7
Summary of EY's review of Callaghan Innovation's due diligence process for the Start-up programme
| Question | Summary of EY's responses |
|---|---|
| To what extent was the due diligence process conducted by the contractor in accordance with generally accepted good practice for undertaking due diligence of this nature in a government procurement process? | The review concluded that the due diligence process was clearly articulated in section 3.8 of the RFP, including the potential use of third parties to help with the due diligence enquiries. The review compared the due diligence process against the Procurement Rules' Conducting due diligence checks. The due diligence process identified issues through testimonials and references from current and recent customers, through client interviews, and from published articles. These are all valid checks listed in the Procurement Rules' Conducting due diligence checks, and the issues identified in those checks are enough grounds for recommending the respondent not progress. The review concluded that, accordingly, this was consistent with good practice. |
| To what extent was the contractor fair in selecting and interviewing those they did? | The review concluded that eight of the nine interviewees could be linked to the RFP document for Manaaki, either by being listed as a referee or by being referred by another referee, and that there was a valid audit trail of how the contractor engaged each referee. This was not the case for Mr B (the business contact of the Chief Executive). The review concluded that interviewing Mr B was not aligned with good practice because the interviewee had no business relationship with Manaaki. |
| To what extent did the interviewees freely give their information, and does the contractor's report fairly represent that information? | Based on a review of the evidence provided, interviewees gave their information freely. The review concluded that the contractor's report fairly represented the information provided by the interviewees. Interviewees were engaged via phone or email. Statements from interviewees were copied verbatim from the email evidence into the due diligence reports. Where interviews were conducted over the phone, the contractor summarised the discussion in an email and requested that interviewees attested to the accuracy of the summary. This summary was then copied into the due diligence reports. |
| To what extent are the due diligence findings reported by the contractor sufficiently evidence based? | The review noted that the due diligence findings were predominantly based on testimony. The review also noted that, with testimony-based evidence, there is a risk that the accuracy of the judgements is affected by personal experiences of the interviewer or interviewee. The contractor provided evidence to support each testimony, either an original email or subsequent confirmation by email that a verbal testimony was accurate. Adverse testimony provided by initial interviewees was validated by similar testimony from others. The review concluded that, for this reason, the findings were sufficiently evidence based. The review noted that:
|
| How does the due diligence process conducted on Manaaki compare with other shortlisted respondents? | The review confirmed that the process followed for all six shortlisted respondents was consistent. |
6.19
EY concluded that:
… unsatisfactory results in one aspect of due diligence process (for example, testimonials or references from current or recent customers, client interviews, and published articles) is sufficient to exclude a provider from progressing to the next stage of a contract opportunity per Callaghan Innovation's RFP requirements. This is what has occurred in the situation in question; we identified no significant deficiencies to address in the due diligence process performed by [the contractor] that would have changed the outcome of the process and decisions made because of the process followed.
6.20
The final question posed in the EY review was "What, if any, remedial actions are recommended to address any deficiencies of the due diligence process, that have been revealed as part of this review".
6.21
The review identified that the following would have increased "alignment with good practice":
- Manaaki should have had an opportunity to respond to issues raised before the due diligence reports were finalised.
- Manaaki should have been informed about the additional people who were identified and contacted as part of the due diligence.
- The contractor should have interviewed only those people who had a clear link to the details included in the RFP response from Manaaki.
- Reference checks should have been done with both the contractor and a Callaghan Innovation representative present.
- People involved should have been instructed not to make social media posts about a process that was undergoing review.
- Ideally, just one due diligence process should have been conducted. If a secondary process was required, the approach and what it would or would not include should have been clearly described.
After EY's review was completed
6.22
Manaaki did not consider that EY's review resolved its concerns because of the review's limited scope. Manaaki has continued to correspond with Callaghan Innovation, including lodging a formal complaint with Callaghan Innovation under Procurement Rule 50 on 17 November 2022.
6.23
This complaint repeated Manaaki's concerns about the perceived conflict of interest and lack of balance in the due diligence reports. Manaaki also raised concerns about information leaks about the procurement. Overall, Manaaki expressed the view that Callaghan Innovation had not acted in good faith.
6.24
Callaghan Innovation subsequently told us that it had raised some of the allegations about Manaaki's behaviour made in the due diligence reports as a health and safety concern with Worksafe New Zealand. Worksafe New Zealand decided not to intervene. Callaghan Innovation did not tell Manaaki that it had made this referral.
6.25
In a letter to Callaghan Innovation in December 2022, Manaaki explained that Callaghan Innovation's actions in the procurement had put great strain on the personal well-being of its staff and on its business.
6.26
Callaghan Innovation responded to the Rule 50 complaint made by Manaaki on 26 January 2023. The response set out its views on the alleged conflict of interest, the purpose of EY's review, and its reasons for sharing the due diligence reports. Callaghan Innovation also explained what it had done to investigate the leak of information to the public.
6.27
Callaghan Innovation's response to Manaaki identified areas for improvement. These were:
- identifying where there could be a potential conflict of interest so Callaghan Innovation can take action to address any concerns;
- including details of any third parties Callaghan Innovation anticipates using to support a procurement process in the tender documents;
- endeavouring to include the opportunity for tenderers to respond to issues raised during a due diligence process before it has been finalised;
- considering, when Callaghan Innovation cannot share a draft report with a tenderer, offering the tenderer the option to review relevant meeting minutes and any formal written responses that address due diligence findings;
- implementing training on appropriate information sharing between public organisations and the obligations of confidentiality;
- making it clearer to tenderers what Callaghan Innovation might do with due diligence findings, including who it might share information with and for what purposes;
- explicitly informing respondents of any intention to share due diligence findings before sharing them; and
- including in any future due diligence reports a disclaimer that the report has been produced for a specific procurement process with predefined evaluation criteria and that readers should complete their own research to obtain a better understanding of the business in question.
6.28
Callaghan Innovation also published a code of conduct to help people working with founders to understand what good behaviour is and how to identify and address unacceptable behaviour. There is a corresponding guide for founders.23
Our observations about Callaghan Innovation's response to the concerns raised
6.29
In its responses to Manaaki, Callaghan Innovation maintained that it had complied with the Procurement Rules. It appeared that, initially, Callaghan Innovation was unwilling to engage with the matters that Manaaki had raised.
6.30
For example, when Manaaki first raised the contractor's potential conflict of interest, Callaghan Innovation asked Manaaki for evidence before it would consider the matter further. This was despite the procurement team knowing about the contractor's previous work for Company A.
6.31
There has also been a lack of transparency in Callaghan Innovation's responses to Manaaki. Callaghan Innovation's procurement team never explained to Manaaki that it knew about the contractor's previous work for Company A and had decided that it was not a conflict.
6.32
Callaghan Innovation has continued to publicly assert there was no conflict because the contractor did not disclose one, even after the contractor admitted that he was still under retainer with Company A at the time he carried out the due diligence, as explained in paragraphs 4.39-4.40.
6.33
In Callaghan Innovation's response to the Procurement Rule 50 complaint, Callaghan Innovation explained that the conflict "was not known more generally within [Callaghan Innovation] (than the Chief Executive) until Manaaki brought the allegations to our attention in August 2022". This is not accurate (see paragraphs 4.25-4.36).
6.34
There is an inconsistency between how Callaghan Innovation has represented the purpose and scope of EY's review of the due diligence process to Manaaki when it raised concerns, and the documented scope of the review. EY's scope of work specifically excluded consideration of the conflict of interest that Manaaki had raised.
6.35
However, Callaghan Innovation asserted to Manaaki that "We have not ignored the alleged conflict of interest. We have actually taken it as a given." This statement is difficult to reconcile with EY's statement to us that it did not conduct the review on that basis.
6.36
By deciding to limit the scope of the EY review, Callaghan Innovation has left the substantive issue of the conflict of interest (and associated concerns about bias) unresolved. This has led to questions about the reliability of the due diligence findings and shows how perceived conflicts of interest can erode trust and confidence in a process.
6.37
Callaghan Innovation has asserted to Manaaki and to the public that, based on the findings of EY's review, its due diligence process was aligned with good practice. In this case, EY assessed generally accepted good practice by reference to the Procurement Rules' guidelines on conducting due diligence checks. Although these guidelines give practical advice about planning for, and carrying out, due diligence, they should not be read in isolation from the rest of the Procurement Rules.
6.38
Using a narrow definition of good practice means, for example, that this aspect of the review does not address (and was not asked to address) whether the due diligence process incorporated other requirements of the Procurement Rules, such as balance and a process that follows natural justice and that is procedurally fair (although EY raised this as an area for improvement). These were specific concerns that Manaaki had raised and were aspects that we have concluded were missing from the due diligence process.
6.39
Therefore, without understanding the scope of EY's review, Callaghan Innovation's claim that the due diligence was aligned with good practice could be potentially misleading.
6.40
In its January 2023 response to the complaint that Manaaki made under Procurement Rule 50, Callaghan Innovation identified some areas for improvement. However, this response and Callaghan Innovation's earlier responses to Manaaki do not appear to consider the consequences for Manaaki of the outcome of the procurement process and the due diligence reports being made public. We discuss this further below.
Information about the procurement entered the public domain
6.41
A notable feature of the procurement for the Start-up programme was how much information about the procurement has been in the public domain. We have been told that people in the innovation sector had been raising concerns about unethical behaviour towards founders for some time. From December 2021 on, there had been social media posts referring to "bad actors" in the innovation "ecosystem", which Manaaki believed were aimed at it.
6.42
On 7 June 2022, NBR published an article saying that "while other applicants are progressing to the next round of the programme, Manaaki and [We Are Indigo] are facing further questions from [Callaghan Innovation]". The article quoted several people who the contractor had interviewed.
6.43
Callaghan Innovation told us that it had contacted the contractor about the article and that the contractor had confirmed that he had not had any contact with NBR. Callaghan Innovation told us that it has no ability to control the actions of other parties.
6.44
NBR published a follow-up article on 1 September 2022. Both the contractor and a Callaghan Innovation Board member commented on the article on social media. In his post, the contractor described the due diligence as a "robust investigation, fair and thorough review".
6.45
At Callaghan Innovation's request, both comments were taken down. At the same time, the current Chief Executive reminded Callaghan Innovation staff that they should not comment on or "like" any posts about the Start-up programme procurement or its outcome.
6.46
On 14 September 2022, one of the interviewees, Mr B, who Manaaki identified as hostile towards it, posted a copy of the statement he had made as part of the due diligence process on social media. There were further media articles in October 2022.
6.47
We accept that Callaghan Innovation has no influence over what media choose to report and who chooses to speak to the media. However, Callaghan Innovation staff, the contractor, and a Board member commenting on social media posts about the procurement indicates a lack of understanding within Callaghan Innovation of its obligations about confidentiality and the obligations it should place on its suppliers.
6.48
Once alerted to the issue, Callaghan Innovation took steps to respond to the social media postings and advised those involved about Callaghan Innovation's expectations about public comment on procurement matters.
6.49
In November 2022, an anonymous Gmail account sent a redacted copy of the two due diligence reports to an unknown number of people, including the media and our Office.24 This prompted further social media comment.
Callaghan Innovation investigated the unauthorised disclosure of the due diligence reports
6.50
When Callaghan Innovation was told that the due diligence reports were in the public domain, it commissioned a review to identify whether the leak came from Callaghan Innovation. Callaghan Innovation told us that, in parallel, it worked with Google to report and shut down the anonymous email account that had sent the reports and worked with Manaaki to report the matter to NetSafe and Cert NZ.
6.51
Callaghan Innovation's first step was to do a forensic "hygiene" review of its IT system to see whether anybody had downloaded or forwarded the report.
6.52
Callaghan Innovation told us that it found no evidence within its IT systems of anyone at Callaghan Innovation leaking information. However, the review identified that the due diligence reports had been uploaded to the Board's electronic Board papers system. A review of this system revealed that it did not have the functionality to provide an audit trail of documents accessed, downloaded, or forwarded.
6.53
Callaghan Innovation then sent a letter to all current and former employees and Board members seeking written assurance that they had not shared confidential information. Most people provided this or separate assurance by email that they had not leaked information.
6.54
The current Chief Executive sought advice from KPMG about what further work Callaghan Innovation could do to find out whether the reports were leaked from within Callaghan Innovation. KPMG validated the steps Callaghan Innovation had taken and identified further forensic work that it could do.
6.55
However, KPMG advised that it was unlikely that further work would be successful. We were told that, on that basis, the chairperson of the Board decided not to continue pursuing the matter.
6.56
It is deeply concerning that the due diligence reports found their way into the public domain. Appropriately maintaining the confidentiality of information provided to public organisations goes to the heart of trust and confidence in procurement processes and the public sector more generally. This is illustrated by our findings in Parts 4 and 5.
6.57
No-one has been able to identify where the leaks originated from, but Callaghan Innovation commissioned the reports and was ultimately responsible for their safekeeping.
6.58
As we noted in Part 5, Callaghan Innovation decided to share the reports with two other public organisations. We consider that, when it did so, Callaghan Innovation ought to have conveyed the sensitivity of the reports and the need to keep those reports secure. This did not happen.
6.59
That said, after the reports were emailed out from the anonymous email account, Callaghan Innovation took reasonable steps to try to identify the source of the leak and to seek assurances from staff that they had not disclosed the reports.
Callaghan Innovation's planned follow-up actions
6.60
In response to an offer to Callaghan Innovation to comment on a draft of this report, Callaghan Innovation advised us that it has reflected on the process and what changes it might make for the future.
6.61
In addition to the proposed improvements that it identified in its response to Manaaki's Rule 50 procurement complaint (see paragraph 6.27), Callaghan Innovation told us about several changes to its systems and processes that it plans to make. We summarise these in paragraphs 6.64-6.72.
6.62
We commend Callaghan Innovation for reflecting on what changes to its processes it could make, and we will be interested in how this work progresses.
6.63
Callaghan Innovation started making these changes in December 2022, when it released a Founder Code of Conduct that the five Start-up programme providers are required to have in place. This guidance is available for the founder and start-up sector to use more broadly.
Counselling and advice
6.64
To support founders, Callaghan Innovation will cover the cost of one-to-one guidance for any start-up founder who has experienced inappropriate behaviour in the start-up sector, regardless of whether they are a Callaghan Innovation customer or a participant in the Start-up programme.
6.65
Specialist external coaches who can help founders to talk about their experience, receive advice, and decide on next steps will provide this confidential service. If a founder requests it, the service can follow a tikanga Māori approach.
6.66
Callaghan Innovation told us that it is planning a free mental-health counselling service delivered by counsellors who have experience working with founders.
Conflict of interest tools and training
6.67
Callaghan Innovation intends to review its approach to conflicts of interest and will commission an external auditor to conduct this review. Callaghan Innovation will implement conflict of interest training for all staff to help identify where perceptions of conflicts of interest might arise.
6.68
Callaghan Innovation has begun using New Zealand Government Procurement's conflict of interest tool, which provides guidance to users (including third parties) to identify whether they have any relevant conflicts of interest.
Procurement review and changes to due diligence processes
6.69
Callaghan Innovation said that it plans to review its procurement processes and seek advice from MBIE and New Zealand Government Procurement on how to do this.
6.70
Callaghan Innovation has implemented a due diligence guide for its procurements.
Refreshing leadership culture and governance charter
6.71
We have been told that senior leadership at Callaghan Innovation has committed to re-emphasise the following to the organisation:
- Callaghan Innovation's senior leadership encourages being presented with a range of views.
- Advice put to the senior leadership should be adequately documented, along with associated consideration of risk.
- The senior leadership's response to the advice should be recorded.
Information sharing and confidentiality
6.72
Callaghan Innovation said that it will work with MBIE as its monitoring agency and the Public Service Commission on information-sharing practices in procurement processes. It said that it hopes that this will lead to a standard for future procurement processes being developed.
23: See "Code of Conduct and Guide for Founders", at callaghaninnovation.govt.nz.
24: Greive, Duncan (2022), "The troubling backstory and new legal chaos engulfing We Are Indigo", at thespinoff.co.nz.