Part 3: Improving the timeliness and completeness of data collection

3.1
In this Part, we discuss:

• gaps in the data that the six councils are receiving; and
• problems with data quality.

Gaps in the data received from permit holders

3.2
The Regulations mean that councils are now receiving considerable amounts of data about how much freshwater is used. Permit holders are required to record freshwater consumption daily¹¹ and send this data to councils at least annually. More timely delivery of data to councils, like that provided through data loggers and telemetry, can help permit holders ensure that they are complying with permit conditions and managing their freshwater consumption. In the region with the largest number of water takes, data is being delivered daily for about 80% of water takes. Overall, permit holders are collecting data mostly as required by the Regulations, and the six councils are receiving this data.

3.3
Before the Regulations took effect, the six councils had made preparations to store and manage the information from permit holders. They produced data submission guidelines for permit holders and made these publicly available. The guidelines explain automatic and manual methods for submitting data.

3.4
To ensure that permit holders were clear about what was expected of them, the six councils put policies in place to explain their role. For example, one large council had documented what permit holders and data service providers needed to do to ensure that they were collecting and submitting good quality data, and what the council would do if the data was not submitted. The council's document also explains what will happen if data is contaminated or lost.

3.5
Councils have worked together and with the irrigation industry, the Ministry for the Environment, and the National Institute for Water and Atmospheric Research to produce standards for installing and validating meters and for collecting and submitting data. In our view, this is an example of good practice and these standards will help improve data quality.

3.6
Councils were also working to ensure that they were receiving and analysing data from water meters effectively and making improvements where necessary. Some councils were working through issues with their systems and processes to improve their databases, phase out some data management tools, and standardise templates for data transfer. For example, one council was reviewing its data collection, storage, and reporting system as a sub-system of the council-wide Water Accounting System. This review was intended to improve data quality and usability.

3.7
Despite improvements in how data is collected, some problems remain. Measuring water flow through open races and channels is difficult and costly compared with measuring water flow through enclosed pipes. Water meters can fail or send incorrect data. Power supply outages can interfere with the meter system. Meters in remote areas can be difficult to access, portable meters can be unreliable, and sometimes water meters can be affected by issues outside of permit holders' control – such as ants destroying cabling inside telemetry units and electric fences interfering with signals.

3.8
Permit holders are responsible for dealing with these matters. There are sometimes unavoidable gaps or unusual patterns in the water meter data because of these problems.

Overcoming problems with data quality

Problems with data quality

3.9
Although councils are now receiving data about water use that is required by the Regulations, the problems with submitting data can affect the quality of the information. This is partly because different councils have different requirements for how frequently permit holders need to submit data from water meters. Some councils receive more complete and timely information than others (Example 5).

3.10
Manually collected or submitted data also causes significant issues with data quality. Although more water meters, such as telemeters, enable automated data collection, there are still many instances where data is collected manually. This can include handwritten meter readings that are submitted electronically and information recorded and submitted in spreadsheets. This can lead to poor quality data, for example, if handwritten meter readings are misread.

3.11
Other errors, such as misreading meters, can also contribute to poor-quality data. In our view, manual data collection is an issue that affects councils' administrative costs and the quality of data and how it is used to analyse consumption and monitor and enforce compliance.

3.12
Permit holders can contract data hosts, which are private companies that manage the data from water meters. A risk with this arrangement is that councils have less direct control over the supply of data. This can make it difficult for them to build strong business relationships with permit holders and data hosts. When there are issues with the transmission of data from water meters, it can also take longer to resolve the issue. For one council, this issue is particularly prevalent because a large number of data hosts are managing the data for permit holders.

3.13
It can also be time consuming for councils to work with data hosts. This can affect the quality of the data received. For one council, if the data-services provider is unresponsive about data issues or losses, then the council can make the permit holder temporarily non-compliant until the issue is resolved.

Effects of data quality issues

3.14
In our view, it is important for councils to have high-quality data to ensure timely compliance and identify how freshwater could be used more efficiently. The six councils are working to improve the quality of water meter data. For example, some councils (including the council with the largest number of water permits) are working closely with permit holders and data hosts to build their capability and give them a clearer understanding of issues with data quality and possible solutions. This is good practice. In our view, all councils can learn from this approach.

3.15
It is important that all councils outline the specific responsibilities of permit holders and data hosts to ensure that they understand who is responsible for issues with the collection of data, such as when data is missing, lost, or inaccurate. More complete and accurate data will help councils to monitor and enforce compliance with permit conditions.

3.16
To further improve data quality, the six councils are using staff to monitor the quality of data. Where data quality is found to be poor, these councils prioritise its improvement. For example, one council we looked at had assigned a team to review water metering data. The team identifies causes of data quality issues, such as equipment failure, and follows up on these issues.

3.17
There is also room for all councils to improve their own systems and the quality of their water meter data. The six councils are aware of this and are putting in place clear expectations for data standards and guidelines. This includes clearly explaining procedures to manage data when it is received.

3.18
Clearer procedures for data quality have already brought some improvements for a council. When the Regulations were introduced, the council's collection processes were poorly controlled. Spreadsheets were used when purpose-built software had been purchased. This council is now improving its practices for data quality.

11: The Regulations provide regional councils with the discretion to approve the keeping of weekly, rather than daily, records of water consumption.