Part 3: Improving the timeliness and completeness of data collection

Monitoring how water is used for irrigation.

In this Part, we discuss:

Gaps in the data received from permit holders

The Regulations mean that councils are now receiving considerable amounts of data about how much freshwater is used. Permit holders are required to record freshwater consumption daily11 and send this data to councils at least annually. More timely delivery of data to councils, like that provided through data loggers and telemetry, can help permit holders ensure that they are complying with permit conditions and managing their freshwater consumption. In the region with the largest number of water takes, data is being delivered daily for about 80% of water takes. Overall, permit holders are collecting data mostly as required by the Regulations, and the six councils are receiving this data.

Before the Regulations took effect, the six councils had made preparations to store and manage the information from permit holders. They produced data submission guidelines for permit holders and made these publicly available. The guidelines explain automatic and manual methods for submitting data.

To ensure that permit holders were clear about what was expected of them, the six councils put policies in place to explain their role. For example, one large council had documented what permit holders and data service providers needed to do to ensure that they were collecting and submitting good quality data, and what the council would do if the data was not submitted. The council's document also explains what will happen if data is contaminated or lost.

Councils have worked together and with the irrigation industry, the Ministry for the Environment, and the National Institute for Water and Atmospheric Research to produce standards for installing and validating meters and for collecting and submitting data. In our view, this is an example of good practice and these standards will help improve data quality.

Councils were also working to ensure that they were receiving and analysing data from water meters effectively and making improvements where necessary. Some councils were working through issues with their systems and processes to improve their databases, phase out some data management tools, and standardise templates for data transfer. For example, one council was reviewing its data collection, storage, and reporting system as a sub-system of the council-wide Water Accounting System. This review was intended to improve data quality and usability.

Despite improvements in how data is collected, some problems remain. Measuring water flow through open races and channels is difficult and costly compared with measuring water flow through enclosed pipes. Water meters can fail or send incorrect data. Power supply outages can interfere with the meter system. Meters in remote areas can be difficult to access, portable meters can be unreliable, and sometimes water meters can be affected by issues outside of permit holders' control – such as ants destroying cabling inside telemetry units and electric fences interfering with signals.

Permit holders are responsible for dealing with these matters. There are sometimes unavoidable gaps or unusual patterns in the water meter data because of these problems.

Overcoming problems with data quality

Problems with data quality

Although councils are now receiving data about water use that is required by the Regulations, the problems with submitting data can affect the quality of the information. This is partly because different councils have different requirements for how frequently permit holders need to submit data from water meters. Some councils receive more complete and timely information than others (Example 5).

Example 5
Marlborough District Council phased out manual recordings of water-use data in 2014, anticipating potential problems with human error in the collection and quality of data. The Council was concerned that relying on annual returns put it at risk of losing data for a whole year if there was an unexpected issue or meter failure. To mitigate this, permit holders were required to install automated recording systems called data loggers. Data loggers allow the Council to collect comprehensive information on when and how much water has been taken.

Manually collected or submitted data also causes significant issues with data quality. Although more water meters, such as telemeters, enable automated data collection, there are still many instances where data is collected manually. This can include handwritten meter readings that are submitted electronically and information recorded and submitted in spreadsheets. This can lead to poor quality data, for example, if handwritten meter readings are misread.

Other errors, such as misreading meters, can also contribute to poor-quality data. In our view, manual data collection is an issue that affects councils' administrative costs and the quality of data and how it is used to analyse consumption and monitor and enforce compliance.

Permit holders can contract data hosts, which are private companies that manage the data from water meters. A risk with this arrangement is that councils have less direct control over the supply of data. This can make it difficult for them to build strong business relationships with permit holders and data hosts. When there are issues with the transmission of data from water meters, it can also take longer to resolve the issue. For one council, this issue is particularly prevalent because a large number of data hosts are managing the data for permit holders.

It can also be time consuming for councils to work with data hosts. This can affect the quality of the data received. For one council, if the data-services provider is unresponsive about data issues or losses, then the council can make the permit holder temporarily non-compliant until the issue is resolved.

Effects of data quality issues

In our view, it is important for councils to have high-quality data to ensure timely compliance and identify how freshwater could be used more efficiently. The six councils are working to improve the quality of water meter data. For example, some councils (including the council with the largest number of water permits) are working closely with permit holders and data hosts to build their capability and give them a clearer understanding of issues with data quality and possible solutions. This is good practice. In our view, all councils can learn from this approach.

It is important that all councils outline the specific responsibilities of permit holders and data hosts to ensure that they understand who is responsible for issues with the collection of data, such as when data is missing, lost, or inaccurate. More complete and accurate data will help councils to monitor and enforce compliance with permit conditions.

To further improve data quality, the six councils are using staff to monitor the quality of data. Where data quality is found to be poor, these councils prioritise its improvement. For example, one council we looked at had assigned a team to review water metering data. The team identifies causes of data quality issues, such as equipment failure, and follows up on these issues.

There is also room for all councils to improve their own systems and the quality of their water meter data. The six councils are aware of this and are putting in place clear expectations for data standards and guidelines. This includes clearly explaining procedures to manage data when it is received.

Clearer procedures for data quality have already brought some improvements for a council. When the Regulations were introduced, the council's collection processes were poorly controlled. Spreadsheets were used when purpose-built software had been purchased. This council is now improving its practices for data quality.

Recommendation 1
We recommend that the Ministry for the Environment review the part of the Resource Management (Measurement and Reporting of Water Takes) Regulations 2010 that allows for manual data collection and annual data provision, and work with councils that have oversight of water metering, to ensure that people and organisations holding water permits regularly submit accurate data using automated processes.
Recommendation 2
We recommend that councils continue to work with people and organisations holding water permits and intermediary data service providers to improve the timeliness and completeness of water-use data received.

11: The Regulations provide regional councils with the discretion to approve the keeping of weekly, rather than daily, records of water consumption.