Part 4: Service monitoring and evaluation
4.1
In this Part, we discuss how the agencies monitor and evaluate prisoners’ mental health services.
Our expectations
4.2
We expected the agencies to have systems for monitoring and evaluating prisoners’ mental health services and ensuring that these systems contribute to improving services.
Our findings
4.3
Overall, there are some systems in place that allow the agencies to monitor and evaluate prisoners’ mental health services. The Department and the Ministry have identified that these need improvement. Both the Department and the Ministry are implementing systems or have work planned that should contribute to improvements in monitoring and evaluation.
4.4
The Department has some systems for monitoring and evaluating its primary health care services. These include:
- a continuous quality improvement policy;
- an annual audit schedule; and
- Regional Clinical Quality Assurance Adviser positions.
4.5
These systems are relatively recent initiatives. They are still in the early stages of implementation, so we were unable to determine their effectiveness. However, the systems appear to provide a good basis from which to progress.
4.6
The Department’s continuous quality improvement policy requires each health unit to implement a system with regular monitoring of standards, services, and health outcomes. It also intends to use feedback from these activities and clinical reviews to improve services. The Department told us that it used these processes to ensure that services are responsive to, and effective at meeting, the needs of Māori.
4.7
The Department was also introducing an annual audit schedule for its health units. Each month it would audit one area such as medication charts or health promotion. It intended to use the information from these audits to improve practice.
4.8
The Department has Regional Clinical Quality Assurance Advisers who assist with monitoring and evaluation activities. Their focus is on initiating quality improvement activities within the regions. The advisers meet with the Department’s National Clinical Leader every three months to share information and plan work programmes.
4.9
In 2006, the Department did an internal audit of its prisoner health services. The findings of this internal audit initiated a review of the Department’s health service configuration. The Department was consulting on the revised structure at the time of our audit. We encourage any changes that will ensure consistent practice throughout health units, and formalise mechanisms for identifying and implementing best practice in the Department’s health services.
4.10
In April 2007, the Department, the Police, and the Ministry of Justice collaborated on a gap analysis of mental health pathways through the justice sector. They examined their processes for identifying and referring mentally unwell people to the appropriate health provider and identified actions to address the gaps. Tasks for the Department’s Prison Services to improve mental health service provision included:
- producing guidelines for getting information from community health providers;
- working with courts and the Police to ensure that information from their processes accompanies a prisoner when they arrive in prison;
- implementing a new mental health screening tool;
- developing policy to guide management of prisoners with personality disorders and offenders with high and complex needs;
- developing an agreement with the Ministry of Health on the management of acute forensic patients who are waiting for inpatient treatment;
- developing primary mental health assessment and treatment training packages for clinical staff; and
- increasing specialist input for those who do not meet the threshold for inpatient treatment.
4.11
In our view, the gap analysis of mental health pathways through the justice sector is an important piece of evaluative work. It has the potential to improve the delivery of mental health services to prisoners, and our report refers to several pieces of work arising from the gap analysis.
4.12
Given the Ministry’s strategic role in providing prisoners’ mental health services, it has a crucial part in service monitoring and evaluation. The Ministry’s main planning documents, forensic mental health service user censuses and forensic frameworks, provide a good basis for monitoring and evaluating services.
4.13
While the Ministry’s draft forensic framework document identified issues that the Ministry needed to consider in planning services, it did not provide detail on how the Ministry would address these issues. It may be that RFPS plans will provide this detail once they are finalised, but it would be useful for the Ministry to have its own mechanism for monitoring service implementation against its forensic framework.
4.14
In our view, the Ministry needs to translate information from its planning documents into a planned implementation process with actions to measure progress against. We make a recommendation about this in Part 2 (see recommendation 4).
4.15
In its draft forensic framework, the Ministry acknowledged there were few indicators of forensic service performance and recognised a need to establish service guidelines and a monitoring system. The Ministry sought input from RFPS on assessment and reporting criteria.
4.16
In our view, establishing agreed service performance guidelines has the potential to provide an effective mechanism for service monitoring and evaluation. We were pleased to see that the Ministry took a co-ordinated approach in producing its draft forensic framework, seeking views from both RFPS and the Department.
4.17
Each RFPS we visited has a documented quality assurance programme and collected information on referral timeframes and service throughput. Clinical audits, review of clinical pathways, and obtaining consumer feedback are common systems for monitoring and evaluating services.
4.18
As mentioned above, we consider that service performance guidelines will be a useful addition to these activities.