Appendix 1: Our audit methodology

This appendix sets out more detailed information about our audit approach. It discusses:

• our audit fieldwork; and
• the questionnaire completed by general practitioners.

Our audit fieldwork

Our audit fieldwork was carried out from September to December 2008. We spoke to staff in the Ministry's head office and in Work and Income's national office about the management of sickness benefits and invalids' benefits. These staff represented the service development, policy analysis, planning and performance, finance, and research and evaluation functions of the Ministry and Work and Income.

We visited five of Work and Income's 11 regional offices, and two service centres in each of those regions.

In the regional offices and service centres, we spoke to Work and Income's regional health advisors and regional disability advisors, regional and service centre managers, training co-ordinators, employment co-ordinators, staff responsible for purchasing and managing service contracts, and case managers.

Review of documents and records

We examined regional and service centre plans and strategies where these were relevant to sickness and invalids' beneficiaries, reports on performance against targets for numbers of sickness and invalids' beneficiaries, data about the beneficiary population, management reports, and documentation used for case management. We also examined information held about sickness and invalids' beneficiaries in SWIFTT, the system used by Work and Income to record information about recipients of financial assistance and the assistance they receive.

To assess how Work and Income staff were deciding whether people were eligible for a sickness benefit or invalid's benefit, we examined available computer and paper records for a sample of beneficiaries. We also used these records to assess case management processes.

Our findings on how well eligibility and case management processes were working at the time of our audit were based on our examination of the sample, combined with the knowledge we gained from other sources. We have referred to numbers of cases in the report to illustrate the extent of what we found in the sample we examined. The numbers sometimes relate to a subgroup of our sample. Where cases relate to a subgroup, we have given the size of that subgroup. The figures should not be extrapolated to all sickness and invalids' beneficiaries.

Sampling case files

In consultation with the Ministry of Social Development (the Ministry), we chose a sample of 160 sickness beneficiaries and 160 invalids' beneficiaries, using data supplied by the Ministry from beneficiary records current at the end of June 2008. We selected 16 sickness beneficiary cases, and 16 invalid's beneficiary cases for each of the 10 service centres we visited.

We drew the cases from two broad incapacity groups recorded by the Ministry:

• psychological or psychiatric conditions – this group includes people with stress, depression, bipolar disorder, schizophrenia, and intellectual disability; and
• musculoskeletal disorders – this group includes people with back pain, muscle strains, and arthritis.

We chose cases from these groups because these are the two major categories of incapacity among sickness and invalids' beneficiaries, and because both types of incapacity have grown significantly in past years. The growth in mental health disorders has been particularly marked. Stress, bipolar disorder, schizophrenia, and depression have become the main mental disorder growth categories for sickness and invalids' beneficiaries. Sickness and invalids' beneficiaries with mental health disorders and musculoskeletal disorders are also among the most likely to remain on a benefit for the longest time.

These two incapacity groups encompass a wide range of medical conditions, with some short-term and others permanent, and with varying levels of severity and effect on the person's ability to work.

From the sample population provided to us by Work and Income, we chose individual beneficiaries across all age groups, but including:

• people who were recorded as having moved to the sickness benefit or invalid's benefit from another benefit, such as the unemployment benefit or domestic purposes benefit; and
• people who had been on the sickness benefit or invalid's benefit for shorter or longer periods of time.

We included these in order to establish whether any differences in practice existed in such cases.

Because our examination of case files was carried out later in 2008, a small number of our chosen sample group were no longer receiving a sickness benefit or invalid's benefit. In these cases, the Ministry's computer system retained only limited records of their history on that benefit.

Each of the service centres we visited had retrieved files for the chosen sample of beneficiaries. Previous medical certificates were occasionally missing from these. In a small number of cases across the sample, paper files had not been retrieved, so we chose other beneficiary cases in their place.

Questionnaire for general practitioners

We also invited general practitioners (GPs) to complete a questionnaire, because they are the health practitioners who fill out most of the medical certificates that applicants for a sickness benefit or invalid's benefit must provide to Work and Income. The questionnaire asked GPs about this role, the design of the medical certificate, and their communication with Work and Income.

In total, 150 GPs responded to our questionnaire. We analysed responses to each of the questions, as well as their comments. The questions that GPs answered are reproduced below.