Part 4: Career Services

Central government: Case studies in reporting forecast performance information.

Background

4.1
Career Services is a Crown entity. It was established under the Education Act 1989 to maintain and make available information and advice about occupations and post-compulsory education and training. It receives revenue of more than $16 million each year. More than 90% of this funding comes from Vote Education as a Non-Departmental Output Expense.

The SOI explained the importance of good decision-making about education, training, and career choices

4.2
Its 2010/11 SOI was a compact document that used an inviting layout and provided several useful diagrams.

Strategic context

4.3
The SOI explained the importance of good decision-making about education, training, and career choices, noting that the recession had made good information and advice to support such choices more important than ever. Figure 1 reproduces Career Services' description of its strategic context.

Figure 1
Strategic context for Career Services

Raising New Zealanders' awareness of where to get career decision-making support is an important step in giving everyone the opportunity to maximise their potential. Our ongoing challenge is to continue to extend our reach by making the best use of our low-cost web-based and phone-based services, while ensuring that personal assistance is available to the groups and individuals who need it most. Young people in particular need support to navigate through the range of learning and career options open to them. Career Services is committed to supporting government initiatives such as the Youth Guarantee, to ensure young people are making well-informed decisions about their futures. We make the most of the latest technologies to deliver services with particular appeal to young people (such as personalised online My Career Spaces and our texting services). We also focus on building the career skills of parents and teachers so they can support young people as they map out their career futures. As the country emerges from the recession, it becomes even more important to have a highly skilled workforce, to help lift productivity and contribute to economic growth. It is vital, too, that those affected most by current unemployment levels – such as young people, Māori communities and Pasifika families – are not left behind. The first step in raising skill levels in the workforce of today and tomorrow lies in the ability to make good decisions about work and learning. New Zealanders need to know how to find independent and accurate information about jobs, education and training. As importantly, they need to make well-informed decisions that take their circumstances, skills and interests into account. They need to develop the job resilience that will help them avoid long periods of unemployment.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 2.

4.4
Career Services set out its own place within the wider public sector work, education, and skills development spheres, showing how it contributes to Government priorities and actions (see Figure 2).

Career Services set out its own place within the wider public sector work, education, and skills development spheres, showing how it contributes to Government priorities and actions

Figure 2
How Career Services contributes to Government priorities and actions

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 6.

4.5
Career Services presented a brief environmental scan to set out the significant issues and challenges affecting the services it provides and its responses to those challenges.

Figure 3
Career Services' environmental scan

TABLE 1: HOW OUR PRIORITIES AND SERVICES RESPOND TO THE ENVIRONMENTAL ISSUES
Government Priorities
Issue	Response	Our Strategic Priorities	Our Services
Lifting public sector performance	Continuing to generate more innovative and cost-effective ways of delivering services	Integrated self-help for all Evidence	channelling users primarily to web-based self-help and 0800 advice line services – the most cost-effective ways of reaching the greatest number of users delivering more group guidance exploring new services such as online guidance, virtual networking and wikis; seeking innovative ways to harness emerging technologies.
Ensuring New Zealand has the skills it needs	Ensuring people are aware of our services Working with young people and their influencers, as well as schools, tertiary providers and ITOs	Awareness Young people and their influencers Partnerships	providing freely available online tools to help people match their skills and interests to jobs or school subject choices enabling web users to click through to job vacancies and watch video clips about particular occupations building others’ capability (such as parents and teachers) to support young people to make learning and career decisions.
Youth Guarantee	Helping young people identify career pathways; building the capability of schools, parents and whānau to support young people	Young people and their influencers	providing group training sessions to build the skills of parents and whānau, teachers, career professionals and others to help guide young people through learning and career decisions providing advice and support to schools to help build effective career education programmes for students.
Māori success	Further developing our whānau-based approach, based on what we have learnt from the Whānau Decision-making Project	Young people and their influencers Evidence	using the results of the Whānau Decision-making Project to improve our support for young Māori making career decisions providing group guidance sessions for Māori secondary school students to help them develop their career decision-making skills and awareness of education and work opportunities.

Demographics
Issue	Response	Our Strategic Priorities	Our Services
Growing, ageing population	Web-based and phone-based services for people of all ages	Integrated self-help for all	all age service, personalised and differentiated to meet people’s individual needs.
Increasing ethnic diversity	Tailoring resources and services to meet the needs of different groups	Integrated self-help for all Partnerships	providing targeted sections of the website to meet the needs of particular groups, such as Māori, Pasifika, migrants and refugees. providing group guidance sessions (and smaller peer support sessions) for migrants and refugees to inform them about New Zealand workplaces and employers’ expectations.
More young people	Reaching more young people through group work, capability-building and tailored web resources	Young people and their influencers	providing texting, webchat and online tools (such as CV 4 Me) aimed at helping young people with learning and career decisions; personalised online My Career Spaces exploring virtual networking and wikis that appeal to young people working with the secondary school, tertiary and industry training sectors to ensure students can access the information and advice they need to make

Economy
Issue	Response	Our Strategic Priorities	Our Services
Economic Growth	Helping people maximise their potential	Integrated self-help for all	up-to-date web-based database of occupations that reflects changes to job descriptions, salaries and labour market trends working closely with government agencies, both at the national and regional level to support relevant actions in the labour and education sectors, such as the Re-Start package.
Emerging from a recession; addressing national debt	Tailoring services to people impacted by unemployment Finding ways to work smarter	Integrated self-help for all Evidence	tailoring sections of our website to workers facing job uncertainty and redundancy providing services to workers made redundant, through our 0800 advice line with additional guidance available from career consultants for those needing extra support working with businesses (for example, to support those facing closure or redundancies) supporting local employers and staff at the local level.
Increasing unemployment	Tailoring services for people facing unemployment	Integrated self-help for all Partnerships	working with the Ministry of Social Development to support initiatives to assist businesses and workers facing closure or downsizing working closely with the Department of Labour to ensure our information about occupations and local labour markets is as up-to-date as possible.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 12-14.

Specifying and presenting the outcomes framework

The previous year's result for each outcome measure was usefully set out as comparative information

4.6
Career Services' narrative discussion made its overall purpose and direction clear. A Results Framework diagram (see Figure 4) identified elements, including Career Services' contribution to the overall education sector outcome and government priorities, and set out its primary outcome – "Many more New Zealanders can make sound career decisions".

4.7
The Results Framework diagram also set out Career Services' four outcome statements (for example, "More people using our web-based service can match their skills and interests to jobs") that were the immediate and direct outcomes (or the impacts) of Career Services' outputs.

Figure 4
Career Services' results framework diagram

Figure 4: Career Services' results framework diagram.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 16.

Main measures and targets for outcomes and impacts

4.8
The SSP contained 12 outcome measures for the four outcome statements. These sat alongside the service performance measures for each output. The previous year's result for each outcome measure was usefully set out as comparative information (see Figure 5). The measurement method for all the measures is an independent annual evaluation.

Figure 5
Main measures and targets for outcomes and impacts and their relationship to outputs

Results/Measures	Target 2010/11	Target 2012/13	How it will be measured
Web-based services We provide web-based information and tools (such as quizzes and wizards) to help people understand their skills and ambitions, find out more about the labour market, explore training and job opportunities and develop career plans and CVs.
Outcome
% of people using web-based self-help resources who can identify a good match between personal skills and interests and career/job choice	88% [achieved 87% in 2008/09]	90%	Through an independent annual evaluation
% who have made a career-related decision aided by our services	70% [new measure]	80%	Through an independent annual evaluation
Quantity measures
Number of visits from New Zealand-based people accessing web-based career information	2,500,000 [2,150,488 in 2008/09]	2,750,000	Google Analytics: website visits from NZ-based computers
Number of My Career Space registrations	20,000 new registrations [new measure]	50,000	My Career Space
Quality measures
% of people accessing online information who find it useful and meaningful	80% [79% said it was useful in 2008/09]	85%	Through an independent annual evaluation
% of people who say website is easy to use and up-to-date	75% [72% said it was easy to find their way around in 2008/09]	80%	Through an independent annual evaluation

Notes on Measures Outcome: We would expect some small improvements over time. Quantity: By way of comparison, the Retirement Commissioner's financial planning website, www.sorted.org.nz, received 1.2 million visits during the whole of 2008. Individuals may visit our website more than once and unfortunately it is not currently possible to determine the number of unique visitors. We would expect the number of visits to continue to increase over the next two to three years, but for numbers to eventually reach a maximum level that would be sustained over time. The number of My Career Space users is a new measure to gauge the use of this recently developed resource for people to save their career information online. Over time we will use the My Career Space information to learn more about our users and improve our services. Quality: The words "Up-to-date" have been added for 2010/11.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 30.

In our view, the focus on impacts as a layer that is close to the delivery of the output is useful and appropriate for Career Services

4.9
The main measures are close in focus to the delivery of the output and also focus on the perceptions of Career Services' clients. For example, the measure "% of people using web-based self-help resources who can identify a good match between personal skills and interests and career/job choice" focuses on the clients of Career Services.

4.10
In our view, the focus on impacts as a layer that is close to the delivery of the output is useful and appropriate for Career Services, given its role within the wider public sector work, education, and skills development spheres, its organisational resources, and its overall size and scale.

4.11
The SOI also describes a three-year evaluation programme that includes an evaluation of key assumptions, such as how web-based services help people to identify a good match between personal skills and interests, and career/job choices (see Figure 6). Immediately after the three-year evaluation programme, Career Services reported some initial evaluation results that are useful for understanding the impact that its activities have had to date.

Figure 6
Information about Career Services' evaluation strategy

Key Indicators We have developed an outcomes framework and indicator matrix that help us monitor the results we are achieving. The diagram below shows some key indicators in each of the four Results-based accountability categories. Further work on measures is being undertaken jointly with other agencies.
Three Year Programme Our three year external evaluation programme is aimed at ensuring our services are meeting users' needs. The programme is based on: continuing to build on a core of information about the effectiveness of our services, whether it meets people's needs and how we can improve services examining users' total experience with our integrated services: web, phone, face-to-face each year, having more in-depth investigations into results for a particular client group (e.g. young Māori) or a particular product (eg. texting services). Each year, we will evaluate: Web-based services how well the web-based services help people to identify a good match between personal skills and interests and career/job choice whether people accessing web-based services find them useful and meaningful and easy to use. Assisted advice whether people accessing our 0800 Advice Line know which steps to take next and find that the advice meets their needs. Guidance services how well group sessions enable students to gain a greater understanding of how to make effective learning and career decisions how well group sessions enable migrants and refugees to understand how to make effective career decisions in a New Zealand context how well one-to-one guidance helps people identify relevant learning and career pathways. Capability-building services how well the skills building sessions prepare people to help others with career decisions whether assistance to schools meets their needs. At the end of each year, we re-examine evaluation priorities in light of survey results and new service developments. For 2009/10, we focused particularly on the whānau decision-making project. For 2010/11, we will focus on secondary school students. We will: follow up with a group of young Māori and Pasifika students who attended group guidance sessions to find out what progress they have made with respect to their career plans, six months after the guidance session repeat the Career Decision-making Abilities evaluation that was conducted in 2008, tracking secondary students as they exit school, across a range of factors related to their career-related decisions.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 22.

Potential improvements

4.12
Career Services also states its intention to measure its contribution to government priorities for the use of Career Services by target groups (such as young people, and young Māori and their whānau). Comparative or baseline information on these matters could usefully be included in the SOI.⁸

Linking outcomes to impacts to outputs

4.13
The Results Framework diagram also sets out the relationships between Career Services' outputs and impacts, and the target groups and drivers that shape its output delivery.

4.14
After Career Services' brief environmental scan of the significant issues and challenges affecting its services, the SOI sets out Career Services' intended response. These intended responses link to Career Services' model for delivering services to individuals and groups, and its improvement intentions for output service effectiveness and efficiency.

4.15
The SOI (see Figure 7) notes that Career Services ensures value for money for New Zealanders by, for example:

channelling people, as far as appropriate, to the lower-cost web-based and telephone-based services; and
gathering evidence about what people want and which aspects of its services work best.

Figure 7
Career Services' integrated delivery model for services to individuals and groups

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, page 19.

Specifying outputs and output classes

4.16
Career Services has one output class – Provision of Information and Advisory Services. It identifies four outputs within the output class, and provides indicative costs for each output. Figure 8 describes the services delivered within each output.

Figure 8
Description of the services delivered within each of Career Services' outputs

Web-based services Our self-help web-based resources provide people with: up-to-date, comprehensive and independent job, course and training, and labour market information good practice career education resources for teachers and careers advisers online career tools such as ‘CV 4 Me' (a CV Wizard), skill matching quizzes and ‘My Career Space' (where people can save their CVs and other career-related information) We also offer live webchat, so anyone in New Zealand can get instant online answers to their questions about courses and occupations. Assisted advice Our advisory services include: our 0800 advice line, which provides general career information; advice about careers, jobs and training courses; needs assessment; and referral to in-depth guidance if callers' needs can't be met in other ways a texting service for requesting career information (external evaluation results showed that two-thirds of participants surveyed would not have contacted Career Services if the texting service didn't exist. follow-up calls to check whether people need further information or help, an opt-in appointment reminder service and a tips of the day service. CV feedback for young people who have used our online CV 4 Me tool. Guidance services We offer: customised career planning in a group setting to secondary school Māori and Pasifika students, prospective tertiary students, migrants and refugees and other groups. individual face-to-face career planning assistance to targeted people, including young Māori and Pasifika people, workers who have been made redundant and recent migrants telephone guidance, so people can access in-depth personal career guidance without having to visit one of our offices. This makes our services more accessible to people who live in remote locations, have care-giving responsibilities and/or have a disability (this service is also available outside normal office hours) online access to a career consultant. Capability building services We work with a range of organisations and groups to build their ability to support others – especially young people – with learning and career decisions. In particular, we provide: customised advice and professional development assistance to schools. customised advice to parents, whānau and others who support young people to make career decisions.

Source: Career Services (2010), Statement of Intent 2010 to 2013, Wellington, pages 3-4.

4.17
The grouping of these outputs into one class seems appropriate to us because all outputs relate to providing information, guidance, and advice. All the outputs are externally focused and cover the significant goods and services that Career Services provides.

The important feature of Career Services' output performance information is that the focus and weight of the measures is on relevant dimensions of the quality of services

Performance measures and targets for outputs

4.18
Career Services identifies output measures as either quantity or quality measures. All the measures include specific criteria or means for measuring performance. Although quantity measures have a demand dimension to them (such as the number of website visits), this information is relevant to Career Services' effectiveness and efficiency improvement intentions.

4.19
The important feature of Career Services' output performance information is that the focus and weight of the measures is on relevant dimensions of the quality of services (in particular, whether services meet people's needs).

8: "Every young person has the skills and qualifications to contribute to their and New Zealand's future" and "Māori enjoying success as Māori".

page top