DES Star Ratings Supporting Document
This document provides supporting information for DES Star Ratings, including Q&As.
Question
How is commencing participants in Ongoing Support rewarded in the 2013 DES Performance
Framework?
Answer
There is a perception from a small number of providers that there is a disadvantage in the DES Star
Ratings model in commencing a participant in Ongoing Support. Analysis of this issue confirms that
there is no disadvantage to commencing a participant in Ongoing Support.
The introduction of the 52 Week Sustainability Indicator combined with changes to the Ongoing Support
performance measure in the 2013 DES Performance Framework ensure that appropriate incentives are
in place for providers to commence participants who require Ongoing Support into the phase. In
addition, the use of regression in determining expected performance takes into account the amount of
time a participant has had in Ongoing Support, rewarding providers for sustaining participants in longterm jobs through Ongoing Support.
52 Week Sustainability Indicator
The 52 Week Sustainability Indicator gives providers a strong incentive to place participants who require
Ongoing Support into the Ongoing Support phase. It is important that providers place participants who
require Ongoing Support into the phase, after they have achieved a 26 Week Employment Outcome, so
as to not risk their performance against the 52 Week Sustainability Indicator. The 52 Week Sustainability
Indicator has a weighting of 10 per cent which potentially can have a greater impact on a provider’s final
Star Rating than will small changes in the Ongoing Support measure performance.
2013 definition Ongoing Support performance measure
If a participant remains in the Ongoing Support phase until the achievement of the 52 Week
Sustainability Indicator and has completed at least one Ongoing Support Assessment that recommends
further Ongoing Support*, the participant will be included in the denominator of the Ongoing Support
performance measure.
Under the 2013 definition Ongoing Support performance measure, if a participant is exited as an
Independent Worker (after they are included in the denominator) then their employment status will be
verified six week after the exit.
Providers would be risking losing credit in the Ongoing Support performance measure if they exit a
participant as an independent worker who actually requires it as the participant may fall out of
employment in the next six weeks.
* One Ongoing Support fee (status of pending or approved) required for Flexible Ongoing Support participants
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 1 of 7
Controlling for time in Ongoing Support with Regression
The use of statistical regression to control for the time a participant has spent in Ongoing Support also
ensures that there is no disadvantage in the provision of long-term Ongoing Support to participants who
require the assistance. The relationship between time in Ongoing Support and average expected
outcome rates is shown in the Figure 1 below, based on the modelling of the 2013 definition Ongoing
Support measure.
Figure 1: Actual versus expected performance – Duration in Ongoing Support (DES-ESS)
Providers who keep participants in Ongoing Support for longer periods of time have a lower expected
performance, reflecting the relative challenge in maintaining a participant in employment for a longer
period of time. Providers who keep a participant in the Ongoing Support for a shorter period of time
(because they exit them as an Independent Worker for example) have a higher expected performance.
Noting, as mentioned before, an Independent Worker exit does not ensure a guaranteed outcome in the
Star Ratings with the addition of the employment verification test.
For example, if a provider that had assisted 10 participants in the Ongoing Support Phase was to exit
them all as Independent Workers after just one day since they entered the Ongoing Support
denominator then, on average, their expected performance rate would be 99 per cent. This is in effect
the rate that the average provider would achieve with participants in Ongoing Support for one day – one
in one hundred would lose their employment after one day. If all 10 participants did maintain their
employment six weeks after their Independent Worker exit (i.e. their employment status was verified),
then their actual outcome rate would be 100 per cent. Their actual versus expected score would be
about average, at 1.01 (100 over 99).
However, if just one participant was not to maintain their employment until the six week mark then
their actual outcome rate would be 90 per cent and they would have a score of 0.91 (90 over 99).
Already this provider is operating at close to 10 per cent below their expected performance.
Similarly, if the provider did not exit any participants as Independent Workers then after 500 days
(approximately one year and four months) they would have an expected outcome rate of 69 per cent.
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 2 of 7
This is what the average provider would have achieved with participants who have had 500 days in the
Ongoing Support denominator and have maintained their employment. To perform better than average,
the provider would need to maintain seven out of their 10 participants in Ongoing Support over the
period of 500 days in the denominator.
Question
How will my Star Ratings be impacted by the removal of past performance data associated with ceased
contracts?
Answer
As the Star Ratings is a relative measure of performance, removal of ceased contracts will have flow on
consequences to the Star Percentage and Star Ratings of continuing contracts. Below are the impacts of
the most recent removal of ceased contracts.
The below analysis does not take into account what may be the performance of the new contracts
commenced following the business reallocation process.
DES-ESS contracts that ceased on 4 March 2013 (Request for Tender)
The 2012 DES-ESS Tender saw over 300 of the generally poorer performing Contracts exit the DES-ESS
Market. These Contracts are now no longer included in the DES-ESS Star Ratings calculations. Removal
of these contracts will result in an increase of the National Average performance across Performance
Measures. Modelling using the December 2012 Star Ratings population and removing the ceased
Contracts from the calculations, on average there was a reduction of 16 Star Percentage points for
continuing Contracts. The modelling found that not every continuing Contract would drop 16 Star
Percentage points, but over 90 per cent of the continuing Contracts change in Star Percentage was
within six Star Percentage points of this average change (i.e. within six points of a 16 point reduction).
DES-DMS contracts that ceased on 3 March 2014 (Business Reallocation)
The 2013 DES-DMS business reallocation saw over 40 of the generally poorer performing contracts exit
the DES-DMS market. These contracts are now no longer included in the DES-DMS Star Ratings
calculations. Removal of these contracts resulted in an increase of the National Average performance
across Performance Measures.
Modelling using the December 2013 Star Ratings population and removing the ceased contracts from
the calculations found that, on average, there was a reduction of eight Star Percentage points for
continuing contracts. The modelling found that not every continuing contract would drop eight Star
Percentage points, but the majority of continuing contracts’ change in Star Percentage was within five
Star Percentage points of this average change.
DES-DMS contracts that ceased on 2 March 2015 (Request for Tender)
The 2014 DES-DMS Tender saw almost 100 CRS Australia contracts exit the market, which had varying
levels of performance. These ceased CRS Australia contracts accounted for around 40 per cent of all
performance data, across all performance measures in DES-DMS. These contracts are now no longer
included in the DES-DMS Star Rating calculations. Removal of these contracts resulted in a moderate
increase of the National Average performance across the Performance Measures.
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 3 of 7
Modelling using the December 2014 Star Ratings population and removing the ceased contracts from
the calculations found that, on average, there was a reduction of three Star Percentage points for
continuing contracts. The modelling found that not every continuing contract would drop three Star
Percentage points, but the majority of continuing contracts’ change in Star Percentage were either up or
down by six Star Percentage points or less.
Question
How will the past performance of Remote JSA and DES Contracts be treated in the Star Ratings
calculations?
Answer
The Remote Jobs and Communities Program (RJCP) commenced on 1 July 2013. JSA and DES sites in
designated remote locations will cease operating and RJCP sites will commence operating in those
locations.
Where contracts prior to the RJCP operated in both remote and excised locations of an ESA and will
continue to operate in the excised locations after the closure of their remote sites, performance data for
the remote sites will continue to be included in the calculations of contract level Star Ratings. The ESAs
impacted are:










Far West NSW
Mt Isa QLD
Western Downs QLD
North Country SA
Port Lincoln/Ceduna SA
Goldfields/Esperance WA
Kimberley WA
Mid-West/Gascoyne WA
Alice Springs NT
Townsville QLD (non-remote ESA with excised remote area of Palm Island)
For JSA and DES Sites and Contracts that were located in Fully Remote ESAs, these Contracts ceased on 1
July 2013 and their past performance is removed from the June 2013 Star Ratings and future
calculations. The Fully Remote ESAs are:






Oxley NSW
Katherine NT
Tennant Creek NT
Top End NT
Far North QLD
Pilbara WA
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 4 of 7
Question
How is disability information sourced for Star Ratings regression calculations for participants who are
ESAt/JCA exempt?
Answer
ESAt/JCA exempt participants do not have a ESAt/JCA report, which contains barriers and medical
conditions information. Providers are required to enter the primary disability information for these
participants into the Employment Services System (ESS) for the DES Star Ratings regression calculations.
The Primary disability information is entered in the following location in ESS (see screenshot below):
ESS > Job Seekers > Registration & Referrals > Registration (Circumstances tab)
Other information like education level, ex-offender status and Indigenous status are sourced from
participants’ JSCI, including eligible school leavers. Providers should ensure that the JSCI is accurate and
current for participants on their caseload.
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 5 of 7
Question
If there are multiple Sites for a given Contract, but only one Site and the Contract receive a Star Rating,
how can the Star Rating be different between the Site and the Contract?
Answer
Contract and Site level Star Ratings are calculated separately. Each Site must have sufficient data to
receive a Site level Star Rating. A provider may have several Sites within an ESA, but only one of these
Sites may have sufficient data to receive a Star Rating. However, Contract level Star Ratings incorporate
the performance of all the Sites for the Contract, regardless of whether each Site had sufficient data to
receive a Star Rating in its own right. Therefore, the Contract level Star Rating will be influenced by the
performance of the Site that received a Star Rating, as well as the performance of all the other Sites that
make up the Contract. This can result in two different Star Ratings between the Site and the Contract.
Trim File Number: D15/371490
DES Star Ratings Supporting Document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 6 of 7
Document Change History
Version
Start Date
Effective
Date
1.1
13 Aug 15
12 Sep 15
1.0
9 Mar 15
9 Mar 15
Trim File Number: D15/371490
DES Star Ratings Supporting Document
End Date
Change & Location
Narrative: Added ‘DES-DMS contracts that
ceased on 2 March 2015 (Request for
Tender)’ section to the answer on page 3.
12 Aug
15
Original version of the document
Effective Date: 12 September 2015
Last Updated: 24 June 2015
Page 7 of 7