Case Study: New Beginnings-Evaluative questions, indicators, targets and measures
*Note indicators, targets and measures were generated by DGMT for the sake of illustrating the concepts being communicated through the website and does not necessarily
represent the actual indicators, targets and measures used by New Beginnings.
Programme elements
Inputs
•
−
−
−
−
−
−
•
•
•
•
‘Stipended’ staff skilled in terms of
the following:
Training administration
Training facilitation
Subject experts
Counselling
Networking and relationship
management
Management
Access to training curriculum via
membership to Bergzicht Training
Franchise
Training venue for various courses
Training equipment and resources
for various courses
Overheads and transport
Evaluative Questions
Where the inputs available?
Where the inputs available when they
were needed?
Where the inputs sufficient?
Indicators
Target
Measures
Not applicable
Activities
Provide counseling and other
interpersonal services to create an
enabling environment assisting students
to be motivated and emotionally able to
participate in training courses
−
−
Were the students motivated and
emotionally able to participate in
courses?
Was there an enabling environment
for participation?
Offer and administer life skills course
Offer and administer three job skills
courses in:
− Office Management
− Frail care
− Hairdressing
Seek out and manage relationships with
partner organisations that can place New
Beginnings graduates in employment
Were there students that participated in
the life and job skills programmes?
Did the organisation seek out and manage
relationships with other organisations
that can place graduates?
−
− Student retention rate
− Motivation self-rating (by students)
− Course rating (by students)
Note: This aspect can be studied in more depth
with qualitative methods
−
Number of registered students for the life skills
programme
−
100%
retention rate
−
Course
administration
statistics
Pre-and post
course student
questionnaire
16 per course
cycle
Course administration
statistics
Number of registered students for the job skills
programme
See outputs
Page 1
−
16 per course
cycle
Programme elements
Outputs
Graduates with life skills
Graduates with job skills
Successful relationships with
organisations that can either do
placements or are willing to place
graduate students
Evaluative Questions
Were there graduates from the life and
job skills training programmes?
Does the programme have successful
relationships with organisations that are
willing to place graduates?
Indicators
Number of graduates from life skills programme
Number of graduates of job skills programme
Number of organisations that New Beginnings is
collaborating with to place graduates.
Target
100% of 16 per
course cycle
100% of 16 per
course cycle
Per year aim to
have 5 that will
place students and
10 that will
provide job
mediation services
Measures
Course administration
statistics
Management reports
Immediate Outcomes (1-3 years)
New Beginnings graduates apply the life
skills they have attained to improve their
lives
Life skills are passed onto children,
spouses and other connections
New Beginnings graduates create their
own businesses
New Beginnings graduates are able to
look for and successfully find employment
New Beginnings students are able to
(a) Has there been an increase in the life
skills of students as a result of their
participation in the programme?
(b) Are they applying the life skills they
have acquired to improve their lives?
(c) Are graduates passing these life skills
on to significant others and
connections?
Are the graduates able to successfully
look for and find employment or to start
their own businesses?
If they found/were placed in employment
(a) Skill and knowledge tests results for:
o Handling personal finances
o Job seeking & work ethics
o First aid
o Conservation
o HIV & STI & personal health
o Laundry & cleaning
o Kitchen & cooking
(b) & (c) Self (the student/graduate) and family
rating (of themselves and of the graduate)
on application of life skills after the
student’s participation in New Beginnings
skills courses
Note 1: B & C can be studied in more depth with
qualitative methods.
Note 2: B &C can also be tested before and after
the intervention, but a reasonable period of time
would have to take place between the pre &
post-test during which many things can occur
that could influence the finding of the post-test
(i.e. they might acquire skills from other places,
or forget what they have learned).
Number of graduates that looked and found
employment within six months of graduating or
that started their own business within one year
of graduating
Number of graduates that stayed in
Page 2
(a) 90% of
students per
course have an
average of 75%
and higher for the
lifeskills
coursework after
testing
(a) Course test/exam
(b) Student and family
questionnaire
(administrated within 1
month after
graduation)
85% of graduates
Alumni follow-up
system
95% of graduates
Alumni follow-up
Programme elements
remain in employment
Graduates from New Beginnings are
successfully placed in employment
Evaluative Questions
do they manage to retain this
employment?
Have New Beginnings managed to
successfully place some of their graduates
in employment?
Indicators
Target
employment for the full term of the employment
contract or were still in employment 1 year after
a permanent placement
Number of organisations that New Beginnings is
collaborating with to place graduates.
Measures
system
Per year aim to
have 5 that will
place students and
10 that will
provide job
mediation services
Management reports
Long term outcomes (4-6 Years)
(a) Income level of New Beginnings graduates
prior to participation and after participation
Unemployment is reduced leading to
increased income and reduced poverty
Desired Impact (in 7-10 years):
Has the poverty of New Beginnings
graduates been reduced and has this and
the increase in their life skills lead to an
increase in their life quality?
If yes, is this due to the New Beginnings
programme, at least in part?
Improvement in life quality of participants
(b) Self-reported life quality level of New
Beginnings graduates prior to and after
participation on the following (assessed
retrospectively):
o Looking after physical health
o Eating a balanced diet
o Freedom from anxiety
o Self- confidence & esteem
o Life satisfaction
o Quality of place of residence
o Ability to care for significant
others and pets
Note: B can be studied in more depth with
qualitative methods.
If the New Beginnings students:
− Get employed or create their own
employment (especially in the areas that
they received technical training in) after
they graduate - and there is a subsequent
increase in income level;
− And if the majority of them manage to
retain this employment/income for a
reasonable period of time;
this evidence is already sufficient to conclude
that the New Beginnings programme is effective.
Page 3
Income level checked at
6 months intervals as
part of alumni followup system
(a) 100% increase
in income level
(since all students
enter the
programme
without any
source of income)
(b) Minimum of
40-60% increase
life quality selfrating
Retrospective
questionnaire assessing
self-reported life quality
level before and after
participation in the
programme. Based on
existing life quality scale
items (look at
Department of Land
affairs Quality of Life
Survey)
Note: A version of this
questionnaire can also
be administrated upon
registration with the
programme, but
because ratings are
subjective and people’s
standards change over
time, students might
not realise that they are
actually rating
something at the same
point of the scale on the
Programme elements
Evaluative Questions
Indicators
It would probably not be necessary for New
Beginnings to do quasi-experimental
(comparison of groups that have participated
and those that have not) to further illustrate
their effectiveness. But, they could, once they
have a reasonably large sample*, do a
comparison of the ratings of newly registered
students that have not yet participated in the
programme (as a control group) and the
retrospective ratings of programme alumni.
*(read more about sampling in the Evaluation
section of the Growing Confidence website)
Page 4
Target
Measures
pre-test and on the
post-test, not because
there was no
improvement, but
because their standards
have also improved.
It is thus important that
they are also asked to
rate their lives before
and after participation
in the programme
retrospectively to
ensure a fair
comparison.