Move More, Sit Less
A toolkit for evaluating physical activity
programs in your workplace
Acknowledgements
This toolkit was developed by:
Michelle Daley, Jen-Kui Maxwell: Heart Foundation NSW
Sarah Burks-Young, Josephine Chau, Lina Engelen and Dheepa Jeyapalan,
Prevention Research Collaboration (PRC), The University of Sydney
The Heart Foundation and PRC would like to thank all of the organisations
and people who provided feedback on draft versions of the Move More, Sit
Less: A toolkit for evaluating physical activity programs in your workplace
PRO-179
© 2017 National Heart Foundation of Australia
ABN 98 008 419 761
Suggested citation: Heart Foundation and PRC University of Sydney. Move
More, Sit Less: A toolkit for evaluating physical activity programs in your
workplace 2017.
Terms of use: This material has been developed
for general information and educational purposes
only. It does not constitute medical advice. This
document has been developed by the National
Heart Foundation of Australia and the University
of Sydney and is based on independent research
and the available scientific evidence at the time of
writing. The information is obtained and developed
from a variety of sources including but not limited
to collaborations with third parties and information
provided by third parties under licence. It is not
an endorsement of any organisation, product or
service.
While care has been taken in preparing the content
of this material, National Heart Foundation of
Australia and the University of Sydney, their
employees and related parties cannot accept any
liability, including for any loss or damage, resulting
from the reliance on the content, or for its accuracy,
currency and completeness.
This material may be found in third parties
programs or materials, including but not limited to
show bags or advertising kits. This does not imply
an endorsement or recommendation by National
Heart Foundation of Australia and the University
of Sydney for such third parties organisations,
products or services, including their materials or
information.
The entire contents of this material are subject to
copyright protection. This publication may be used,
reproduced or adapted for individual workplace
purposes, with appropriate acknowledgement. No
part may be reproduced or adapted for commercial
gain in any form or language without prior written
permission fromNational Heart Foundation of
Australia and the University of Sydney.
1. Getting started............................................................................................. 4
Contents
Foreword:............................................................................................... 4
1.1 Why evaluate?................................................................................. 5
1.2 Thinking ahead................................................................................ 6
2. Designing your evaluation.......................................................................... 7
2.1 What evaluation designs are available?............................................ 7
2.2 What type of data should I collect?.................................................. 9
2.3 What worked and why?................................................................. 10
2.4 Which evaluation design should I choose?.................................... 11
2.5 What outcome do you want to measure?....................................... 12
2.6 How do I choose a data collection tool?........................................ 12
3. Conducting the evaluation....................................................................... 16
3.1 How do I plan the evaluation?....................................................... 16
4. Understanding and interpreting your results.......................................... 18
4.1 How do I interpret quantitative results?.......................................... 18
4.2 How do I interpret qualitative results?............................................ 18
5. Sharing your findings................................................................................. 19
5.1 How do I report my findings?........................................................ 19
5.2 How do I present my findings?...................................................... 20
6. Review and improve................................................................................. 21
7. Resources, tools and appendices............................................................. 22
7.1 Links to helpful sites and external tools......................................... 23
8.References.................................................................................................. 24
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
3
1.0
Getting started
Foreword
This toolkit builds upon the existing Healthy Workplace Guide: Ten steps to implement
a workplace health program, Second edition.
It is specifically designed to assist your workplace in evaluating programs that target
moving more and sitting less. It provides step-by step instructions on which methods to
choose, what questions to ask, and how to analyse and interpret your results.
Step
Step10.
10.
Evaluateyour
your
Evaluate
program and make
program
and
make
improvements
improvements
Step 1.
Gain support from
management
Step 2.
Step 9.
Introduce the
concept and
identify needs
Manage your
program
Part 3.
Step 8.
Implementing
your program
Part 1.
Getting Started
Step 3.
Gain support from
employees and
establish program
responsibilities
Promote your
program
Part 2.
Step 7.
Constructing your
program
Step 6.
Select incentives
and rewards
Step 5.
Identify program
activities, develop
an action plan
and budget
Figure 1:Ten Steps to implementing a workplace health program.
4
heartfoundation.org.au
Step 4.
Develop goals
and objectives
Identify additional
support
1.1
Why evaluate?
Program evaluation is an important step in achieving a healthier workplace because
it helps monitor whether your program is reaching its goals. It provides meaningful
feedback on program components, what worked, and why or why not.
Why are you evaluating your program?
There are many reasons as to why your workplace may deliver and
evaluate programs. Some include:
JJ To respond to a directive from management;
JJ To respond to an identified workplace health need;
JJ To understand what’s working;
JJ To justify ongoing funding for the program/s;
JJ Personal interests;
To make adjustments to a program that may be required over time;
JJ
To ensure your program is delivering optimal benefit and making as
much of an impact and difference as possible.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
5
1.2
Thinking
ahead
There are a few things that your workplace should consider before planning a
program and its evaluation. Some things to consider are shown below in Table 1:
Table 1: Considerations to make before planning your program
and its evaluation
Program aim
What health behaviour is your program
targeting? e.g. sitting less, physical activity
Practicalities
Program timeframe
• Where are you up to in your program?
• How long do you have to conduct the
evaluation?
• Are there other time pressures?
Program size
• How many employees is your
program targeting?
• How many program sites are there?
Accessibility
• Are your employees easy to engage with?
E.g. do they have access to emails, regular
working hours, or do they work remotely?
Resources
Human resources
• How much time can you or someone in your
workplace spend on this evaluation?
• Do you have other workplace champions who
can help?
Support
• How supportive is your workplace?
• How engaged are the employees?
• Will recruiting participants
be a barrier?
Budget
• Are there any financial restrictions to
consider?
Other
considerations
Literacy or language issues
• Are there any literacy or language barriers that
should be considered?
Cultural or gender sensitivities
• Are there any cultural or gender issues that
may impact your program?
6
heartfoundation.org.au
2.0
Designing
your evaluation
2.1
What
evaluation
designs are
available?
There are two ways to evaluate your program depending on where you are up to in
your program planning. These designs are shown below in Figures 2 and 3:
A. After program delivery (single measurement point)
JJ Suitable design if you choose to evaluate after your program has started
JJ Provides a snapshot of behaviours after a program has started
JJ ‘After’ refers to approximately four to six weeks after the program has started
(not after the end of the program)
JJ Can indicate an association between the program and the behaviours
measured, but cannot say that one causes the other.
Measurement
(after program
has started)
Move More, Sit Less program delivered
PROGRAM TIME
Program start
Four to six weeks
Figure 2: 'After' program design
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
7
B. Before and after program delivery (two measurement points)
JJ S uitable design if you are planning your evaluation before your program
has started
JJ Recommended design, because it allows you to show changes in behaviour
resulting from your program by collecting information before and after
the program
JJ Requires forward planning
JJ ‘Before’ refers to normal working conditions with no changes
JJ ‘After’ refers to approximately four to six weeks after the start of your program.
JJ The aim is to assess health behaviours after the program has been
implemented. Even if your program was shorter than four weeks, we still
recommend this timeframe; the key is to measure lasting health change
JJ If you have an ongoing program, it is good to continually monitor it from timeto-time.
Measurement one
(before program
has started)
Measurement two
(after program
has started)
Move More, Sit Less program delivered
PROGRAM TIME
Program start
Figure 3: 'Before and After' program design
8
heartfoundation.org.au
Four to six weeks
2.2
What type of
data should I
collect?
There are broadly two types of data: quantitative and qualitative data. Both are
valuable to the evaluation of your program.
Quantatative data numbers and quantities
JJ The change in the number of days of 30 minutes of
physical activity:
e.g. the average number of days of 30 minutes of physical
activity increased from 2 to 3 days.
JJ T
he number of flights of stairs participants climbed:
e.g. 60% of participants reported climbing 4 to 5 flights of
stairs per day before the program.
Qualitative data words and qualities
Quotes from participants about the program:
JJ e .g. "I really liked having the option to stand when it
suits me"
JJ T
hemes from observations:
e.g. participants seem to walk around more after receiving
a reminder email to move more.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
9
2.3
What worked
and why?
This toolkit primarily focuses on measuring the success of your program related
to changes in physical activity and sitting less. In addition to evaluating and
measuring the behaviour changes, you may wish to explore process related
outcomes such as how your program was delivered, what did and didn’t work, and
what areas could be improved.
This will provide insight into:
JJ Whether the program was delivered as intended;
JJ The ease or difficulty of delivery;
JJ Participant support and acceptance;
JJ Participant engagement and retention;
JJ Employees' overall impression;
JJ Suggestions for improvement.
Depending on your workplace, this information can be obtained by a survey, in
a focus group, or in a more informal manner, such as simply asking employees
informally or at a team meeting.
Refer to Appendix A 1 - Process Evaluation Guide for a question guide template.
Why evaluate the program delivery?
Regardless of how effective your program is in achieving behaviour
change, you should always evaluate how your program was
delivered. It will provide you with useful feedback into what worked,
why or why not and how the program can be improved.
10
heartfoundation.org.au
Now that you have reviewed the different types of evaluation design and types of
information that you can collect, you can start to plan your evaluation. The flow
chart shown below (Figure 4) is designed to help you choose the best evaluation
plan and recommend what type of information to collect.
2.4
Which
evaluation
design should
you choose?
Start Here
How many
participants are in
your program?
Less than 10
After program
•
Qualitative
evaluation
We have
already started
the program
After program design
recommended
•
Qualitative
evaluation
10 or more
Where are you up
to in your program
planning?
Planning the
program
Before and after
design recommended
•
Quantitave and/or
qualitative evaluation
Figure 4: How to plan your evaluation
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
11
2.5
What outcome
do you want to
measure?
Before you can choose the best evaluation tool, you need to consider what specific
behaviour changes your program targets and what type of information you are
interested in collecting. This will help develop the foundation of your evaluation.
Ask yourself:
What specific behaviour(s) does your program aim to change?
Examples:
JJ Sitting
JJ Standing
JJ Physical activity
What do you want to know in relation to this behaviour?
Examples of types of information you can collect to measure change:
JJ Number of days physically active
JJ Daily step count
JJ Minutes per workday spent sitting or standing
 Number of breaks from sitting per hour
2.6
How do you
choose a data
collection tool?
Now that you have selected an evaluation design and decided what behavioural
changes you want to measure, you can choose the evaluation measurement
method(s) and tool(s) best suited to your needs. Table 3 outlines key information
specific to each type of behaviour, in particular:
JJ
JJ
JJ
JJ
The measurement method;
The amount of resources, expertise and time required;
The type of information the tool provides and;
Where to find the instructions to use the tool.
A combination of measurement methods and tools is often the most
informative. Choosing measurement methods that allow you to
collect both quantitative and qualitative information will provide you
with additional insights and background information to support your
findings.
Refer to Section 2.2 for more information about quantitative and
qualitative data
12
heartfoundation.org.au
A range of measurement methods are described below in Table 2.
Table 2: Examples of measurement methods for your Move More, Sit Less
programs
Measurement method
Description
Pedometer
This is an inexpensive, commercially available device
that measures the number of steps a person takes when
walking, running or performing other daily movement.
Monitoring employees’ daily step-count before and after
a program can show meaningful changes in physical
activity levels.
Activity trackers
This is a commercially available device measures the
number of steps a person takes, their activity levels
and sometimes flights of stairs climbed. Monitoring
employees’ steps or activity levels before and after a
program can show meaningful changes.
Focus group
This is a facilitated group discussion that allows you to
gather qualitative information about your program. This
delivery mode can shed light on how employees’ activity
patterns have changed and their thoughts and opinions of
the program.
Observations
This involves visual counting and will require one or
more people to conduct the observations. It can provide
insights into workers behaviours and how spaces
are being used, such as staircases, without requiring
expensive or complex technologies.
Survey
This involves constructing questionnaire (either paperbased or online) that uses a number of questions to
measure physical activity and/or sitting, as well as open
ended questions to gather feedback on the program.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
13
Table 3: Choosing your data collection tool
What
do you
want to
measure?
What can
you use?
What resources do you need?
Time
Sitting
Physical
Activity
Stair
Use
14
Cost
What type of
information
is collected?
What tools
are available?
Where can
you find
instructions?
(Associated
appendices)
Knowledge
Focus
Group



Focus Group
Guide
FG 1, FG 1a,
FG 2
Observation



Observation
Guide
Sit 1
Survey



Occupational
Sitting,
Physical
Activity
Questionnaire
(OSPAQ)
Sit 2
Focus
Group



Focus Group
Guide
FG 1, FG 1a,
FG 2
Survey



Occupational
Sitting,
Physical
Activity
Questionnaire
(OSPAQ)
PA 1
Survey



Single Item
Question
PA 2
Activity
Tracker



Step Tracker or
Activity
Levels
tracker
PA 3
Pedometer



Step Tracker
PA 4
Focus
Group



Question
guide
FG 1, FG 1a,
FG 2
Observation



Visual Count
SU 1
Survey



Stairs Climbed
Question
SU 2
Activity
Trackers



Stairs
Tracker
SU 3
Self-Report



Self-Report
SU 4
Key
Qualitative data
heartfoundation.org.au
Quantitative data
Do you already collect other useful information?
Your workplace may collect information such as billable hours or
sick days. While this toolkit will not cover using this information,
you should consider including any available measures that may be
useful.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
15
3.0
Conducting
the evaluation
3.1
How do I plan
the evaluation?
Now that you have chosen your evaluation tool(s), you need to consider the
following things before you can get started:
JJ Who will collect the data?
JJ Where and when will it be collected?
JJ How will you engage participants in the evaluation
and raise awareness?
JJ Have you considered employee privacy
and confidentialty?
With these considerations in mind, refer to instructions for the specific tool(s)
you’ve chosen from Table 3 and start evaluating your program.
What if you want to measure more than one thing?
 Depending on your program, you may be using multiple tools
from Table 3 in the same measurement method e.g. You may use
the sitting (OSPAQ) and stairs (EPIC) questionnaire to measure
sitting and stairs use in the same survey.
 You can also combine multiple tools such as a focus group
and a one-week pedometer measurement. Try to conduct these
measures at the same time to make it as simple as possible for
yourself and the participants.
16
heartfoundation.org.au
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
17
4.0
Understanding and
interpreting your results
Now that you have conducted the evaluation, it’s time to interpret and understand
the information so that it can be reported and presented in a meaningful way.
All of the tools located in the appendices have specific instructions to help you
understand what your results mean.
4.1
How do you
interpret
quantitative
results?
4.2
How do you
interpret
qualitative
results?
The data you have collected before and after your program will allow you to
measure changes in sitting and physical activity behaviour(s). How you interpret
the information you collect will depend on which delivery method(s) and tool(s)
you have chosen to measure your program’s progress toward achieving its goal(s).
Refer to the appendix relating to the specific measurement tool you have chosen
for detailed instructions on how to interpret and understand your results.
Understanding and interpreting qualitative information is about summarising the
central themes and ideas explored within focus groups and open-ended survey
questions. This will give you a broader understanding of the ideas and themes
raised by participants in relation to health behaviours and your program.
A few key questions to explore are:
JJ What themes emerged from your focus group?
JJ How often did the themes emerge?
JJ Were these themes expected or is this new information?
JJ What are the broader implications of these themes and why are
they important?
You can highlight quotes that you feel support your interpretation. Make sure that
the quotes are de-identified to ensure confidentiality and that no information could
be used to identify the speaker.
For more information on reporting qualitative findings see Appendix FG 2.
18
heartfoundation.org.au
5.0
Sharing
your findings
5.1
How do you
report your
findings?
Now that you have collected and interpreted information for your evaluation, it
can be used to create a report or presentation. If a more formal report is required,
results can be presented in a narrative format that includes six key elements:
1. Executive summary: provides an overview of the program goal(s), evaluation
and key findings
2.
Background: presents background information and provide a purpose for the
program and evaluation.
3. Program description: describes the program, its components and the delivery.
4. Methods: answers the ‘who, how, where and when’ questions.
5. Findings: reports on your results.
6.
Conclusions and future recommendations: reflects on the results and what
they mean.
Please refer to the sample report (Appendix A 2) for more detailed information on
a Move More, Sit Less program.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
19
5.2
How do you
present your
findings?
It is important to share your findings and experiences with the workplace and
other relevant contacts, both to raise awareness of the program but also to share
knowledge of successes and failures in the program. Here are a few things to
consider before presenting your findings:
now your audience: Be conscious of the amount of
K
time available and knowledge base of your audience.
se visuals: Images and charts are a great way to
U
explain your program findings. A brief PowerPoint
presentation is often an easy way to keep your audience
engaged while communicating a clear message
resent the facts and keep it short: Keep your
P
presentation brief and present the results of your
program clearly, in an unbiased manner. A short, sharp
presentation will have a greater impact, especially
if your presentation has been incorporated into an
existing meeting.
hank you: Don’t forget to give acknowledgement to
T
those involved, where appropriate
20
heartfoundation.org.au
6.0
Review
and improve
The final step of evaluating your program(s) is reviewing the findings and reflecting
on how current and future programs can be improved. Remember that program
management and evaluation is an evolving process, and requires continual review
and improvement.
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
21
7.0
Resources, tools
and appendices
Appendix
Associated Appendices
Excel spreadsheets
associated with
appendix
FG 1
Focus Group 1 - Preparing and running a focus group
-
FG 1a
Focus Group 1a - Question guide
-
FG 2
Focus Group 2 – Preparing, analysing and reporting
qualitative data
-
Sit 1
Sitting 1 – Observational tool
-
Sit 2
Sitting 2- Occupational sitting and physical activity questionnaire
- OSPAQ
-
FG 1
Focus Group 1 - Preparing and running a focus group
-
FG 1a
Focus Group 1a - Question guide
-
FG 2
Focus Group 2 – Preparing, analysing and reporting
qualitative data
-
PA 1
Physical Activity 1- Occupational sitting and physical activity
questionnaire - OSPAQ
-
PA 2
Physical Activity 2 - Single item leisure-time physical
activity questionnaire
-
PA 3
Physical Activity 3 - Activity trackers
Daily step tracker
Step tracker group
PA 4
Physical Activity 4 - Pedometers
Daily step tracker
Step tracker group
FG 1
Focus Group 1 - Preparing and running a focus group
FG 1a
Focus Group 1a - Question guide
FG 2
Focus Group 2 – Preparing, analysing and reporting
qualitative data
SU 1
Stairs 1- Observational tool
SU 2
Stairs 2- EPIC Physical activity questionnaire
(stair climbing question)
22
heartfoundation.org.au
Appendix cont.
Associated Appendices cont.
Excel spreadsheets
associated with
appendix cont.
SU 3
Stairs 3 - Activity trackers
Stair tracker group
SU 4
Stairs 4 - Self teport tracking
Stair tracker group
A1
Process evaluation question guide
-
A2
Sample report
-
7.1
Links to
helpful sites
and external
tools
Free (limited) Survey Building Site - surveymonkey.com
Free online step challenge tracker/program – 10000steps.org.au
Downloadable toolkit containing information, tools and resources workplaces
need to create an effective health and wellbeing program www.worksafe.vic.gov.au/wps/wcm/connect/
f7093280439cdf6db37eb3145ee8dc5e/?a=17203
Step-by-step information about how to develop an evaluation plan:
http://ctb.ku.edu/en/table-of-contents/evaluate/evaluation/evaluation-plan/main
Information and resources of how workplaces can evaluate their workplace
wellness program:
http://workplaces.healthier.qld.gov.au/getting-started/step-five-evaluation/
Move More, Sit Less A toolkit for evaluating physical activity programs in your workplace
23
8.0
References
1.Heart Foundation, Cancer Council NSW and PANORG University of Sydney.
Healthy Workplace Guide: 10 steps to implementing a workplace health
program. Second edition. 2016.
2.Berkowitz, S. Using Qualitative and Mixed Method Approaches. Chapter 4
in Needs Assessment: A Creative and Practical Guide for Social Scientists, R.
Reviere, S. Berkowitz, C.C. Carter, and C. Graves-Ferguson, Eds. Washington,
DC: Taylor & Francis. 1996.
3.Bogdan R. B. & Biklin, S. K. (1998). Qualitative Research for Education: An
Introduction to Theory and Methods, Third Edition. Needham Heights, MA:
Allyn and Bacon.
4.Center for Disease Control and Prevention. Introduction to Program Evaluation
for Public Health Programs : A Self-Study Guide. 2011; (October):1–100.
5.Chau, J. Y., Van der Ploeg, H. P., Dunn, S., Kurko, J., & Bauman, A. E. (2012).
Validity of the occupational sitting and physical activity questionnaire.
Medicine and science in sports and exercise, 44(1), 118-125.
6.Department of Health V. Community Assessment Tool Kit: Nutrition and
Physical Activity Fit and Healthy Vermonters Community Assessment Tool Kit:
Nutrition and Physical Activity.
7.Eliot & Associates. All rights reserved. Guidelines for Conducting a Focus
Group. Duke Univ Website [Internet]. 2005; 1–13.
8.Milton, K., Bull, F. C., & Bauman, A. (2010). Reliability and validity testing
of a single-item physical activity measure. British Journal of Sports Medicine,
bjsports68395.
9.Pols, M. A., Peeters, P. H., Ocke, M. C., Slimani, N., Bueno-de-Mesquita,
H. B., & Collette, H. J. (1997). Estimation of reproducibility and relative
validity of the questions included in the EPIC Physical Activity Questionnaire.
International Journal of Epidemiology, 26(suppl 1), S181
10.Program Development and Evaluation, Basics of Good Evaluation Reporting,
Quick Tips #14, University of Wisconsin-Extension, Madison, WI. 2002.
http://www.uwex.edu/ces/pdande/resources/pdf/Tipsheet14.pdf
11.Seidel J V. Qualitative Data Analysis. Ethnogr [Internet]. 1998;4(c):1–15.
12.Stewart D, Shamdasani PN, Rook DW. Analyzing Focus Group Data. Focus
Groups Theory Pract. 2006;109–34.
13.Wareham, Nicholas J., et al. "Validity and repeatability of a simple index
derived from the short physical activity questionnaire used in the European
Prospective Investigation into Cancer and Nutrition (EPIC) study." Public health
nutrition 6.04 (2003): 407-413.
14.Worksafe Queensland. Workplaces for Wellness: Evaluate your workplace
wellness program.
https://workplaces.healthier.qld.gov.au/member/load/media/?f=evaluate-yourprogram.pdf
24
heartfoundation.org.au