Designing
Effective Surveys
with Qualtrics
Workbook
Designing Effective Surveys with Qualtrics – Workbook
This workbook accompanies the Designing Effective Surveys with Qualtrics training session.
To get started with Qualtrics, visit cardiffmet.qualtrics.com. If you are on campus, you will
automatically be logged in and presented with the My Projects page.
Qualtrics is feature-rich, but intuitive to use. A quick overview of the Qualtrics environment
can be found on page 3 of this workbook.
The first task in this workbook guides you through the basics of creating and using questions
in Qualtrics. The remaining tasks are less step-by-step in nature, but suggestions for
appropriate help articles and tutorials will be listed below the task. These articles can be
accessed by searching the Qualtrics help pages (also known as Qualtrics University).
Suggested keywords for the search will appear in boxes like this:
Help?



Search Qualtrics for:
About Blocks
Display Logic
Skip Logic
The link to Qualtrics Help & Feedback can be found in the top right of any Qualtrics page:
© 2017 Cardiff Metropolitan University
2
Designing Effective Surveys with Qualtrics – Workbook
Currently selected
project – click to change
Modules – edit, distribute and
view your survey data
Menus and options for
the current module
Remove this question
Move this question
up or down
Add a question below
Right pane – contains options
for the selected question and
tools such as Add Page Break
and Copy Question
Add logic to this
question
© 2017 Cardiff Metropolitan University
3
Task 1 – Creating Surveys and Questions
We will begin by setting up a new survey, then we’ll create some questions and adjust some
question options. The survey theme is “Sustainability”.
Help?



Search Qualtrics for:
Creating a Project
Creating Questions
Formatting Questions
1. From the Projects page, click the Create Project button.
2. Entitle the survey “Sustainability” and place it in a folder called “Training”.
3. Create a new question. The question should read: “Please indicate your level of agreement,
or disagreement, with the following statement: Cardiff Met is committed to sustainable
development.”
4. Format the statement using the Rich Content Editor. (E.g. make the text bold and change
the font colour).
Designing Effective Surveys with Qualtrics – Workbook
5. Use the Formatting (right) pane to add Automatic Choices. Choose the Disagree – Agree
scale.
6. Use the New Question Below (+) button to add another question.
7. Edit the question to read: “How familiar are you with the term "sustainable development"?”
8. Qualtrics should automatically select the “Familiar” scale, if it doesn’t select it from the
Formatting pane. Set the number of Choices to 5.
© 2017 Cardiff Metropolitan University
5
Designing Effective Surveys with Qualtrics – Workbook
9. Change the response options to:
 Extremely familiar
 Moderately familiar
 Somewhat familiar
 Slightly familiar
 Not familiar at all
10. Move the “How familiar question…” above the “Please indicate…” question by using the
move buttons on the left of the screen:
11. Use the Preview Survey button to test your survey.
Extension Idea
12. Add a third question between the two existing questions: “Briefly describe what sustainable
development means to you.”
13. Change the Question Type to Text Entry.
14. Change the Text Type to Multi Line.
15. Preview the survey and test.
© 2017 Cardiff Metropolitan University
6
Designing Effective Surveys with Qualtrics – Workbook
Task 2 – Question Types and Simple Logic
In this task we will use some new question types to create questions relating to “energy usage”. We
will also use some simple survey logic – Carry Forward Choices and Skip Logic, to adjust survey flow
and question content, based on respondent’s previous answers.
Help?





Search Qualtrics for:
Matrix Table/Rank Order/Slider
Carry Forward Choices
Skip Logic
Validation
Add Page Break
Part A
1. Create a new Matrix Table question “How often do you perform the following activities?”
2. Change the Automatic Scale Points to Never – Always.
3. Add some statements relating to using energy in the home to the matrix, including: “Use
central heating to heat your house when no-one is home”
4. Create a new Rank Order question: “Rank these activities by how costly you think they are in
terms of energy use”.
5. Carry forward all statements from the “How often do you perform…” question.
6. Preview the survey and test.
Part B
1. Add a new Multiple Choice question “In what circumstances do you use central heating to
heat your house when no-one is home?”
2. Add some options, e.g. “Heating left on by mistake”, “To prevent pipes from freezing”.
Include a “None of the above” option.
3. Change the Answers option for this question to “Multiple Answer”.
4. Make the “None of the above” answer option exclusive.
5. Move the new question above the rank order question.
6. Add Skip Logic to skip the “In what circumstances…” question if the answer to the “Use
central heating…” statement from the matrix question is “Never”.
7. Apply Force Response validation to the “How often do you perform…” question, to ensure
the respondent completes this question.
8. Add appropriate Page Breaks to the survey then preview the survey and test.
Extension Idea
Write a new Slider question: “Use the sliders to indicate how effective you think the following home
improvements are with regards to saving energy”.



Add some response options, e.g. “Draught excluders”, “Cavity wall insulation”.
Enable the Snap to Grid option for the question.
Add appropriate Labels to the slider.
© 2017 Cardiff Metropolitan University
7
Designing Effective Surveys with Qualtrics – Workbook
Task 3 – Display Logic & Piped Text
For the remaining tasks we will create an evaluation survey for Cardiff Met Open Days. Part of the
survey has already been created; it is available in the Qualtrics Training survey library.
Some of the survey has been created, but we need to add appropriate Display Logic to ensure the
respondents only get asked questions that are relevant to them. We will also use Piped Text to
insert situation-dependent text to make the survey questions more readable.
Help?



Search Qualtrics for:
Survey Library
Display Logic
Piped Text
1. If you have not already done so, create a new project based on the Open Day Evaluation –
Library Copy survey in the Qualtrics Training library.
2. Add appropriate logic to Q4 (“The Open Day began with a presentation about the School of
Sport…”). This question should only be asked if they attended a School of Sport Open Day.
3. Add appropriate logic to Q5 (“The Open Day began with a presentation about life at Cardiff
Met…”). This question should only be asked if they did not attend a School of Sport Open
Day.
4. Use Piped Text to pipe the School name from Q2 (“Which Cardiff Met Open Day did you
attend?”) into:
a. The question text for Q5 (“degree programmes offered by the School”).
b. The first response option of Q6 (“Tour of School facilities”)
5. Preview the survey to test the logic and piped text.
Extension Idea
Add a follow-up question to Q7 (“How useful were these experiences in deciding whether to study at
Cardiff Met”). The question should determine why the respondent found an activity (e.g. Visit to
Cardiff city centre) useless; it should only be asked about experiences that were marked as useless.
TIP: A Text Entry question type with Text Type Form will be useful here, as will Carry Forward.
Task 4 – Look and Feel
We will now customise the aesthetics of the survey and ensure survey options are set appropriately.
Help?



Search Qualtrics for:
Look and Feel
Survey Options
Email Triggers
© 2017 Cardiff Metropolitan University
8
Designing Effective Surveys with Qualtrics – Workbook
1. Experiment with the Look & Feel options. You might want to:
a. Try a different design template (though the Cardiff Met template is recommended).
b. Change the Next and Previous button text.
c. Add a progress bar.
d. Add the survey title to the header of the survey.
2. Use Survey Options to:
a. Change the Survey Title that appears in the browser or window title.
b. Set a custom end of survey message.
c. Send a thank you email to the respondent when they have completed the survey.
3. Polish up the survey – add Page Breaks where appropriate, check spelling, punctuation,
grammar etc.
4. Preview and test the survey; ensure it looks good and test whether the survey logic is
performing as expected.
Extension Idea
Setup an Email Trigger to alert yourself whenever particularly negative feedback is received.
Task 5 – Sharing Projects
We will now take a look at how to share and collaborate on surveys with Qualtrics.
Help? Search Qualtrics for:
 Sharing a Project
1. Pair up with the person sat next to you.
2. On the My Projects page: use the Share Project button on the drop-down menu to share
your Open Day project with your partner. Give them the following permissions:
a. Edit: Edit Surveys and Set Survey Options
b. View Results: Download Survey Results and View Survey Results
c. Copy
After you have shared the project with your partner, they should receive a confirmation
email.
3. Your partner should now have shared their project with you.
4. Go to your partner’s project, it is available under the Shared with Me section of the My
Projects page.
5. Edit the Survey Options of their survey: enable the Back button. Note, you should not be
able to edit any other aspect of the survey.
6. Check your Open Day survey; your partner should have enabled the Back button for you.
© 2017 Cardiff Metropolitan University
9
Designing Effective Surveys with Qualtrics – Workbook
Task 6 – Revision
We will now practice what we have learnt so far by adding a couple more questions to the Open Day
survey and applying appropriate logic.
1. Create a new Matrix Table question – “How satisfied were you with the various aspects of
the tour of the campus facilities?”
2. Choose the Dissatisfied – Satisfied scale. Reduce the number of Scale Points to 5.
3. Add some statements to the matrix. E.g. “Length of tour”, “Variety of locations visited”.
4. Move the question between Q9 (“Please rank the usefulness…”) and Q10 (“The next couple
of questions…”).
5. Write a new follow-up question to Q8 (“Aside from the Open Day, which resources…”); use it
to determine which university comparison sites they used. Include a “None of the above”
option, which should be exclusive.
6. Use Skip Logic to only ask this question if the respondent selected “University comparison
website” in Q8.
7. Complete Q9 (“Please rank the usefulness of resources…”) by:
a. Using the Rich Content Editor to place the sub-instructions (“Most useful at the
top”) on a new line and in italics.
b. Changing the question type to the most appropriate type
c. Carrying forward choices from Q8 (“Aside from the open day, which resources…”)
d. Adding appropriate logic; it should only be asked if the respondent has used two or
more resources.
Extension Idea


Add validation to survey questions, where appropriate.
Use Display Logic to display the campus facilities question (Q14) and the accommodation
tour questions (Q10, Q11 and Q12), only if they participated in that activity.
© 2017 Cardiff Metropolitan University
10
Designing Effective Surveys with Qualtrics – Workbook
Task 7 – Blocks and Survey Flow
In this section we will use blocks to organise the questions in our questionnaire. We will also use a
new logic technique, survey flow, to direct the survey.
Help? Search Qualtrics for:
 Survey Flow Overview
 Branch Logic
If you completed the Extension Idea in the previous section, remove the Display Logic before
completing this task.
1.
2.
3.
4.
Create a new block, “Accommodation”.
Move the questions relating to accommodation into the new block.
Repeat steps 1 and 2 for the campus facilities question.
Note that you cannot move questions with Skip Logic into a new block, therefore remove
the Skip Logic from Q8 (“Aside from the Open Day, which resources…”) then move it and the
other “resources” questions into a new block.
5. Add appropriate Display Logic to replace the Skip Logic removed from Q8.
6. Rename the initial block to “Introduction”.
7. Edit the Survey Flow so that the:
a. “Accommodation” and “Campus Facilities” blocks are only asked if the respondent
selects that activity in Q6 (“Which activities did you participate…”).
b. The “Resources” block is asked after the other blocks.
Extension Idea
Add another block of questions to conclude the survey. Try to establish an overall impression of their
Open Day experience. Include questions that:



Evaluate whether the Open Day was useful in deciding whether to choose Cardiff Met.
Determine whether the Open Day encouraged the respondent to choose Cardiff Met.
A Text Entry question to allow the respondent to leave comments about the Open Day.
Adjust the survey flow to ensure this block of questions is always asked last.
© 2017 Cardiff Metropolitan University
11
Designing Effective Surveys with Qualtrics – Workbook
Task 8 – Distribution & Embedded Data
Once a survey has been created it needs to be distributed. Primarily this is done through an
Anonymous Link, or by sending it to a Contact List. Embedded Data can be added to survey
responses to provide more information about the respondent, without asking unnecessary
questions. Embedded data can also be used to generate samples and with survey logic.
Help?




Search Qualtrics for:
Distributions Overview
Creating a Contact List
Embedded Data
View Samples
Part A – Distribution
Anonymous Distribution
From the Distributions module choose Web as the distribution channel, then click Use Anonymous
Link. This activates the survey and generates an anonymous survey link that can be posted to
websites or distributed via posters, flyers etc.
The survey can also be distributed to social media (Twitter, Facebook, LinkedIn etc.) through the
Social Media option in the Distributions module. You are not restricted to one distribution method,
use the left panel on the Distributions module to choose other options, such as QR code.
The Distributions module displaying the QR Code distribution method
© 2017 Cardiff Metropolitan University
12
Designing Effective Surveys with Qualtrics – Workbook
Identifiable Responses – Contact Lists
In order to identify your respondents, you either need to ask for their details during the course of
the survey (i.e. include name, email address questions), or you need to create a Contact List:
1. Go to the Contacts screen then click the + Create Contact List button.
2. Type in a name for your Contact List and choose a folder (optional). Click Next.
3. Use the Import From a File option to import contacts from a Comma Separated Value (CSV)
file. Download and open the Example Document, this is a template that you will use as a
starting point for your Contact List.
4. Add some names and email addresses (including yourself) to the CSV file. You can use these
as examples, if you like:
Joe
Gareth
Tom
Train
Johns
Jones
© 2017 Cardiff Metropolitan University
[email protected]
[email protected]
[email protected]
13
Designing Effective Surveys with Qualtrics – Workbook
5. Save the CSV file to your computer. Ensure the Save as type option is set to CSV (Comma
delimited):
6. Return to Qualtrics, use the Browse button to select the file you have just saved, then click
Add Contacts to update your contact list with the new contacts.
Distributing to a Contact List
Return to the Projects screen then choose the Distributions module and select Emails.
1. Click the + Compose Email button, then complete the Compose Email dialogue as follows:
a. Select your Contact List from To menu.
b. Check the From details are correct.
c. Select when to send the email from the When menu (choose the Send Now option)
d. Add a Subject to the message.
e. Complete the Message section with an introduction to your survey.
2. To save the message to your library, click the Save As link and give the message a name.
© 2017 Cardiff Metropolitan University
14
Designing Effective Surveys with Qualtrics – Workbook
3. Click Send Now to distribute the survey to your Contact List.
Part B – Embedded Data & Samples
1. Add two Embedded Data fields to your Contact List:
a. From the Contacts module, use the Export List action to export your list to a CSV file.
b. Open the CSV file in Excel and add three fields (columns) to the file: “Date of Open Day”,
“Country” and “Age”. These fields will hold embedded data – the date of the Open Day
attended by the respondent, the country they live in and their age.
c. Add some appropriate dummy data to the two new columns – include some
respondents from the UK that are aged 17-21 and some respondents from outside the
UK over the age of 21.
d. Save the file to your hard drive, remember to set the Save as type to CSV.
e. Return to Qualtrics then use the Import List action to update your Contact List.
2. Create a sample from your list – “Overseas Mature Students” (open the contact list then List
Options  Generate Sample). Use the embedded data to set conditions for your sample; for
the purpose of this survey we will define mature students as those over the age of 21.
3. Distribute your survey to your new sample.
Extension Idea
Embedded Data can be piped into survey questions, or used with survey logic to adjust survey flow,
based on the values of the embedded data.


Create a new question – “Did you attend the Student Finance Talk on DATE”. Use Piped Text
to pipe in the date of the Open Day from the Embedded Data.
Use Display Logic or Survey Flow, and Embedded Data to only display this question to UK
students who are aged 21 or younger.
Task 9 – Response Data
Responses can be viewed and analysed via the Data & Analysis module. There are two main ways of
looking at responses:


A tabulated view of the responses is available on the Data tab.
Responses can be exported as a CSV file (for use with Excel), SPSS or XML formats.
Help? Search Qualtrics for:
 Data & Analysis Overview
 Generating Test Responses
1. First, generate some test data for your survey: go to the Survey module  Tools  Test
Survey… (generate about 50 responses).
© 2017 Cardiff Metropolitan University
15
Designing Effective Surveys with Qualtrics – Workbook
2. Go to the Data & Analysis module to view the responses in a table. From this table you can:
a. Filter the responses using the Add Filter menu
b. Delete responses by selecting them and then using the With Selected menu.
c. View an individual response using the Actions menu to the right of the table.
3. Next, try downloading the data via Export Data… option on the Export/Import menu.
TIP: The Legacy Export Format contains all the data for your survey. The Download Data
Table option downloads just the data that you have filtered.
4. Open the data in Microsoft Excel to get a feel for the format of the response data.
Extension Idea
Export the data again (using legacy format), but this time use More Options and choose Use choice
text instead of Use numeric values. Open the data in Excel and compare the differences between
the two files.
Task 10 – Reporting
The Reports module allows you to view pre-defined reports, modify them to suit your needs or
create your own from scratch. There are two types of reports: Results and Printed Reports. Printed
Reports is a legacy format; much of the functionality is also available in a Results report. We will
therefore focus on the Results report as the main way of presenting and analysing aggregated data.
Help? Search Qualtrics for:
 Reports Overview
 Report Pages
1. Select the Reports module then choose the Results tab. You will then be presented with the
default report for your survey. This report contains tables and visualisations for each
question in your survey. We will now look at how you can customise this report.
2. Click on the data table for the first question (“Which Cardiff Met Open Day…?”). Use the
right-pane to change options for this table:
a. Change the metric to Percentage
b. Adjust the number of decimal places to 0.
c. Add a title “School”
3. Select the bar chart visualisation for the first question then make the following changes.
a. Change it to a Pie Chart.
b. Set the Inner Radius of the pie chart to zero.
4. Navigate to Q6 (“Which activities did you…?”). Add a bar chart visualisation with the
following options:
a. Breakout By: Q2 (“Which Cardiff Met Open Day…?”)
b. Title: “Activities by Open Day”
© 2017 Cardiff Metropolitan University
16
Designing Effective Surveys with Qualtrics – Workbook
c. Show the Legend.
5. Navigate to Q7 (“How useful were these experiences…?”) and make the following changes:
a. Change the data table to a Statistics Table
b. Add a Gauge Chart visualisation below the data table visualisation.
HINT: This shows the average rating for each experience on its own gauge. The fuller the
gauge, the better the experience.
6. Add a filter to the report; filter for undergraduate open days (Q3).
7. Use the Report Options menu to finish off the report:
a. Rename it to “Open Day Evaluation – Undergraduate”.
b. Export the report as a PDF.
Extension Idea
Experiment with the Printed Reports section of the Reports module. Printed Reports have more
customisation options, but aren’t quite as easy to use as the Results reports. You might want to try:





Creating a new report
Moving and resizing charts or tables.
Changing report, page, table or chart options.
Adding pages, tables, graphs or text.
Exporting the report.
© 2017 Cardiff Metropolitan University
17