HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
Cover Page
Assessment Name
User Testing + Result
Team members
Surendra Panday, Mathew Brian
Flay, Melaine Jayne Goetze
Project Manager of the week
Surendra Panday
Project Supervisor/ Lecturer
Barbara White
Unit name
HIT381
Due date
Week 10, Thursday
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 1 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
CONTENTS
1.
Introduction ........................................................................................................................ 4
TABLE 1 - Participants ......................................................................................................................... 4
2.
Testing overview ................................................................................................................. 4
TABLE 2 - Analysis of CHILDREN tested ................................................................................................ 4
TABLE 3 - Analysis of ADULT tested ..................................................................................................... 5
3.
Discussion ........................................................................................................................... 6
TABLE 4: Problems, issues, solutions ................................................................................................... 6
4.
Interface strengths and weaknesses .................................................................................... 7
Information obtained with conclusion questions: ..................................................................................... 7
Information obtained with comments from the user: ............................................................................... 7
5.
Conclusion........................................................................................................................... 7
6.
Appendix............................................................................................................................. 8
6.1 Consent forms Standard ................................................................................................................ 8
6.2 Script ............................................................................................................................................ 8
6.3 Set of questions used for kids ........................................................................................................ 9
6.4 set of questions used for adult user ............................................................................................... 9
6.5 Scanned Copies of Consent Forms for all users ............................................................................. 10
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 2 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
USER TEST RESULTS PART B
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 3 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
1. INTRODUCTION
Include:
Questions:
The goal of the usability test was to find out how intuitive was to use web application "Magic Rainbow Words?"
Measures:
Gold Team conducted Usability Test and Pilot Test to examine issues related to website with 5 users which included 4 users for
usability test, and 1 user for pilot usability test.
TABLE 1 - PARTICIPANTS
DETAILS
CHILDREN
ADULTS
Participants Names
Isaac Pandey (5 yr)
Archer Skinner (7 yrs)
Charlotte Braunack (8 yrs) Kate Worden
Gender
male
male
female
female
Occupation
student
student
student
Director w orking in
Government
School
Wulagi Primary School
Girraw een Primary
Girraw een Primary
N/A
Year at School
Play School
year 1
year 3
N/A
Golden words / level
gold level
red level
purple level
N/A
Screen Time
5 seconds
3-4 hours
1 hour
12 w ords / gold level
M obile device use
iPhone, samsung tab,
Android phone - samsung
note 4
iPad
Ipad
phone, ipad, laptop
2. TESTING OVERVIEW
Briefly outline what you did and use tables to summarise all findings.
Eg 4 live users were asked to perform 7 major tasks related to the goals of the user study. The success rate and time
to complete each task is summarised below (Table 1)
Table1: An overview of task success.
TABLE 2 - ANALYSIS OF CHILDREN TESTED
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 4 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
Question
did not m anage to
com plete task
com pleted task w ith
difficulty
com pleted task
successfully
Looking at this home screen do you know where to
start?
100%
Can you go ahead and click on where you think you
might go to next?
100%
Have you seen these words before?
33%
67%
100%
What do you think you will do on this level
Can you tell me what you think the buttons are for?
33%
Can you see what button you might press to get to
the home screen?
33%
67%
33%
33%
Do you know how to listen to the word?
100%
Do you see how you can get to see the next word?
100%
Going through the words can you understand what
happens at the end of the level?
33%
67%
Are you able to understand what to do on the quiz
page?
33%
67%
Do you know what the colours are for?
33%
67%
TABLE 3 - ANALYSIS OF ADULT TESTED
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 5 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
did not m anage to
com plete task
Question
com pleted task w ith
difficulty
com pleted task
successfully
Find start screen
100%
Find next screen, and click
100%
100%
Find out what each button on given screen do
Find out how to listen to the word
100%
Find out what happens at end level
100%
Attempt quiz
100%
Come back to home screen
100%
3. DISCUSSION
From the quantitative data listed above by testing 4 live users it has been found that:




Most of the users had problems coming back to home screen from 2 users test, and other users were fine
with that.
Most of users seem to be fine with going to next level, starting game, listening to word, identify levels, and
colors etc.
Issues were also number of texts used in buttons were not understood by young child user.
2 users had problem finding what each button did on pages
TABLE 4: PROBLEMS, ISSUES, SOLUTIONS
Problem #
Issues
Possible Solutions
Problem 1:

home button is on top, kids
would not understand what
home button means to them...

Users said sound coming from
button was too quick and hard
to grasp what was said.

Problem finding home button from
other pages
Problem 2:
Most of junior kids had problem
attempting quiz page

CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1

Maybe use text as well with
icon
Also make home button more
visible and big.
Maybe use different bootstrap
or their sound setting may have
been set to low volume
Page 6 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
4. INTERFACE STRENGTHS AND WEAKNESSES
From the qualitative data gathered from the post test questions and general comments gathered during the testing
process strengths and weaknesses of the app interface were noted. Strengths were:




Use of colour in level was good, kids liked clicking on levels and easily identified levels and was a good fun
factor.
Use of numbering system on levels were praised by adult user and also kids seemed to understand level
Idea of reading word and learning them was good for younger kids using app coded by Gold team
Choice of text used on home screen of game was good enough and was praised by adult user.
INFORMATION OBTAINED WITH CONCLUSION QUES TIONS:
Problem 3: Choice of using text in button was right for older kid of 7 and 8 years old, but it wasn't right for kid of 5 or 6
who wants to learn word-using app.
Possible solution:
We could minimise use of text in button, and instead use icons, images etc.
INFORMATION OBTAINED WITH COMMENTS FROM THE USER:
Problem 4: During pilot test, user had mentioned responsiveness problem such as application web page was not
fitting with Android screen.
Possible solution:
Using responsive CSS rules for screen of 500px can solve problem.
Problem 5:
Not all the levels have quiz pages
Problem 6:
The quiz words contained captilised words where the levels did not – fix for uniformity
Possible solutions:
Problem can be solved easily. Our development team has aimed to finish up all levels with quiz pages.
Note: Some of the results obtained with observation and comments were not added to this list as being irrelevant.
These results contain the following:


Some users had their volume set to low so could not listen to words, and were fixed later on.
Some users had problem with Internet connection failing and not hearing sound because of their own natural
problem, they were told by users to testers.
5. CONCLUSION





The use of colours was good and easily identified the levels
Fonts chosen are very appropriate – words are clear and easy to read
The levels are numbered which is good for those not yet familiar with the Golden words system of learning
sight words.
Use of audio made the app very interactive and young users could view harder levels with ease
The lack of a ‘reward’system means that users with learning difficulties are encouraged to use the app
without ‘failing’the levels. The use of the audio button further helps children with learning difficulties.
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 7 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine






User was not able to easily identify how to return to home page.
Young users were not able to understand home button at first time.
The ‘hear word’ button spoke words so fast that users confused about what word they heard.
The words on table such as level 1 to 12, users expected that those to be touched and would take to new
page, but user had to press buttons to get those words. Maybe a swipe screen or words could be linked,
that’s what was expected to happen which did not happen.
On the home screen user was confused about whether to click start button or to press levels on screen
Users gave expectation that quiz page would be on each level (level 1 to 12), but currently we have that on
only ‘gold’ and ‘aqua’ levels.
6. APPENDIX
6.1 CONSENT FORMS STANDARD
I agree to allow my child to participate in the usability study conducted and recorded by
________________________________________________
I understand and consent to the use and release of the recording by CDU students. I understand that the
information and recording are for research purposes only and that my child’s name and image will not be used
for any other purpose. I relinquish any rights to the recording and understand the recording may be copied and
used by CDU without further permission.
I understand that participation in this usability study is voluntary and I agree to immediately raise any concerns
or areas of discomfort my child or I might have with the study administrator.
Please sign below to indicate that you have read and you understand the information on this form and that any
questions you might have about the session have been answered.
Date:_________
Child’s name: ________________________________________________
Please print your name: ____________________________________________________
Please sign your name: ____________________________________________________
Thank you!
We appreciate your participation.
6.2 SCRIPT
Hi, ___________. My name is ___________, and I’m going to be walking you through this session today. Before
we begin, I have some information for you, and I’m going to read it to make sure that I cover everything. You
probably already have a good idea of why we asked you here, but let me go over it again briefly.
We’re asking some children to try using a special mobile web site that we’re working on to help children learn to
read so we can see whether it works as intended. The first thing I want to make clear right away is that we’re
testing the site, not you. You can’t do anything wrong here. You don’t have to worry about making mistakes. As
you use the site, I’m going to ask you as much as possible to try to think out loud: to say what you’re looking at,
what you’re trying to do, and what you’re thinking. This will be a big help to us. Also, please don’t worry that
you’re going to hurt our feelings. We’re doing this to improve the site, so we need to hear your honest reactions.
If you have any questions as we go along, just ask them. But if you still have any questions when we’re done I’ll
try to answer them then. And if you need to take a break at any point, just let me know. You may have noticed
the video recorder. If its ok with you and your mum or dad, we’re going to record what happens on the screen
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 8 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
and our conversation. The recording will only be used to help us figure out how to improve the site, and it won’t
be seen by anyone except the people working on this project. And it helps me, because I don’t have to take as
many notes.
Do you have any questions so far? OK. Before we look at the site, I’d like to ask you just a few quick questions.
First, I’m going to ask you to look at this page and tell me what you make of it: what strikes you about it, whose
site you think it is, what you can do here, and what it’s for. Just look around and do a little narrative. You can
scroll if you want to, but don’t click on anything yet. q Allow this to continue for three or four minutes, at most. 5
Thanks. Now I’m going to ask you to try doing some specific tasks. I’m going to read each one out loud . We’ll
learn a lot more about how well the site works that way. And again, as much as possible, it will help us if you can
try to think out loud as you go along.
Scenario 1
Looking at this screen do you know where to start?
Can you go ahead and click on where you think you might go to next?
So you have a list of words here – have you seen these words before?
What do you think you will do on this level?
Looking at one of the words can you tell me what you think each button will do?
Do you know how to listen to the word?
Do you see how you can get to see the next word?
Going through the words can you understand what happens at the end of the level?
Are you able to understand what to do on the quiz page?
Can you see what button you might press to get to the home screen?
What do you think the colours represent?
Thank the child for being part of the study then stop the video recorder and and escort them out.
6.3 SET OF QUESTIONS USED FOR KIDS
.
What is your name?
2.
What is your age?
3.
What year are you in at school?
4.
What school do you attend?
5.
Do you use the golden words for learning sight words at school?
6.
Do you use a mobile device at home?
7.
What kind of device is it?
8.
How many hours a week do you spend on this device?
6.4 SET OF QUESTIONS USED FOR ADULT USER
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 9 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
List of Tasks : To be handed to user while you are testing on web based application
1.
Looking at this screen do you know where to start?
2.
Can you go ahead and click on where you think you might go to next?
3.
So you have a list of words here – have you seen these words before?
4.
What do you think you will do on this level?
5.
Looking at one of the words can you tell me what you think each button will do?
6.
Do you know how to listen to the word?
7.
Do you see how you can get to see the next word?
8.
Going through the words can you understand what happens at the end of the level?
9.
Are you able to understand what to do on the quiz page?
10.
Can you see what button you might press to get to the home screen?
11.
What do you think the colours represent?
12.
Do You have any comments as how to improve current state of app ?
6.5 SCANNED COPIES OF CONSENT FORMS FOR ALL USERS
Users Tested By Surendra
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 10 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
Mel's Consent forms signed
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 11 of 12
HIT381 Human Computer Interaction Design – Deliverable 4: User Test Results (Part B)
Gold Team - Surendra, Mathew, and Melaine
CHARLES DARWIN UNIVERSITY | 2016 SEMESTER 1
Page 12 of 12