Introduction to Web Authoring
Class
mtg.
#21
Ellen
Cushman
cushmane
@msu.edu
www.msu.edu/~cushmane/wra210.htm
www.msu.edu/~cushmane/wa2.htm
Where we are











M 3/17 assignment #4 intro, assign groups, brainstorm orgs.
Note the changes
W 3/19 selecting organizations, problem analysis procedure
in content as per
M 3/24 proposal workshop
3/26 discussion
W 3/26 web standards (proposals due 1-2 pages)
M 3/31 designing test, planning
W 4/2 testing and compliance, executing
M 4/7 visual rhetoric, intro (readability (fonts & white space), rhetoric and psych of
color schemes)
W 4/9 visual rhetoric, 2 (space: real estate, eye/hand movement, information layout)
M 4/14 Workshop usability testing (Progress report due 3-6 slides)
W 4/16 Presentation workshops.
M & W 4/21- /23 Presentations on websolutions Reports due. 4/23
Today in Class | Next Class
Designing a usability test
Activity: design the test
materials with your group
Your test materials should
be done by next class so
we can do a run through
with them.
Test the test materials!
Execute the test!
Analyze the
results!
Revise the test
materials!
Steps in Planning a Usability Test
Determine User Groups We Want To Test
Determine What We Want to Test
Determine Our Test Metrics
Write or Choose our Scenario
Create data gathering tools
Create a test protocol
Create a data analysis protocol
Planning a User Test:
User Groups
Your initial user needs and genre analysis
work (interviews, surveys, etc.) should allow
you to identify goals each user group might
have when visiting the site.
Eg. “prospective members”
Goal: Evaluate the benefits of membership.
Using what? : Info about past events and
upcoming events…(what else?)
Planning a User Test: Scenarios
Use the info you have gathered to write a
sample scenario, including information
about
Who
What
Why
Where
But not How!
Try your own scenario!
Write one about a typical user for your site.
Remember…don’t put “how” info into the
scenario. Stick to: who, what, when, why,
where…
For instance: Elementary school teachers what to attend the Bright Ideas
Conference in the Spring. They hope to see their favorite authors and learn
some new methods in the classroom. They’re around the state of MI, OH,
and IN and they’re coming to MSU.
Determine Who to Test
User Group: Students
•Consider
your design
priorities
•Think about a
representative
sample of the
group you
choose.
Target Audience: Those looking for oncampus housing for the coming year
User Goal: Get first choice of places to
live, roommates, etc.
Determine What to Test
Based on the user goal, list specific, observable
outcomes we can solicit from users in one or
more tasks.
Example Outcomes:
 Starting at home page, locate…
 After test, recall…
 Submit all information necessary to…
 Answer the following question….
Determine Test Metrics:
Performance Criteria
Specific Criteria for Success for each Outcome
 User locates X piece information and writes it down
on test form
 User is able to find and download X to the desktop
(yes/no)
 User is able complete X task in less than 10 minutes,
total; (anticipate sources of non-task related delays)
Note: Common performance metrics are
based on task success, time, and # of errors
Determine Test Metrics:
User Satisfaction Criteria
Specific Criteria for Success for each Outcome
 User finds the site helpful, well-suited to the task (4
or 5 on a 5 pt. scale)
 User finds the site easy to use (4 or 5 on a 5 pt.
scale)
 Users are confident that they completed the task
successfully (4 or 5, etc.)
Note: Common satisfaction metrics are based
on confidence of task success, perceived
difficulty, and frustration level
Write a Scenario + Tasks
 Create a background scenario to orient the participant to their
role and goals…can be drawn from your “use case”
 Create individual task descriptions that match up with each
observable outcome
 Sequence tasks so as to avoid interference issues (e.g. learning
effects. In other words, don’t ask to achieve same outcome
twice because they’ll use what they learned in the first trial to
accomplish the task.)
Create Test Materials
1.
2.
3.
4.
5.
Disclaimer; thank you note.
Background information sheet on user’s “role”
Scenarios & task sheets with blanks for observations and
metrics at bottom.
Questionnaire/Interview Questions: post-task & post-test
Observation notes sheets
Sample task sheet
1. You are considering attending a sailing club event. Use the site
to determine the most appropriate upcoming event for learning
about the club.
For this, we’d want to identify ahead of time the “answers” (info
located on page X) and decide on some metrics for determining
success (most direct route takes 3 clicks off main page).
Leave Space for your observations in the middle of the sheet.
Bottom of the sheet should have your test metric:
Actual number of clicks to accomplish task:
Bottom of the sheet should also have a post-task question.
Sample User Satisfaction Sheet
 User finds the site helpful, well-suited to the task
 5 point scale here
 User finds the site navigation easy to use
 5 point scale here
 User’s frustration level with the site
 5 point scale here
 User finds the site content easy to use
 5 point scale here
 User is confident that s/he completed the task
successfully
 5 point scale here
Test Day! Before the test
Remind all of your users of the agreed upon time, place, etc.
Double check the room and equipment
Be sure all test materials are present
Be sure the computer is in the correct beginning state (turn off
the screen saver)
 Do any pre-test data gathering that is needed




Homework
 Finish your test
materials for next class