Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
A Heuristic Evaluation of Ohiosci.org
Executive Summary
Site evaluated:
The Ohio Academy of Science
http://www.ohiosci.org/
Goal:
The goal of this heuristic evaluation is to test the overall usability of the
website and to determine improvements that can be made to enhance
the user’s experience.
Method:
The evaluation team consisted of three graduate school students
enrolled in the Usability of Digital Information class at Pratt Institute.
One student served as a lead evaluator. Each evaluator performed two
tasks, using Jakob Nielsen’s ten usability heuristics as a guide, and noted
issues on paper worksheets, then passed their findings to the lead
evaluator, who prepared this report.
General Recommendations:
The Ohio Science website offers an abundance of interesting and engaging information, with
much of its content directed toward students and teachers in university, middle and high school
science programs. Most of the recommendations for improvement are related to format, not
content, and many should be simple to implement. Specific recommendations and detailed
solutions are included in the recommendations section in the body of this report.
Generally, we recommend:









Creating a uniform layout and navigation structure common to all pages.
Dividing content into major subsections, such as Programs, Publications, Events, About Us,
and Membership.
Creating a uniform directory structure and page titles to indicate to users where they are in
the site structure and to help search engines find and index pages properly.
Replacing PDF links with HTML pages to ensure accessibility and a smooth visit for all users.
Creating a style guide governing font and color selection for headings, links and dividers.
Creating a single page for information on Joining, and providing alternative options so users
can decide whether to join and pay online or print and mail in a membership form.
Archiving and removing posts from the home page based on an agreed upon time limit.
Making simple corrections to broken links.
Ensuring that users can always get back to the home page via a link at the top of the page,
such as a logo or site title.
1
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
A Cognitive Walkthrough of OhioSci.com
Table of Contents
Introduction
3
Methodology
3
Findings and Recommendations
4
Limitations
10
Conclusion
10
2
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
Introduction
We performed a heuristic evaluation of The Ohio Academy of Science website, ohiosci.com, to
determine the usability of the site and suggest improvements. A heuristic evaluation is a
usability inspection method in which a team of peers, usually designers, evaluate an interface
against a specific set of heuristics or principles. These principles are more “rules of thumb” than
specific usability guidelines, but are effective in spotting issues that might be in conflict with
website usability.
Methodology
In a heuristic evaluation, the evaluators review a website against a set of usability principals.
(Nielsen, Jakob. How to Conduct a Heuristic Evaluation. Retrieved on October 21, 2012 from
http://www.useit.com/papers/heuristic/heuristic_evaluation.html). The evaluation is conducted
individually by a group of three to five people, then results are presented to the team leader
who compiles a report of the findings. Nielsen recommends three to five as an optimum number
of evaluators to ensure that the majority of usability problems will be located.
Jakob Neilsen’s Ten Usability Heuristics include the following items:










H1: Visibility of system statues
H2: Match between system and the real world
H3: User control and freedom
H4: consistency and standards
H5: Error prevention
H6: Recognition rather than recall
H7: Flexibility and efficiency of use
H8: Aesthetic and minimalist design
H9: Help users recognize, diagnose, and recover from errors
H10: Help and documentation
Source: Nielsen, Jakob. Ten Usability Heuristics. Retrieved on October 21, 2012 from
http://www.useit.com/papers/heuristic/heuristic_list.html
Evaluators attempted to complete two tasks and wrote down any problems they encountered
into a paper worksheet, indicating what the problem was, where it occurred, which of Nielsen’s
heuristics were violated and a severity rating, as follows:
0 = I don't agree that this is a usability problem at all
1 = Cosmetic problem only: need not be fixed unless extra time is available on project
2 = Minor usability problem: fixing this should be given low priority
3 = Major usability problem: important to fix, so should be given high priority
4 = Usability catastrophe: imperative to fix this before product can be released
Source: Nielsen, Jakob. Severity Ratings for Usability Problems. Retrieved on October 21, 2012
from http://www.useit.com/papers/heuristic/severityrating.html.
3
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
The two tasks included locating a way to join the Oho Academy of Science via an online payment
form, and finding the name of the student who received the State Science Day award from
Buckeye Valley Local High School in Delaware, OH. These tasks were selected to represent a
simple task, Joining, which can be completed from the home page, and a second task that
requires browsing more deeply into the site to reveal additional issues.
Findings and Recommendations
The Ohio Academy of Sciences offers an abundance of interesting and useful content, including
public programs, events, history and access to the peer-reviewed Ohio Journal of Science. Ease
of use and access to this important information is a major goal for the website. The evaluation
team noted some issues that should be corrected to improve the user experience of the
website. The following are a list of issues to be addressed, arranged from the most severe to the
least severe.
1. Layout is inconsistent.
#
1
Problem Description
Inconsistent layout
Location
Throughout site
Heuristics
Violated
H1
H2
H4
H8
Severity
4
Figure 1: Home Page
4
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
Figure 2: Some pages, like the Online Store, repeat elements of the home page layout, but
many, like the Mission page, either appear as a text document with minimal formatting or
have very different layouts.
A consistent layout indicates to the user that, no matter which page on the site the user is
viewing, he will know that it is a part of the same site. If the layout is consistent, a user can
understand where he is and how to get back to where he was before. It is also an important
indicator that the user has not navigated to an entirely different website. Finally, a consistent
layout is less jarring, as a user navigates from page to page, because the content “wrapper”
does not change.
The four-column layout of the current home page is a good start. We suggest reducing the
layout to two or three columns with a global navigation across the top and main content within
a wide center column. The same layout should be repeated, globally, on all pages of the
website.
We also note that flagship events, such as the annual State Science Day, could be designed with
their own, separate website, url and layout, similarly to the Ohio Journal of Science. If this is
done, a link should be provided back to the main ohiosci.org website, but it would not need to
be a part of the ohiosci.org global navigation. If State Science Day remains part of the
ohiosci.org site, it should repeat the same overall design as the rest of the site.
2. Global navigation is not uniform.
#
Problem Description
Location
1
Requires global navigation
Throughout site
Heuristics
Violated
H1
H2
H3
H6
Severity
4
Most websites include a global navigation to divide site content into sections and to allow users
5
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
to navigate seamlessly between these sections and back to the home page. A consistent global
navigation allows users to know where they are at any time and allows the user the freedom to
move from one area of content to another without using the browser’s back button. It also
saves the user from having to remember how he got to where he is. Often, secondary navigation
elements are included such as search, a contact method and the site privacy statement. These
are not major content sections, but are useful and often included in a utility navigation at the
top of the page and in the footer.
The Ohio Academy of Science home page has four places that include navigation-like elements:
across the top, in the left column, in one of the right columns and in the footer. Some of these
links, such as the Join page repeat and some, such as the Consent form do not need to be a part
of the global navigation. Many pages, such as the Mission page, shown in Figure 2 above, have
no navigation at all. The website content should be arranged into topics and subtopics to
represent major global and secondary levels of navigation and repeated in a uniform format on
all pages of the website. Typical global sections for a nonprofit website may include Programs,
Publications, Events, About Us, and Membership.
An evaluation of commonly retrieved content, as well as consideration of content that the
Academy would like to highlight, would provide insight in creating an effective the navigation
hierarchy. For example, the annual State Science Day event appears to be an important,
signature event for the Academy. There is enough content about this event to include it as a
major section of the ohiosci.org site, if not a separate, branded website entirely. Once the
hierarchy of major sections is completed, use these section names in the directory names and
page titles to indicate to users where they are in the site structure and to help search engines
find and index pages properly.
3. Many, if not most, pages on the website are PDF format.
#
Problem Description
Location
#
Problem Description
Location
1
PDFs instead of HTML
Throughout site
Heuristics
Violated
Severity
Heuristics Severity
Violated
H2
4
H3
H4
PDF format is useful for documents that the user is intended to download and save for future
use or for forms that must be printed. Web page content and any information that is meant to
be viewed and which might point to additional content on the site should be presented in HTML
format. We recommend converting PDF content into HTML format and integrating it into the
updated layout.
6
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
4. Too many font types, colors and sizes.
#
Problem Description
Location
1
Needs fewer font types and
colors
Throughout site
Heuristics
Violated
H1
H4
H8
Severity
3
Font face and color are also important indicators of content divisions, links, and text that is
meant to be read. The home page in Figure 1 above shows an abundance of font faces, sizes,
text colors and background highlights. Because there are many different font styles on the page,
it is difficult to tell what is a link or a header. Font face and color are not consistent on all pages
and occasionally within the same page, lending an overall look and feel that can be confusing to
users and difficult to read. The result is that many words and phrases compete for the users’
attention.
The best way to fix this problem is to create a style guide to govern the use of fonts and colors
throughout the site. Decide on standard heading and link formats, and a standard font to use for
page content and repeat the use of the style throughout the site.
5. Multiple Join pages.
#
1
Problem Description
Multiple Join
Pages
Location
Heuristics
Violated
http://ohiosci.org/JOIN.pdf
H1
http://ohiosci.org/join.htm
H2
http://www.ohiosci.org/memform.htm
http://www.ohiosci.org/AbouttheAcademy.pdf
http://www.ohiosci.org/store/membership.html
Severity
2
The Ohio Academy of Science relies on member dues to pay for its programs. It is imperative
that its website has an easy to find and easy to use way to become a member. The expectation
with a website is that one should be able to complete the membership process online; however,
it is also good to provide alternatives to users who are more comfortable downloading a form
and submitting payment via conventional post.
This website provides both online and downloadable methods for joining, but there are too
many possible entry points. We found two downloadable PDF forms, one online printable form
in HTML format, one HTML membership page linking to two of the above forms and an online
membership payment form. Because various “Join” links produce different results, it is unclear
which downloadable form or format is the correct and most recently updated form. Certainly,
allow users to choose whether to submit a membership payment online or download a
brochure, but stick to a single form for each process.
7
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
Many Ways to Join the Ohio Academy of Science
Figure 3: HTML Join Page
http://ohiosci.org/join.htm
Figure 4: HTML Online Membership Store
http://www.ohiosci.org/store/membership.html
Figure 5: HTML Join Page 2
http://www.ohiosci.org/memform.htm
Figure 6: PDF Membership Form
http://ohiosci.org/JOIN.pdf
It should be noted that an evaluator found a minor problem with one of the PDF forms (Figure
6) in which a graphic that looks like a button does not behave like a button when it is viewed on
screen. Whichever downloadable member form is selected to serve as the printable form should
be reviewed to ensure that it does not confuse the user while viewing it in a browser.
8
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
Figure 7: Another 2-Page PDF Member Form
http://www.ohiosci.org/AbouttheAcademy.pdf
6. Too much information on the Home Page.
#
Problem Description
Location
1
Too much information on
Home Page
Home page
Heuristics
Violated
H1
H2
H4
H8
Severity
2
The Ohio Academy of Sciences promotes a wide assortment of news, events and programs,
which are presented in blog style on the home page, with each new event or activity posted
above the last, so that the latest is always on top. Chronology is important and the sheer
number of events indicates an impressive amount of activity overseen by the organization;
however, every event and news item does not need to live on the home page in perpetuity. We
recommend archiving events that are older than a date agreed on by management. This could
be events that occurred in a prior term or that are more than a year old. Repeating events, such
as Ohio State Science Day could have their own content section and archive within the site.
9
Evaluation 2: Heuristic Evaluation
Noreen Whysel
LIS 644-01
7. Home page link is broken.
#
Problem Description
Location
1
Home page link is broken
Home page
Heuristics
Violated
H1
H3
H9
Severity
2
Typically a home page link is included in the global navigation on every page and allows a user to
navigate to the main page of a website from anywhere. Often, the logo is linked to the home
page, as is the case on the Ohio Academy of Sciences website. The current website has a
“Home” link on the home page, in the top navigation, that points to “default.htm” and results in
a 404: Page Not Found error. This is not a huge problem, since users don’t usually click Home
when they are already on the home page. But any broken link should be corrected, particularly
one this simple to fix. It is easy to correct by removing the “default.htm” from the anchor
reference. It should also be corrected by implementing a new global navigation, as suggested
above.
Limitations
One of the limitations of a Heuristic Evaluation is that the approach was designed for software
interfaces. A website is more open-ended than a software interface and offers more potential
paths to information. It is difficult to find and evaluate all of the possible paths.
Another limitation is that it is an expert review that does not study the behavior of actual users
of the site in question. An expert review may not find all of the problems that a novice user
would. Nielsen, himself, says that “As a discount usability engineering method, heuristic
evaluation is not guaranteed to provide "perfect" results or to find every last usability problem
in an interface.” (Nielsen, Jakob. How to Conduct a Heuristic Evaluation. Retrieved on October
21, 2012 from http://www.useit.com/papers/heuristic/heuristic_evaluation.html). Nielsen
recommends that any remaining problems could be found by combining heuristic evaluation
with a “simplified thinking aloud test.” (Nielsen, Jakob. (1994). Guerrilla HCI: Using Discount
Usability Engineering to Penetrate the Intimidation Barrier. Retrieved on October 23, 2012 from
http://www.useit.com/papers/guerrilla_hci.html).
Conclusion
The Ohio Academy of Science website has a lot to offer people who are interested in scientific
activities and research in Ohio. The amount of information available on the site represents a
challenge for website designers who want to be sure the site visitors can find the information
they need. This heuristic evaluation has uncovered a number of issues and suggested solutions
that, if implemented, will result in a website that is easy and enjoyable to navigate.
10