Usability Test Results for “Fast Food Drive-Thru”

Final Report
Systems 323
December 5, 2001
Prepared by:
Christy Barnett
Bob Etris
Rob Janiczek
Jason Kearns
Dustin Marks
Carolyn Quil
http://www.people.virginia.edu/~rlj5n/newsys323
Table of Contents
I. CollegeQuest – Planning
 Project Goals………………………………………………………………………1
 Target User Population...………………………………………………………….1
 Intended Functionality.……….……………..………………………...…………..2
 Market Assessment Efforts……………..…………………………………………3
II. CollegeQuest - Implementation
 Product Description…………..……………………………………..…………….4
 Interface Design…………………………………………………………………...5
 Usability Test Design and Results……….………………………………………..7
III. Further Discussion
 Maintainability…………………………………………………………………...11
 Future Developments….…………………………………………………………12
Appendix A: Screen Shots – Market Survey………………………………………... A-1
Appendix B: Screen Shots – CollegeQuest………………………….…...………......B-1
Appendix C: Testing Instructions………………………………………..….……..…C-1
Appendix D: Questions/Discussions……………………..………………………..…D-1
Appendix E: Network Interaction Model………………….…………………..…..…E-1
Appendix F: Design Space Analysis…………………………….………………..….F-1
Appendix G: CollegeQuest Walk-Through………………………………….…….…G-1
CollegeQuest - Planning
Project Goals_____________
Our group originally started with a broad idea of using a college search engine to
give the user an optimally planned path along which they could visit schools of their
choice. Further development of this idea led to the following goals:

Provide users with a convenient, easy-to-use product with which they could plan
tours of multiple colleges

Produce an attractive, well designed website

Create a product offering enough functionality to ensure its success without
adding unnecessary options that may detract from the site’s main focus
Further, less explicit goals included completing the product on time and maintaining an
even work distribution among group members.
Target User Population
CollegeQuest’s original primary user population was juniors and seniors in high
school. However, after some thought it seems more intuitive that the parents would play
a large role in planning their children's college tours. Hence, an effective product would
provide an interface functionally appealing and appropriate for both. The user population
also includes guidance counselors and other individuals who assist these students in the
college selection process.
1
Intended Functionality________________
CollegeQuest serves two main purposes. The first is generating a shortest path
itinerary to visit colleges selected or inputted by the user. The tool searches a database to
arrange the optimal trip route, therefore minimizing travel time and eliminating
backtracking. Also, the tool searches available tour times at each school. This itinerary
provides the user with a daily schedule, consisting of tour times, visiting information, and
a profile of each school, as well as driving directions and maps. All of these features
would vastly reduce planning time by combining all the necessary resources into one
website.
The second purpose is allowing the user to search for colleges if they have not
already done so. The site offers a wide variety of search criteria including school type,
majors offered, size of school, cost, geographic location, and average SAT and GPA
range (see screen shots in Appendix B for detailed layout). It then generates results that
match their criteria and provides brief facts for each school in tabular form and a link to a
more detailed college profile. After a user has browsed this information, they can select
which schools they desire to visit and plan a tour. Also, the tour planner may generate
schools along the route that may also interest the user.
Additionally, CollegeQuest offers a member sign-in area. This allows both
frequent and first-time users to save multiple itineraries, saving the effort of having to recompile the trip schedule. The sign-in area also allows the user to change their password,
their username, and displays an error message if an incorrect password is entered. If the
user has forgotten their password, it can be sent to them through a secure e-mail
connection.
2
Market Assessment Efforts
_____
Before beginning product development, a rudimentary level of task analysis was
necessary to get a better feel for what functionality users would desire in this sort of
product. Therefore, our group decided to post a survey online, and distribute it to a
number of high school seniors. Screenshots of this web page and sample questions
appear below in Appendix A of the report.
A total of 14 individuals responded to the survey, ten females, and four males.
Eight thought they might be inclined to pay for the type of service CollegeQuest offers,
while six did not. As a whole, they appear to use a variety of materials to assist them in
their search, relatively equally distributed among guidance counselors, on-line materials,
books, and relatives/friends. Forty two percent voted their biggest determining factor in
choosing a college to be type of majors offered, chosen over such other choices as
reputation, cost, enrollment, and geographic location. Finally, the feature voted “most
useful” in a website designed to assist the selection process was general information on
each college (23.53%), with a “list of additional schools that may be of interest” coming
second (19.61%) and both a general search engine and information on tour dates each
garnering fifteen percent of the vote.
3
CollegeQuest - Implementation
Product Description
_____
CollegeQuest is a web-based college tour-planning tool. There are two basic
functions as outlined in the Intended Functionality section of the report. When the user
chooses the Tour Planner button, they input their starting point (address information),
followed by the schools they wish to visit and the dates for they desire to travel. Though
initially only up to five schools can be accepted, clicking the “I want to visit more than
five schools” link doubles the capacity to ten. CollegeQuest, in turn, outputs a trip
itinerary connecting the various tour destinations by the shortest route possible. This
College Visit Itinerary page also offers a map with both the route and various stops
highlighted. In the daily agenda, the user can access hotels and restaurants in the area,
learn more in-depth visitor information, and view a college profile for each of the schools
on the route. If the user finds the single map at the top of the itinerary page insufficient,
they can click “View Detailed Driving Directions” to see step-by-step directions between
each destination point with a map of each section of the route. Also, it provides a
printable format that the user can print out and take along with them on their trip.
Additionally, if the user forgot to input one of the schools they wanted to visit, they can
click on “Modify this Itinerary” to add it to the route and regenerate their trip plan (see
Appendix B for screen shot). The user can then save these itineraries if they are a
member for later viewing and modification.
If the user arrives at CollegeQuest and does not know which schools interest
them, they can choose the College Search button. At the College Search page, the user
4
can enter a variety of search criteria with which CollegeQuest returns all the colleges
fitting the user's preferences. This criteria includes type of school (public or private),
majors offered, size and cost range, geographic location (region or state), and SAT and
GPA range. The Search Results page returns, in tabular format, quick facts about the
colleges meeting the search criteria, as well as links (College Profile) to additional
information on the schools. From this point, the user can either click “Plan My Tour” to
go to the Tour Planner page (after selecting schools they desire to visit from the list), or
click “Change My Search Criteria” if they would so desire. The search criterion appears
above this button to provide the user immediate feedback and prevent problems with the
gulf of evaluation.
The member sign in area provides all the same features described above in the
Intended Functionality section (see Appendix B for all screenshots of all CollegeQuest
pages).
Interface Design
As with any good design, CollegeQuest attempts to exhibit an attractive user
interface that maximizes utility to the user while minimizing confusion. The original
color scheme of predominantly lime green on white background was discarded in lieu of
a more attractive set of blue tones. Much of the unused white space appearing in the first
generation prototype is now filled with pictures of college age students interacting on
university campuses. The development team worked towards striking a balance between
making it interesting enough to keep a graduating senior excited about going to visit
schools, while keeping a professional, utilitarian feel for parents setting up these various
college tours.
5
At first glance, CollegeQuest may appear to be a simple website. It has three
basic options (Tour Planner, College Search, Sign In), but there is a lot more going on in
secondary and tertiary layers offering the user additional functionality. For example, on
the College Visit Itinerary page, the trip plan itself appears basic and straightforward (an
intended design attribute). However, it offers hotel and restaurant links, along with
visiting information including campus maps, admissions office contact information and
general school statistics. On the surface, CollegeQuest is primarily a dedicated college
tour planner service, while at the same time offering access to the type of information one
might find at other college search websites. Additionally, the College Search option
itself offers the user many customizable features. After matching colleges are returned
from a search, the user can select those they desire to go visit and CollegeQuest enters
them automatically into the Tour Planner page, which helps minimize the necessary
amount of effort on the part of the user to complete their intended task. In general,
CollegeQuest attempts to do the majority of the hefty processing work behind the scenes
to ensure the user enjoys a pleasant trip planning experience.
One constant design feature is that the toolbar at the left of each page nested in the
darker blue background (see appendix B for screenshot) remains on each page. We
attempted to minimize navigation by implementing the Modify this Itinerary! and Change
My Search Criteria! Buttons, which send the user back to their previous page. While the
user remains on a particular page, the button appears to have disappeared leaving only
text in its place, indicating that selecting that page no longer is an option as long as it is
active. Also, the simple layout of the homepage offers users only the information and
links to those features they need to start their college tours. The fading sign-in text just
6
below the title bar increases salience, drawing attention to the user, but remains constant
once in full view so as not to be a distraction. The toolbar buttons on the left of the
screen are listed in the same order as their description and in-text links to the right,
affording the user to assign a natural (intended) relationship between the two elements.
Distracting elements such as moving .gif files or pop-up ads are left out.
In addition, the sign-in page gives the user feedback through each stage of the
process. If the user enters an incorrect password, an error message appears. Once the
login is successful, text just below the title-bar at the top of the screen welcomes them
and provides a link to view their saved trip itineraries. A clearly marked “new user”
registration page takes their necessary information and allows users to choose a username
and password to generate a new account.
Usability Test Design and Results
Four usability tests were conducted on CollegeQuest using two sets of
instructions. The first of these supplied the test subject with a set of two tasks to
introduce the basic functional features of the website. These instructions appear in
Appendix C of the report. These instructions asked the test subject to plan a trip to three
different schools using the Tour Planner as well as find an available hotel and restaurant
in the area. In the second task of the test, the test subject searched for a public or private
university in the northeast, with size greater than 5000 and offering a pre-med major.
The questionnaire/comments, appearing in Appendix D, asked the test subject
some general questions about the overall design and layout of the website. Two members
of the CollegeQuest development team oversaw the testing of each test subject, taking
notes as they completed the task sheet and questionnaire. At the end of each test, the
7
team members and test subject exchanged ideas on their general impressions of the
process and additional changes/editing necessary.
This usability testing stage of the development process brought to light a number
of changes that the development team decided to implement. Some general suggestions
included making more dynamic use of the white space. In order to achieve this balance,
more images were added, particularly on the homepage. Some of the links such as that
for the “sign-in” page and some additional functions on the itinerary page were not yet
functional. Test subjects suggested implementing the member sign-in area at the bottom
of the page and including a link for “first-time visitors,” affording new users access to an
account. In the final design, the development team instead implemented options to both
sign in (for current users) or to create a new account.
It was also suggested to eliminate the constant blue color on the outside of all
pages, either by expanding the content on all the pages to take up more space or by
removing it altogether. We left the blue color to account for resolution differences,
however, in the true implementation, we could fill this space with advertisements. On the
homepage, better alignment between the taskbar on the left and the corresponding
description on the right would help to clarify that the descriptions match up with the
buttons. In our implementation, we decided to expand the description of the website, and
therefore this change was not necessary. Also, the colors of the in-text links needed to all
match; those on the itinerary page had not as of yet matched those of the other pages.
There were also several suggestions for improvements on the itinerary page. The
hotel and restaurant links on the itinerary page needed to be more salient. Possible
solutions included changing the positioning, coloring, or size of the links. Additionally,
8
when the subject clicked on a link there was no way to get back from the hotel and
restaurant sites to our page. To solve this, the links now generate pop-up windows,
allowing easier navigation between layers.
Many of the test subjects identified an unclear wording scheme within the College
Visit Itinerary. For example, describing going to UVA as "Drive to Charlottesville"
might leave some users wondering where they are going if they don't know where UVA
is located. Subjects further suggested avoiding using abbreviations in order to maintain
consistency and a professional appearance. From one test subject’s suggestion, we set
borders around the itinerary, as well as a gray background for color contrast to make it
more salient. The test subjects deemed the “quickest route” page and “1 more day route”
links unnecessary; therefore we eliminated them. Finally, the map provided at the top of
the page needs more options. Zoom functions would be particularly helpful, in order to
optimize the user’s ability to view the detailed map. Due to time considerations, this will
be a future implementation.
On the search page, one test subject noted that the map did not display individual
state boundaries that could lead to confusion about which state belongs in each region
(particularly with those states bordering two regions), therefore we changed the map so it
now includes the state borders. There were also alignment problems caused by differing
resolutions. In order to rectify this, the information on the page is aligned vertically
instead of horizontally. The page also now clarifies which categories allow multiple
selections. One final addition was to expand the functionality of the search engine by
adding options such as SAT scores, tuition, grade point average, etc, which were all
implemented by our team.
9
In conclusion, the test subjects seemed support the project concept and interface.
The theme of the suggestions during the course of post-evaluation conversation seemed
to be "make me want to use this more." The functionality is acceptable, considering the
preliminary stages of the work, but the appearance could be improved. It needs to be
more attractive without compromising functionality.
After making several of these changes, an additional, less formal usability test
was performed. The test subjects included parents, a high school student, and a college
student. The final changes were determined by the suggestions made in this process.
10
Further Discussion
Maintainability
CollegeQuest’s database will require continuous maintenance as new contact
information becomes available, additional schools are added, and new features are
implemented. At the bottom of the homepage, questions and comments are directed to
the Webmaster’s mailing address. In this way, guidance counselors, college
administrators, or anyone looking at CollegeQuest can offer new information or new
ideas for improvement. A full staff of approximately ten to fifteen employees would run
the day-to-day operations of the business. This includes database management
technicians, programmers, website designers, hardware engineers, and customer relations
(to recruit new schools to publish their information through CollegeQuest.
Furthermore, funding presents a large problem. The major production costs
include labor and hardware. One possible solution is to charge a nominal fee to become a
member, though many potential customers might not be willing or able to pay this.
However, in our online survey, many students responded that they would pay for such a
service. Another possibility would be to charge a small percentage of the business our
website generates for the hotels and restaurants listed through the itinerary page. Though
possibly more feasible, this would be difficult to track without member customer
numbers or the like to track how much business is actually generated. One final idea
would involve charging individual high schools a fee to have a computer offering the
website in their computer labs, although this option eliminates a large percentage of the
intended user population: the parents.
11
Future Developments
The development team hopes to implement a number of features in the future that
were not possible for the purposes of this project, due primarily to time and functionality
constraints. The main goal would be to compile a comprehensive database from which
the search results would be generated and tours planned. Also, the true optimal path
would be calculated according to locations and available tour times.
The button bar on the left will be made a frame, so that it stays constant even
when the user scrolls through the information to the right, similar to the current header
frame. Also, to make the sign in option at the top more salient, the text will gradually
appear to its appropriate space. In addition, when the user is signed in, the system would
recognize their home address, GPA, and SAT scores, as well as allow them to make
changes to this information. They could also save their search results each time a query
is run.
Another change might be to develop a “Links of interest page” to offer customer’s
links to similar sites offering information CollegeQuest lacks. Examples of such sites
include Kaplan, Princeton Review, Orbitz.com (travel site), AllAboutCollege.com, and
Expedia. This feature may, however, unintentionally draw potential customers away
from CollegeQuest, an undesirable feature. Additionally, a FAQ’s page would cut down
on potentially needless correspondences with the Webmaster and guide first time users.
The current implementation of the search page only allows users to pick an entire
region or a single state, but nothing in between. A more robust design would include
options to pick multiple states at once, particularly if they are not in the same geographic
region. CollegeQuest will further have a couple of preventative measures not currently in
12
place. For example, clicking on the Save Itinerary button will cause the system to check
if the user is currently logged in, and if not, log the user in before storing the data
permanently. Also, if the user misspells the school’s name, CollegeQuest will offer
suggestions based on both grammatical similarity and similarity of the search criteria.
Finally, eventually CollegeQuest will appeal to graduate students as well. This option
will likely be implemented within the search page as checkbox (undergraduate vs.
graduate) or similar design, appearing the same way in the tour planner option.
On the itinerary page, CollegeQuest will eventually allow the user to make more
specific modifications to the itinerary generated from the Tour Planner function. Options
to switch schools order or edit the suggested route within the itinerary page would keep
users from having to switch too much between pages using the back button or navigation
bar. The trip map at the top of the page will offer a zoom function. Within the text box of
the itinerary, in addition to test links for hotels and restaurants, CollegeQuest will link to
“local attraction/events” updates so users may better plan social events around town
while on their trip. Finally, the suggested schools along the same route will link to
“Visiting Info” and “College Profile” information just as those schools in the trip plan do.
These conveniences should make the trip itself more customizable for the user.
13
Appendix A: Screen Shots – Market Survey
Location:
http://www.votations.com/asp/survey.asp?pollid=30772&EmailID=32091&ValidationID=231962&lang=
Results Displayed:
A-1
Questions Include:
 How are you currently searching for colleges?

What factor is most important to you in choosing a college?

Which features would you find useful in a website designed to assist in the
college selection process? (college search engine, trip planner, general
college information, nearby attractions, list of colleges that interest you, tour
information, and visitor information)

Would you pay for the features you selected in the previous question?

What problems have you encountered in your college search?

How could your college search be simplified?

Are you male or female?
Notice two key points 
Votes for a college trip planner only totaled 11.76%, but a high school student taking
this survey may not know what features a “college trip planner” would offer.

After some consideration within our group, a consensus was reached that the trip
planner would be more useful for the parents actually planning their son/daughter’s
college tour more so than for the students themselves.
A-2
Appendix B: Screen Shots – CollegeQuest
B-1
B-2
B-3
B-4
B-5
B-6
Appendix C: Testing Instructions
Task 1:
 Make an itinerary that includes visits to Georgetown, University of Virginia, and
Virginia Tech.
o Use “BWI” as your starting address
o Leave Method of Travel and Dates as default values
 Find a hotel to stay at during your visit to Georgetown.
 Find a restaurant to dine in near University of Virginia.
Task 2:
 Search for a public or private university in the northeast, with size greater than
5000, with a pre-med major.
 Select all the schools returned by the search and plan a tour using those schools.
C-1
Appendix D: Questions/Discussion
What suggestions do you have for the home page/general set up?
How did you like the search page?
Do you think it is worthwhile on our page?
Would you rather use an existing search engine? (Princeton Review, Kaplan, etc)
Would you like an option to create a personal profile and save your created itineraries?
Layout pros and cons?
Any links with unintended consequences?
Color scheme?
D-1
Appendix E: Network Interaction Model
A visual representation of CollegeQuest:
HOMEPAGE
TOUR PLANNER
SIGN-IN
SEARCH
CHANGE
SEARCH
CRITERIA
NEW MEMBER
SEARCH RESULTS
ITINERARY
OLD MEMBER
PLAN TOUR
MODIFY
ITINERARY
NEW MEMBER
REGISTRATION
LOG OUT
VIEW
SAVED ITINERARIES
DETAILED DRIVING DIRECTIONS
Note – Bear in mind, at any point the user can reach the homepage, trip planner, search
page, or sign-in page. The above is a natural progression of stages as the plans a trip to a
variety of schools
E-1
Appendix F: Design Space Analysis
No relation
Yes relation
Rising College Freshman
Strong Yes relation
Visual Appeal
Ease of Use
Identify Target Audience
Parents
Cost
“Fun Factor”
Both
Usefulness
Trip Directions
What functionality?
College Trip Planner
Search by region
College Search Engine
Search trip if
know destinations
Admissions Info
Admission Info Tool
Campus Map
Search trip if don’t
know destinations
Time Constraints
Error Messages
Implement which
design features?
User Feedback
Degree necessary
to show functionality
“Other Useful Links“
Feasibility
Actual Database Search
Demonstrate
Consistency
F-1
Appendix G: CollegeQuest Walk-Through
The link to the homepage is http://www.people.virginia.edu/~rlj5n/newsys323
Currently, the page is best viewed at 1024 x 768 resolution.
 Create a new account by clicking on Create New Account (found in the top right
corner underneath the header)
o Your username must equal your password (limited to 8 characters), and all
other fields may be filled in any manner.
 Check to see that you have signed in by viewing the welcome message found just
below the header.
 Click on Tour Planner button.
o On this page, choose BWI from the airport list as your starting point.
o Enter University of Virginia, Virginia Tech, and Georgetown University
as your schools to visit.
o Choose April 12, 2002 as your available start date, and April 18, 2002 as
your end date. (These are all hard coded to show what functionality the
database would have)
o Now choose the Make my Trip! button to see your College Visit
Itinerary.
 On the College Visit Itinerary page, you will find a number of links that help the
user gain easy access to important information.
o Click on the hotels link to find lodging at one of the schools
o Click on the visiting info link for campus maps, tour times, admissions
contacts, and more.
o After trying out the options on this page, locate the Save Itinerary link.
 Save the itinerary as Rob’s trip
 Next, click on the College Search button.
o On the College Search page, you will find many options to search by.
 Choose a public or private college
 Choose your major to be Engineering, General.
 Choose the size of your college to be above 5,000, and choose the
tuition to be in any price range.
 For your location, select the northeast and south regions.
 In the personal information section, choose SAT scores between
1200-1400 and a GPA between 3.5-4.0.
 Click the Search button.
G-1
o The Search Results page should return 3 schools, UVA, Tech, and
Georgetown
 Click on a college logo or name to link to that college’s homepage
 Click on College Profile to view more detailed statistics of the
school
 Select the University of Virginia, Virginia Tech, and Georgetown University as
the schools you wish to visit by checking off the boxes next to the school name.
o Now you can Plan your Tour!
 You are directed back to the Tour Planner page, but notice that certain fields are
pre-populated.
o At this point, a tour can be planned as described above
 Now Log out of the system.
 Sign in as username Rob, password Rob.
o View your saved itineraries.
o Erase Rob’s Trip.
 Log out of the system.
G-2