Evaluation
Including:
•
•
•
•
•
•
•
Surveys
Questionnaires
Focus Groups
User Trips/Trials
Observation
Testing/Test Rigs
Product Comparison
Surveys
Surveys are carried out to identify specific aspects that the people within a
market group want in a product. They can be carried out manually or
online. Three things that need to be decided before carrying out a survey
are:
•
•
•
Who are the market? Age, sex, etc.
What types of questions should be asked?
How big should the survey group be? The more data collected the more realistic it is.
Questionnaires
Questionnaires are basically the same as surveys, the main difference
being the choice of responses that can be given by the person being
questioned. Unlike surveys where the responder only has a small number
of responses i.e. EXCELLENT, GOOD, OK, POOR, the responses in a
questionnaire can be very individual depending on the question type. The
questions could be:
Closed Questions : resulting in a yes or no response.
Example- Is the weather nice today?
Yes or No
Open Questions : encourages people to give their own opinion.
Example- What do you think of the weather today?
Ratings Questions: an opinion can be expressed very quickly.
Example- Give the weather today a score, 1 being great and 10 being terrible.
Structured Questions: used when detailed/extended answers are required.
Example- How would you describe the weather today, in terms of temperature, brightness, humidity, etc.
Focus Groups
Focus groups allow companies wishing to develop, package, name or test
market a new product, to discuss, view, and/or test the new product
before it is made available to the public. This can provide invaluable
information about the potential market acceptance of the product.
A focus group is an interview, conducted by a trained moderator among a
small group of potential consumers. The interview is conducted in an
unstructured and natural way where potential consumers are free to give
views from any aspect. Today, using audience response keypads to collect
questionnaire answers is the new industry trend.
User Trips/Trials
The most obvious and realistic feedback that a designer could receive on their product
would come from possible customers. This is why user trials and trips often provide the
most valuable feedback on potential products while they are at an early stage of
production.
User Trial
A user trial involves a group of testers who will receive an early production model of the
new product. They will then have a set time to take the product away and trial it. These
trials should be as close to the way a customer would use the product as possible. This
method of testing is a good way of highlighting any faults or problems that might occur
with the product through common use when it is bought. These issues can then be easily
rectified before the product is released for sale later down the line.
User Trips
User trips are very similar to user trials. The difference being that all of the testers will
gather together with the designers to discuss the product as a group. This allows for
opinions and suggestions to be shared amongst a number of people which provides a
group’s perception to the designers. This method focuses more on looks and aesthetics
rather than use.
Observation
Another form of testing that can be useful for a designer is a user
observation. This is a simple method which can provide real time analysis
of a product.
This method is often as simple as it sounds. The design team get together
with a group of testers who will use the model of the product in a
controlled environment.
While this use is happening, the design team will observe and take notes
on what they see. Testing this way allows the designers to see first hand
any issues that arise with use of the product and can think of ways to
rectify these problems. This is also a good way of judging if the design is
ergonomically friendly and comfortable to use.
Testing/Test Rigs
If a designer has came up with a product which goes through certain stresses
when it is used and also may degrade over time with long use. It is essential that
details such as stresses, tolerance and also the lifespan of the design are tested
and known before it is manufactured.
This is where test rigs become a great asset to testers. These rigs are often
mechanical setups which are programmed to run by computer software. The rigs
will then put a model of the product through its paces and put constant strain on
it, while recording the stresses that occur.
Once tested, the design team now have an accurate account of what problems
may occur with prolonged use, whether each component of the product can
withstand the stress of being used and also they will have a better idea of what
the lifespan of the product would be.
This link will demonstrate a test rig which
is on display in an IKEA store:
http://www.youtube.com/watch?v=FVAOl334s5k
Product Comparison
This method of idea refinement can be done in two ways. The first could be that
the team of designers simply compare features, cost, aesthetics and function to
other products which are currently on the market. Obviously the design team
wants their product to be good enough to compete in the market with other
products, so this provides an opportunity to make sure their product is quality
enough to compete.
The other way of testing this way can be done as an extension of observations.
The design team could observe testers using their product along with other
products on the market. This provides a first hand perspective of how the new
product will manage, against the competitors.
This method can provide very accurate and relevant results, but can also lead to
such major changes in a design that the original theme or aim of a product can be
compromised, just to be better than the competition.
Revision Task 7
Answer the following questions in your jotter:
1. What 3 things need to be decided before carrying out a survey?
2. Describe a focus group.
3. What evaluation technique is being described here? I will give my tin opener to 5
people and ask them to open 3 tins each, I will watch and note down anything important that I witness.
4. Copy and complete the table below, some has been done for you.
Factors
Justification
Function
I want to know if the
amount of water held by
the kettle is good
compared to other kettles.
Ergonomics
I want to know if the
handle is comfortable to
hold.
Aesthetics
I want to know if people
like the look of the kettle.
Evaluation Technique
Justification
It is an easy way to look at
how my kettle performs
compared to others
available on the market.
User Trial and survey- I will
ask 5 people to use the
kettle to make a hot drink
and complete a follow up
questionnaire.
Quick and gives me specific
data.