CONCEPT EVALUATION
How to choose the best concept?
 How to decide as a team?
 How to document the process?

CONCEPT EVALUATION

How would a group come to consensus?
 Agreeing
on a process
 Supporting the outcome of the process
 Merit-based debating
 Remaining cool and rational

Emphasis is on discovery
METHODS OF DECISION-MAKING
External decision
 Product champion
 Intuition
 Pros and cons
 Prototype and test
 Decision matrix

CONCEPT EVALUATION

Possible methods of decision making
External
decision: Boss
Product champion: A member
Intuition: Gut feelings and vote
CONCEPT EVALUATION
 Pros
& cons: Discussion & vote of majority.
 Prototype
 Decision
and test
matrix: Each concept is rated w/r to
specifications and ranked.
CONCEPT EVALUATION IN ME 492

Two-stage methodology
 Concept
screening
Initial screening (when > 10 concepts)
Concept screening matrix (> 5 concepts)
 Concept scoring
Concept scoring matrix (< 5 concepts)
INITIAL SCREENING

Multi-vote technique
 Group
eliminates unlikely concepts.
 Allocate
10 dots to each members
 Choose the top 10 concepts
MATRIX METHODS

Process for using matrix methods
 1.
Identify the criteria
 2. Choose a datum
 3. Rate the concepts
 4. Rank the concepts
CONCEPT EVALUATION
 Concept
screening matrix
 Evaluate
concepts w/r to customer needs
 Use most important criteria
 Choose a reference (datum) concept
A
favorite design
 A competitive product
CONCEPT SCREENING MATRIX
 Compare
each concept with the datum concept
 Give
a score of + for “better than” datum
 Give a score of - for “worse than” datum
 Give a score of 0 for “same as” datum
 Rank
concepts
CONCEPT SCREENING EXAMPLE
PDS Criteria
Datum
Wall
Clearance
0
0
0
1
0
-1
Bolt clearance
0
0
0
0
0
0
0
-1
-1
1
0
-1
0
-2
1
-1
1
1
-1
-1
0
1
1
0
0
-1
-1
1
0
1
1
1
1
-1
3
-1
-1
0
0
-1
1
-3
Max Load
Adaptability
Effectiveness
Weight
Cost
Total
B
C
D
E
F
CONCEPT EVALUATION
Combine and improve concepts
 Select promising concepts.
 Reflect on the process.

MATRIX METHODS

It is important to have a single category for
each important criteria. For example do not
breakdown “cost” into:




Cost
Ease of manufacture / Assembly
Material availability
Labor
This practice inflates some criteria
MATRIX METHODS

If it is preferred to break down criteria, make
sure the “importance” is also divided between
them:


Importance 5
Or




Cost
Materials
Labor
Tooling
2
2
1
This method can be used in Scoring Matrix
CONCEPT SCORING MATRIX

Concept scoring matrix


The main criteria are listed as before
Determine a weighting factor for each criteria


Done in QFD (as importance)
Rating the concepts (1-5):
1 : Much worse than reference concept
 2: Worse than reference concept
 3: Same as reference concept
 4: Better than reference concept
 5: Much better than reference concept

CONCEPT SCORING MATRIX

Alternate Absolute Scale Scoring
 Rating
1
the concepts (1-5):
: Useless – Very inferior
 2: Poor - inferior
 3: Acceptable
 4: Good or superior
 5: Excellent or much superior
EXAMPLE

Consider buying a TV
 Selection
Criteria
 Cost
 Screen
size
 Image clarity
 Warrantee
 Looks
EXAMPLE

Brand-A


Brand-B


Cost: $175, size: 19”, 6 month, 0.19mm, Beautiful
Brand-C


Cost: $350, size: 25”, 1 y, 0.28mm, OK looks
Cost: $430, size: 27”, 2 y, 0.28mm, Very ugly
Brand-D

Cost $275, size 21”, 1 y, 0.15mm, ugly
CONVERSION OF CRITERIA
Excell Good Adeq Poor Useles
Cost
<$200
2-300
3-400
4-500
Size
>=27"
25-27"
21"-25
19-21" 13-19"
Warr
>4 y
2,3
1
.75
.25
Image
<0.1 mm 0.15
0.20
0.25
>0.3 mm
Looks
Snazzy
Ugly
Very
Ugly
Beautiful Not too
bad
>$500
EVALUATIONS MATRIX
Weight Brand-A Brand-B Brand-C Brand-D
Cost
0.2
3
5
2
4
Size
0.3
4
2
5
3
Warrty
0.1
3
1
4
3
Image
0.3
1
3
1
4
Looks
0.1
3
4
1
2
2.7
3.0
2.7
3.4
Total
CONCEPT EVALUATION



Rate and rank concepts
Combine and improve
Select one concept



Consider variation in rates and weights on the final results.
Assess how much difference is really significant.
Reflect on the results

This is “the point of no return” - Everyone on the team must
be in agreement about it.
REFLECT ON THE RESULTS
 Was
your favorite option selected? If not,
here are your options:
1. Ask for a second round of debating
2. Study the decision matrix, why did your
design score lower than others?
3. Can you modify your design to address
weaknesses
4. Did the team overlook strengths of your
design?