Transitioning from Parametric to Buildup
Estimates
Allison Wiley, Megan Dameron, Sarah Grinnell
Brian Brophy, Dick Coleman, Jessica Summerville
June 17, 2004
Outline
• Introduction
• Explanation of the Challenge
• Developing a Buildup Estimate
• Parametric Pullout Using a Toy Problem
[email protected], 703.961.3432
2
Introduction
• Purpose of this Presentation
– Discuss the challenge presented in transitioning from
parametric to buildup cost estimates
– Present some of the ideas and approaches
considered to address the challenge
– Generate ideas and discussion with the audience in
order to advance our thinking
[email protected], 703.961.3432
3
The Challenge
• The transition from a parametric to a buildup cost
estimate
– Develop cost estimates specific to a subsystem
– Calculate the appropriate amount to remove from the
parametric estimate for the system, in order to insert
the buildup estimate
• Utility
– Provides detailed information so that the cost of
operations and support or specific production items
can influence the system design
– Buildups are instructive and can lead to
improvements in parametric estimates
[email protected], 703.961.3432
4
Developing a Buildup Estimate
• Why the transition?
– There could be subsystems that behave in a way that
is inherently different from the legacy subsystems in
the cost data
– It may not be possible to estimate the entire system
at a detailed level, but there may be detailed
information available for one or more subsystems
• Defining “buildup”
– In this presentation, the word buildup refers to an
estimate specific to a subsystem
– Intuitively the authors had in mind traditional
buildup estimates as well as analogies
– Parametric estimates of a subsystem alone could
also be considered
[email protected], 703.961.3432
5
Parametric Pullout
• The term “parametric pullout” refers to the amount of the
parametric estimate that is attributable to a specific
subsystem – in short, the amount of the parametric
estimate that is “pulled out” so that the buildup estimate
may be inserted.
• The idea of parametric pullout and some of the methods
considered to address this problem are explained in the
next several slides with the use of a toy problem.
[email protected], 703.961.3432
6
Parametric Pullout
• Introduction to the toy problem
– You have a CER that estimates the cost of a car
based on its weight
– To reduce confusion, let’s call the car the TE-4
– You have recently developed a buildup estimate for
the transmission alone, and you hope to eventually
have buildup estimates for most of the major parts of
the car
– Let’s call the transmission the BE-2
– The challenge: Incorporate the buildup estimate for
the BE-2 into the total cost estimate for the TE-4
[email protected], 703.961.3432
7
Parametric Pullout (cont’d)
• Method 1: Historical Percentage
– If the transmission of a car is historically 8% of car
cost, remove 8% of the TE-4 estimated cost
– Advantages: Easy to execute, also works for costs
estimated using historical averages instead of CERs
– Disadvantages: Method requires specific historical
data that may not be available in all cases
Legacy Tranmissio n Cost
 Pullout Pe rcent
Legacy Car Cost
Parametric Pullout for BE - 2  Pullout P ercent  TE - 4 Estimated Cost
[email protected], 703.961.3432
8
Parametric Pullout
• Method 1: Historical Percentage
– Toy Problem
Historical Data
Transmission
Car Cost
Cost
Percentage
$28,244
$25,879
$23,482
$21,137
$18,730
$16,380
Average
$
$
$
$
$
$
4,213
3,914
3,615
3,316
3,018
2,719
14.9%
15.1%
15.4%
15.7%
16.1%
16.6%
15.6%
$25,680 x
15.6%
Parametric Cost
Estimate for New Car
Pullout Percentage
Pullout Amount
Transmission Buildup
Estimate
Adjusted New Car
Cost Estimate
$
$
25,680
15.6%
4,015
$
3,650
$
25,315
$25,680 $4,015 +
$3,650
[email protected], 703.961.3432
9
Parametric Pullout
• Method 1: Historical Percentages
– Concerns
– Consider the case where the BE-2 comprises a
much larger percent of TE-4 weight than any
legacy transmission and car. There is a concern
that since the BE-2 weighs so much more, the
percentage method may cause the removal of an
inadequate amount of cost.
[email protected], 703.961.3432
10
Parametric Pullout
• Method 2: Parameter-based
– Use the parameters of the CER to determine the
correct piece to pullout
– Advantage: Requires little data and is easily executed
– Disadvantage: Subsystem may not have parameters
comparable to the parent system. For example, there
is not an intuitive way to pull the cost of a tire out of a
car CER that uses the weight of the car’s electrical
system.
Tip: It is frequently best to take the CER result from the parameters of
the whole system, and the CER result from the parameters of the whole
system minus the subsystem and note the difference. This is especially
important when CERs have multiple variables and/or are non-linear.
[email protected], 703.961.3432
11
Parametric Pullout
• Method 2: Parameter-based
– Toy Problem
– Run the car CER on the weight of the entire car
– Run the car CER on the weight of the entire car, less the
weight of the transmission
– Subtracting the latter from the former yields the amount to be
removed
– Add in the buildup estimate for the transmission
$30,000
Pullout
Amount
Cost of Car ($)
$25,000
$20,000
$15,000
$10,000
$5,000
Car
Weight
Car Wgt. Trans. Wgt.
$0
1,000
2,000
3,000
4,000
5,000
6,000
Car Weight (lbs)
[email protected], 703.961.3432
12
Parametric Pullout
• Method 2: Parameter-based
– Concerns
– In a CER for a total system model, the parameters
may mask or interplay with other parameters
– For example, in a car CER based on electrical system
weights, the electrical system weight acts as a proxy for
other system weights (like the tires, the seats, and the frame
of the car).
– Intuitively, removing a few pounds of electrical system
weight removes an electrical component, but
mathematically it also removes all of the frame of the car
that supports the electrical component.
– It is important to understand the meaning of the CER.
[email protected], 703.961.3432
13
Parametric Pullout
• Method 3: Obtain new CER
– Example:
– Y = legacy car cost – legacy transmission cost
– X = (legacy car weight – legacy transmission weight))
– Results should yield reasonable F and t statistics, and a
reasonable R2
– Use TE-4 minus BE-2 weights in the new equation
– Subtract the new result for a “transmission-less car” from
the existing result for all of TE-4 – this is the pullout
amount
– Advantages: Provides a good check of the existing CER
– Disadvantages: Requires significant resources and data to
accomplish, some CERs will not respond well to this method
without being completely reworked
[email protected], 703.961.3432
14
Toy Problem (cont’d)
• Method 3: Re-run CER
– Return to the original data that produced the CER
Car Weight
(lbs)
5,000
4,500
4,000
3,500
3,000
2,500
Car Cost = 4,499 + 4.75(Car Weight)
Car Cost
$28,244
$25,879
$23,482
$21,137
$18,730
$16,380
R2 = 0.99
t and F significant
Estimated weight of new car: 4,000 lbs
Estimated cost of new car: $23,496
– Subtract the cost of the transmission from the cost of the car,
and the weight of the transmission from the weight of the car
Car Weight Transmission
(lbs)
Weight (lbs)
5,000
550
4,500
515
4,000
480
3,500
445
3,000
410
2,500
375
Car Cost
$ 28,244
$ 25,879
$ 23,482
$ 21,137
$ 18,730
$ 16,380
Transmission
Cost
$
4,212
$
3,914
$
3,614
$
3,317
$
3,017
$
2,719
"Transmission-less
Car" Weight (lbs)
4,450
3,985
3,520
3,055
2,590
2,125
"Transmissionless Car" Cost
$
24,032
$
21,965
$
19,868
$
17,820
$
15,714
$
13,661
[email protected], 703.961.3432
15
Toy Problem (cont’d)
• Method 3: Re-run CER
– Rerun the regression (ensure that t and F statistics
are still significant, and R2 is reasonable)
– Run the new CER on the weight of car to be
estimated, minus the weight of the transmission
"Transmission-less
Car" Weight (lbs)
4,450
3,985
3,520
3,055
2,590
2,125
"Transmissionless Car" Cost
$
24,032
$
21,965
$
19,868
$
17,820
$
15,714
$
13,661
“Transmission-less Car”Cost = 4,167 + 4.46(Car
Weight – Transmission Weight)
R2 = 0.99
t and F significant
Estimated weight of new car w/o transmission: 4,000 –
500 = 3,500 lbs
Estimated cost of new car w/o transmission: $19,791
Estimated Cost of Car
Estimated Cost of Car
w/o Transmission
Parametric Pullout
for Transmission
$
23,496
$
19,971
$
3,525
[email protected], 703.961.3432
16
Parametric Pullout (cont’d)
• Method 3: Obtain new CER (cont’d)
– Concerns
– This method may not behave intuitively
– Changes are observed in the coefficients of unchanged
parameters (when multiple parameters are present)
– What is the expectation for behavior?
$30,000
Pullout
Amount for
Method 2
Cost of Car ($)
$25,000
If Method 2 and Method 3
are to yield the same
result, the new CER must
intersect the Car CER at
the weight of the
“transmission-less car”
$20,000
$15,000
$10,000
Car CER
Possible “Transmission-less
Car” CER
$5,000
$0
1,000
2,000
3,000
4,000
5,000
6,000
Car Weight (lbs)
Case 1: Method 2 and Method 3 yield the same result
[email protected], 703.961.3432
17
Parametric Pullout (cont’d)
• Method 3: Obtain new CER (cont’d)
– Concerns (cont’d)
– What is the expectation for behavior?
$30,000
Pullout
Amount for
Method 2
Cost of Car ($)
$25,000
$20,000
$15,000
Car CER
$10,000
Possible “Transmission-less
Car” CER
$5,000
$0
1,000
2,000
3,000
4,000
5,000
6,000
Car Weight (lbs)
Case 2: Method 3 removes more cost than Method 2 but the new
CER behaves intuitively
[email protected], 703.961.3432
18
Parametric Pullout (cont’d)
• Method 3: Obtain new CER (cont’d)
– Concerns (cont’d)
– What is the expectation for behavior?
$30,000
Pullout
Amount for
Method 2
Cost of Car ($)
$25,000
$20,000
$15,000
Car CER
$10,000
Possible “Transmission-less
Car” CER
$5,000
$0
1,000
2,000
3,000
4,000
5,000
6,000
Car Weight (lbs)
Case 3: Method 3 removes less cost than Method 2, the new CER
may not behave intuitively
[email protected], 703.961.3432
19
Parametric Pullout (cont’d)
• Method 3: Obtain new CER (cont’d)
– Concerns (cont’d)
– What is the expectation for behavior?
$30,000
Pullout
Amount for
Method 2
Cost of Car ($)
$25,000
$20,000
$15,000
Car CER
$10,000
Possible “Transmission-less
Car” CER
$5,000
$0
1,000
2,000
3,000
4,000
5,000
6,000
Car Weight (lbs)
Case 4: The new CER yields an estimate higher than the existing
CER – the method does not produce a parametric pullout
[email protected], 703.961.3432
20
Parametric Pullout
• Second order effects
– In some models, particularly in operating and support
cost, some elements are estimated using a parametric
relationship to another cost element
– If this is the case, it is important to keep careful track of
order of operations, etc. … to be certain that the
appropriate results are captured
Warning: It is easy to get caught in the trap of thinking
that the element which is being pulled out and put back in
is “causing” changes to other elements in the model. This
is not the case - second order changes to elements are
attributable to a total change in the cost element value,
not to the system being incorporated into the model.
[email protected], 703.961.3432
21
Parametric Pullout
• Complexities beyond the toy problem
– The toy problem demonstrates an example of a CER
that is linear and uses just one variable
– There are many complexities of the problem that
occur in non-linear and/or multivariable CERs
[email protected], 703.961.3432
22
Lessons Learned
• Buildups are useful because of the insight they may
provide. The following has occurred during the
authors’ experience with this topic:
– A significant improvement in understanding of the
nuances of a main source of historical data resulted
– Led to an examination of several existing CERs and
led directly to improvements in at least one
– Allowed system designers to focus future cost and
CAIV resources on specific areas of interest
– The details of the process generated discussions with
the system designers that provided further insight into
design and cost issues of all types
[email protected], 703.961.3432
23