Earned Value Management at Saab Naval
Systems Division Sweden
-Planning
Package conversion and Performance
Measurement Techniques
Rasmus Vestberg
Supervisors:
Dag Swartling, Linköping Institute of Technology
Michael Trochez, Saab AB
Master Thesis LiTH-IEI-EX—07/??—SE
Department of Management and Engineering at
Linköping Institute of Technology and Saab Naval
Systems Division Sweden
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Vestberg, Rasmus
Abstract
Abstract
In traditional Project Management one determines the budget and measures the cost against
that budget as project goes on. In Earned Value Management, which has been adopted by
Saab’s Naval Systems Division less than two years ago, one also measures the actual
performance by assigning finished work a value.
As a consequence of the recent adoption of the Earned Value Management the Planning
Department asked for a study of following two questions; How far can one do detailed
planning and how should the value be measured?
The report is based on a studious literature study combined and analyzed together with
historical data from existing projects at Naval Systems Division.
The result indicates that the Naval Systems Division is not using Earned Value Management
at its full potential. A lot of money is spent to get the management system working, but many
things are not implemented in the best way. The figures provided to Naval Systems Division
is not showing the true Earned Value as could be if the system was used in a different way.
One can argue that the Naval Systems Division is still using traditional project management
only dressed in Earned Value figures. This is in large extent a consequence of not measuring
the value in the right way. By using other measurement methods the system will measure the
true performance instead of doing subjective judgment as per today.
As a consequence of this flawed use of Earned Value Management many of the people
working in the project does not accept the system. Also a lot of work is spent clearing up the
Earned Value figures to the customer, instead of using them as a helpful tool in the project
management.
By adapting the entire Earned Value concept several benefits will follow. Today the
forecasting of a projects final outcome is made in a time consuming traditional project
management way instead of using the mathematical approach provided by Earned Value
Management. The Earned Value Management forecasting method is very easy to use and will
save a lot of work and money for Naval Systems Division. This study shows that in this case,
the easier method even provides better accuracy.
2
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
The research also implies that detailed planning could be done very far in advance. No figures
provided by the investigation indicates that detailed planning of nearby work packages are
more accurate than work packages planned far away.
3
Vestberg, Rasmus
Table of Contents
Table of Contents
1
2
INTRODUCTION ....................................................................................................................................... 8
1.1
ABOUT SAAB AB.................................................................................................................................. 8
1.2
BACKGROUND ...................................................................................................................................... 8
1.3
PURPOSE ............................................................................................................................................... 9
1.4
DISPOSITION ......................................................................................................................................... 9
METHOD................................................................................................................................................... 12
2.1
THEORY .............................................................................................................................................. 12
2.1.1
Four Dimensional Studies............................................................................................................. 12
2.1.2
Quality of the Research ................................................................................................................. 13
2.1.3
Interviews ...................................................................................................................................... 13
2.2
USED METHODS .................................................................................................................................. 14
2.2.1
Frame of Reference ....................................................................................................................... 14
2.2.2
Empirical Study............................................................................................................................. 15
2.2.2.1
Interviews ............................................................................................................................................ 15
2.2.2.2
Analyzing historical data..................................................................................................................... 15
2.3
3
METHOD CRITICISM ............................................................................................................................ 19
2.3.1.1
Literature ............................................................................................................................................. 19
2.3.1.2
Interviews ............................................................................................................................................ 19
2.3.1.3
Analyzing historical data..................................................................................................................... 20
FRAME OF REFERENCE ...................................................................................................................... 22
3.1
TRADITIONAL PROJECT MANAGEMENT ............................................................................................... 22
3.2
EARNED VALUE MANAGEMENT ......................................................................................................... 23
3.2.1
History of EVM ............................................................................................................................. 23
3.2.2
This is EVM ................................................................................................................................... 24
3.2.2.1
SPI and CPI ......................................................................................................................................... 27
3.2.2.2
Baseline - Scope the Project ................................................................................................................ 29
3.2.2.3
Scheduling........................................................................................................................................... 31
3.2.2.4
Performance Measurement Techniques............................................................................................... 33
3.2.2.4.1
Weighted Milestones...................................................................................................................... 33
3.2.2.4.2
Fixed Formula ................................................................................................................................ 34
3.2.2.4.3
Percent Complete Estimates........................................................................................................... 35
3.2.2.4.4
Percent Complete and Milestones .................................................................................................. 35
3.2.2.4.5
Equivalent Completed Units .......................................................................................................... 36
3.2.2.4.6
Apportioned Relationship .............................................................................................................. 36
3.2.2.4.7
Level of Effort................................................................................................................................ 37
3.2.2.5
Forecasting .......................................................................................................................................... 38
3.2.2.5.1
Mathematical EAC – Best Case Scenario ...................................................................................... 38
3.2.2.5.2
Cumulative Cost Performance Index – Most Likely ...................................................................... 39
4
Earned Value Management at
Vestberg, Rasmus
4
3.2.2.5.3
Cumulative CPI times SPI – Worst Case Scenario ........................................................................ 39
3.2.2.5.4
Predicting completion date............................................................................................................. 40
3.3
EARNED SCHEDULING ........................................................................................................................ 40
3.4
GOLD PRACTICE ................................................................................................................................. 44
EMPIRICAL STUDY ............................................................................................................................... 46
4.1
EVM AT NAVAL SYSTEMS DIVISION .................................................................................................. 46
4.1.1
Project Organization..................................................................................................................... 46
4.1.2
Earned Value................................................................................................................................. 48
4.1.2.1
Planning Packages.......................................................................................................................... 48
4.1.2.1.2
Work Packages............................................................................................................................... 49
Performance Measurement Techniques............................................................................................... 49
4.1.2.2.1
Percent Complete Estimates........................................................................................................... 50
4.1.2.2.2
Level of Effort................................................................................................................................ 50
4.1.2.3
Scheduling........................................................................................................................................... 51
4.1.2.4
Forecasting and Predicting Completion Date ...................................................................................... 51
4.2
FORECASTING ACCURACY AT NSDS .................................................................................................. 52
4.2.1
Accurateness Relative Entire Work Package ................................................................................ 52
4.2.2
Accurateness Relative Remaining Work........................................................................................ 55
4.3
NSDS FORECASTING VS. CUMULATIVE CPI FORECASTING ................................................................ 57
4.3.1
Relative Entire Work Package ...................................................................................................... 58
4.3.2
Relative Remaining Work.............................................................................................................. 62
ANALYSIS................................................................................................................................................. 68
5.1
PLANNING PACKAGE CONVERSION ...................................................................................................... 68
5.1.1
Relative Entire Work ..................................................................................................................... 68
5.1.2
Relative Remaining Work.............................................................................................................. 69
5.2
6
Scoping the Project at Naval Systems Division................................................................................... 48
4.1.2.1.1
4.1.2.2
5
Saab Naval Systems Division Sweden
PMT ................................................................................................................................................... 70
5.2.1
Analyze based on Frame of Reference .......................................................................................... 71
5.2.2
Analyze based on Empirical Study ................................................................................................ 71
DISCUSSION............................................................................................................................................. 74
6.1
PMT ................................................................................................................................................... 74
6.1.1
Automatic Earned Value ............................................................................................................... 74
6.1.2
NSDS subjective Measurement Technique .................................................................................... 75
6.1.3
Level of Effort................................................................................................................................ 76
6.2
PLANNING PACKAGE CONVERSION ..................................................................................................... 77
6.2.1
Forecasting ................................................................................................................................... 77
6.2.2
Rolling planning’s affect on EVM ................................................................................................. 79
6.3
6.3.1
MISCELLANEOUS ................................................................................................................................ 81
Earned Scheduling ........................................................................................................................ 81
5
Vestberg, Rasmus
6.3.2
7
Table of Contents
Earned Value Acceptance ............................................................................................................. 82
CONCLUSIONS........................................................................................................................................ 84
7.1
PLANNING PACKAGE CONVERSION ..................................................................................................... 84
7.2
PMT ................................................................................................................................................... 84
7.3
FORECASTING ..................................................................................................................................... 85
7.4
MISCELLANEOUS ................................................................................................................................ 85
8
REFERENCES .......................................................................................................................................... 88
9
APPENDICES ........................................................................................................................................... 92
9.1
APPENDIX 1 ........................................................................................................................................ 92
6
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
7
Vestberg, Rasmus
Chapter 1 - Introduction
1 Introduction
This chapter will treat the background and subject of this master thesis.
1.1 About Saab AB
SAAB (Svenska Aeroplane Aktiebolag) was originally founded in the spring 1937. During
the first years SAAB only produced airplanes for the Swedish military defense. Today the
SAAB brand is closely related to the cars which they started to manufacture in the 1940’s.
However the car division was sold to General Motors a couple of years ago who are using the
brand through a license. This means that Saab AB has nothing to do with the SAAB cars
today. In the fifties and sixties SAAB increased the direction to also include computers,
missiles, space technology etcetera. 1
Today SAAB has changed its name to Saab AB and is in popular speech called Saab
Technologies. The direction of the company has been further increased and Saab AB is today
split into three different segments; Defense and Security Solutions, Systems and Products and
Aeronautics. These different segments are further split into several business units. One of the
business units included in Defense and Security is Saab Systems, with over 1300 employees,
which is further split into five divisions. One of those divisions is the Naval Systems Division
Sweden (NSDS). 2
The Naval Systems Division is the division for this thesis.
1.2 Background
The Earned Value Management (EVM) is quite recently adopted at Naval Systems Division.
The model is in use but without wide acceptance and cooperation from the staff. The level of
detail planning when setting up a project is subject for constant discussion. Guidance
regarding planning package conversion and overall Earned Value Management is needed so
projects can have a shorter start-up time without loosing quality.
1
http://saabnet.saabgroup.com
2
http://saabnet.saabgroup.com
8
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
1.3 Purpose
The purpose of this master thesis is to study the use of Performance Measurement Techniques
(PMT) and Planning Packages in Earned Value Management at Saab Systems Naval Systems
Division. Different PMT will be compared to each other to tell which ones are best suited for
Saab Systems Naval Systems Division. Studying the accuracy of the project planning and
measuring aims to provide guidance for conversion of Planning Packages. The thesis aims to
answer the following two main questions:
•
In which time frame is the knowledge regarding future work high enough to
breakdown planning packages into work packages?
•
How should NSDS measure the work to attain most valid Earned Value metrics?
1.4 Disposition
In chapter 2 the different methods used during this thesis is discussed in detail. The pros and
cons with used methods are discussed to provide the reader with information regarding
subject of errors in the work.
Chapter 3 treats the Frame of Reference subjective to the purpose of the thesis. Common
knowledge and facts found in literature is described in detail to give the reader a basic
background of Earned Value Management.
Chapter 4 will present the figures received from analyzing the historical data. The presented
figures will be described and analyzed in chapter 5. The connection between the analyzed
figures and the frame of reference is made first in chapter 6, where furthermore my own
reflections and ideas are presented. Also a description of how work is conducted today at
Naval Systems Division is displayed in chapter 4.
In chapter 7 the conclusions based on chapter 3 to chapter 6 are presented in relative compact
form aiming to answer the main questions in the purpose.
9
Vestberg, Rasmus
Chapter 1 - Introduction
For people with wide knowledge about NSDS and EVM it is possible to skip chapter 3. Focus
in the reading should then be at chapter 2.2.2 and chapter 4 to chapter 7. However I
recommend reading the entire report for best understanding of conclusions made.
10
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
11
Vestberg, Rasmus
Chapter 2 - Method
2 Method
This chapter will describe the different methods used for conducting this study. The chapter is
divided into three parts. The first part will cover the theory behind the used methods followed
up by an illustration of how they have been used. The last part will discuss the possible
disadvantages with the chosen methods.
2.1 Theory
This part will present a few theories behind some of the used methods.
2.1.1 Four Dimensional Studies
A study can consist of many different methods. Common for all different methods is that they
have to be either one or the other of all the following four dimensions. 3
Qualitative or Quantitative Research
The research approach can be either qualitative or quantitative. The qualitative approach is
focused on one or a few objects studied in detail. The quantitative approach concentrates on
several objects which could be analyzed with mathematical methods.
Primary or Secondary Information
Information could be divided into two categories, primary and secondary information. The
primary information is collected by one self while the secondary information is collected by
someone else. The important difference is who makes the collection from the originally
source. Observing something or interviewing the source is example of primary information
while reading about something is example of secondary information.
Case or Overall Study
A case study focuses on one case only. The case could have very varying extension but is still
limited to that specific case. An overall study aims to provide a picture of the common
approach for all similar cases.
3
Lekvall and Wahlbin (2001)
12
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Time Series or Fixed in Time Analyze
A fixed in time analyze is based on the circumstances at one specific moment. The moment
could be either today or some earlier point. A time series analyze extent over several time
periods. The analyzing could be made currently or afterwards based on historical
measurements. The important concern is to se, measure and analyze the changes over the
chosen period.
2.1.2 Quality of the Research
The quality of the research can be split into the reliability and the validity of the research.
Reliability
Reliability is the probability of obtaining the same results if the research was conducted in the
same way once again. High reliability means that the result would be pretty much the same. In
qualitative surveys it might be difficult to obtain the same results again since for example the
interviewed persons might answer different from time to time. Quantitative research is more
likely to be of high reliability because of the mathematical methods used.4
Validity
The validity tells how well the research reflects the reality. A high level of validity indicates
that the research reflects the reality in a good way. Inversely from reliability the validity is
usually higher in qualitative research because of simplifications made in the quantitative
research.5
2.1.3 Interviews
Interviews can be made in several ways; for example personal interviews, telephone
interviews, written questionnaire etcetera.
In a personal interview there are some things to think about; the degree of standardization is
one of them. A completely standardized interview consists of identical questions posed in the
4
Lekvall and Wahlbin (2001)
5
Lekvall and Wahlbin (2001)
13
Vestberg, Rasmus
Chapter 2 - Method
same order for all interviewed. The level of structuring is another thing to determine.
Structuring means to what extent the questions have predefined replying options.6
It is also important to gain the trust from the interviewed person if the answers should be
really truthful. Therefore the most sensitive questions should come in the end of the interview,
when as much trust as possible has been gained.7
2.2 Used Methods
This chapter will describe the used methods during this research.
2.2.1 Frame of Reference
To gain basic knowledge about Earned Value Management existing literature has been
studied. The basics behind Earned Value Management have been gathered from literature
published by the Project Management Institute, PMI. Their ideas about Earned Value
Management are directly connected to the American National Standard Institute/Electronic
Industry Association (ANSI/EIA) standard concerning EVM. Also the ANSI/EIA standard
has been used to further support the understanding of EVM basics.
Several articles have been used to extend the picture provided by PMI. Different alternative
theories about and complementing EVM has been studied in the search for the best suited
method for the Naval Systems Division.
The internal document “Projekthandbok” has been studied to get a basic knowledge about the
routines at the Naval Systems Division. This document describes when, why and how a
project should be managed at Naval Systems Division.
It could have been possible to attain knowledge regarding Earned Value Management from
people working at Naval Systems Division. However, this has been ignored to prevent any
influence of the understanding and conclusions.
6
Patel and Davidsson (2003)
7
Cassel and Symon (2004)
14
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
2.2.2 Empirical Study
When the Naval Systems Division introduced Earned Value Management, in the beginning of
2006, they also started saving EVM reports. One report for each project each month is
displaying the EVM figures at WBS-level. The reported figures are; Planned Value, Earned
Value, Actual Cost, Schedule Variance and Cost Variance for both current period and
cumulative to date. For the completion; the Budget, the Latest Revised Estimate and the
Variance is displayed. Those reports have served as the platform for this study. To fully
understand NSDS’s project organization several interviews have been made.
2.2.2.1 Interviews
To make sure the understanding, provided from the “Projekthandbok”, of the project work at
Naval Systems Division is the truly used and not just the hoped-for, personal interviews have
been conducted. This has provided the research with primary information. The above
mentioned “Projekthandbok” has not been updated since the introduction of Earned Value
Management, why this supplementation was absolutely necessary. The interviews have been
made with people at different positions to get the entire scope.
All the interviews have been made with a very low degree of standardization and structuring.
Basically the interviews consisted of a conversation where the interviewed was asked to
describe their work, their contact with the project organization and their contact with Earned
Value Management.
If the interviews would have been made with high standardization and structuring I do not
think I would have got the entire picture. The results would have been relying on my
capability to ask the right questions. Since many of the interviews were held early in the
process I did most likely not have had the understanding needed to ask the right questions, at
that time.
2.2.2.2 Analyzing historical data
To be able to tell how far in advance knowledge about the work is vast enough, to tell when
we can convert planning packages into work packages, we can look at the forecast. The
15
Vestberg, Rasmus
Chapter 2 - Method
forecast is based on the knowledge about the work to be done, thus the same thing as when to
convert planning packages. See 3.2.2.5.
When the forecast is stabile within 10% of the Actual at Completion (the final outcome) the
Naval Systems Division would feel satisfied with the knowledge of the work to be done.8
This means that if we can, based on the EVM reports, show any trend indicating when the
forecasting stabilizes within 10% of its final outcome, this would give a scientific/statistical
point for planning package conversion.
8
Michael Trochez
16
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Figure 1 is an example of the method. Each line represents the forecasting relative the Actual
at Completion for different work packages. The broken line highlights the 10% limit. As seen
in the figure, the forecasting stabilizes below 10% somewhere between 5-7 months before its
completion date (all work packages has their completion date at the end of the graph). This
would mean that we could plan at work package level 6 months ahead (the median in the
figure).
40%
20%
10%
jun-0
7
maj07
7
apr-0
mar-0
7
7
feb-0
jan-0
7
d e c06
6
6
okt-0
nov0
6
sep-0
06
aug-
6
jul-0
jun-0
6
maj06
6
0%
apr-0
Forecast relative Actual at Completion
30%
-10%
-20%
-30%
Date
Figure 1 Example of hoped-for data.
The work packages to be studied had to be finished. This is the only way to know the precise
outcome of the work package. To determine which work packages that really where finished I
have studied the project Cost Performance Reports. Finished work packages have an Earned
Value equaling the Planned Value, regardless of the performance of that specific work
package.
When the work packages to be measured were selected the graphs were built through the
following formula:
17
Vestberg, Rasmus
Forecasting Accuracy =
Chapter 2 - Method
Estimated Cost at Completiontime of measurement − Actual Cost at completion
Actual Cost at completion
Formula 1
It is obvious that as the work package are approaching completions there will be less work left
to forecast. Therefore the estimate will always approach the actual when the work package
reaches completion.
If we use Formula 1 with following figures; Actual Cost Cumulative = 900, Actual Cost at
Completion = 1000 and Estimated Cost at Completion = 1100. Then the Forecasting
Accuracy will be 10% from perfect. Since there is only 100 hours left of work but estimated
200 hours, it is obvious that the estimate is pretty bad although the figure looks okay. That
also makes it interesting to look at the ability to forecast the work packages relative the
amount of remaining work.
Forecasting Accuracy relative remaining work =
=
Actual Cost at completion − Estimated Cost at Completiontime of measurement
Actual Cost time of measurement − Actual Cost at completion
Formula 2
Using Formula 2 the simple example above will be 100% wrong instead of 10%.
This second method will be difficult to use when the baseline has been changed, why the
periods in the study with this problem have been left out. Also the case when the final cost
has been achieved is left out, since the formula will give division with zero and the work
package is finished anyway.
It could have been possible to determine the time frame of planning package conversion by
interviewing people responsible for the plans. This would however only represent the wished
for time frame without any scientific support.
18
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
2.3 Method Criticism
This section will highlight some things to have in mind when looking at the research. Some of
them might have affected the results of the study.
2.3.1.1 Literature
There is always a good thing to establish the facts from several independent sources. This
could perhaps have been done more thoroughly by using even more different sources.
However, it is not likely that result from the literature study would have been different even if
more sources would have been used. The sources used are the fundamental source for the
subject in mind. And the publisher is in some way one of the owner of the method available in
Earned Value Management. The literature study about EVM could be said to be of both high
reliability and validity.
The internal “Projekthandbok” is not updated for a couple of years which could affect the
understanding of the situation at the Naval Systems Division. The main concept of this study
is anyway focused on EVM which is not included in the “Projekthandbok”. Therefore the
interviews have been the main source of this information and confusion based on non updated
documents would be no issue. Based on this the validity could be questioned but should
anyway not affect the study in general.
2.3.1.2 Interviews
It is always a risk of missing vital information while conducting an interview alone. One
might for example miss important body language while taking notes. Since the interviews
were made with very low level of standardization and structuring the reliability is low and
there is a great risk that one might get different answers doing a new interview. However the
validity is high since the interviews were made to build the entire picture of the situation at
Naval Systems Division. Several people have contributed to the understanding of NSDS’s
project management why the gathered picture would be very valid.
19
Vestberg, Rasmus
Chapter 2 - Method
2.3.1.3 Analyzing historical data
The analyzed data origins from one type of report being available since the beginning of
2006. Only a couple of the projects at Naval Systems Division have been reported with this
method. Among those projects there is only one that have reached so far that some work
packages have been completed. This is an obvious problem. The analyzed data is of far too
small quantity to be statistical.
Old finished project have been studied to see if they could contribute to the gathering of data
but without success. Very little information is stored regarding old projects. When there is
information stored about old projects it is not the wanted, or the information is at a too high
level telling us nothing about the work packages. Although the findings can not be called
statistical they should work as a warning signal indicating a very obvious trend.
20
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
21
Vestberg, Rasmus
Chapter 3 – Frame of Reference
3 Frame of Reference
This chapter will treat the Frame of Reference subjective to the purpose of the thesis
3.1 Traditional project management
In traditional project management the most important measurements are today’s date and the
actual amount of money spent on the project. The amount of money spent is compared to the
budgeted cost for the reached date to measure the performance of the project. The actual
performance is never measured.9 Thinking about it, the data displayed in Figure 2 does not
say anything about the projects performance. To determine the true performance of the project
one would have to look at the schedule result in the same format as the cost. Often the
financial people breaks out the cost by function while scheduling people develop their
displays from tasks in the Work Breakdown Structure (WBS). Although many companies use
this approach these two does not match.10
1600
Budget
1400
1200
Cost
1000$
1000
800
600
400
200
0
jan
feb
mar
apr
maj
jun
jul
Date
Figure 2 Traditional Project Management Performance
9
ANSI/EIA-748-A-1998
10
Fleming and Koppelman (2005)
22
aug
sep
okt
nov
dec
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
3.2 Earned Value Management
Earned Value Management is a method used for handling of medium and large scaled
projects. The details about EVM and the differences between EVM and traditional project
management systems will be displayed in this section.
3.2.1 History of EVM
The Earned Value concept is not in anyway new, in fact the basics and the ideas have been
used for more than a century. In the late nineteenth century the industrial workers in the
American factories started using the concept of earned value. The workers measured their
earned work and actual expenses which they compared to the planned standards. These
measurements were their way to control the work in the factories. Since they also had defined
the cost variance, consisting of the actual money spent relative to the work achieved, they
used the core of earned value although they did not address it that way. One could say that
this basic definition of cost variance is what characterizes earned value.11
In the end of the 1950’s the United States Navy introduced the Program Evaluation Review
Technique (PERT) as a network scheduling and risk management device. In 1962 PERT was
extended to PERT/costs aiming to add resources to the time scheduler. Since the computers
back then was not powerful enough PERT did not live on for many years. However PERT left
something to the afterworld. The implementation of PERT/costs had required some new
reporting formats from the contractors. One of those formats contained “value of work
performed” versus “actual cost”.
By the second half of the 1960’s the United States Air Force made a new effort to oversee the
industry performance. Instead of imposing a specific management control system they wanted
the industry to satisfy some criteria with their existing management control systems. The
result was an earned value concept called Cost/Schedule Control Systems Criteria (C/SCSC)
which comprise 35 criteria that one had to adapt while being a contractor for the Department
of Defense (DoD).
11
Fleming and Koppelman (2005)
23
Vestberg, Rasmus
Chapter 3 – Frame of Reference
During the next three decades the C/SCSC evolved to be more and more difficult to use. The
basic criteria were added with implementation guidelines, surveillance manuals and
implementation checklist which were intended to be used as guidelines used with good
judgment. However the guidelines were not always used as guidelines but instead as
peremptory rules. Furthermore the private sector never adapted the C/SCSC. One of the main
reasons why the industry never adopted the C/SCSC, except the rigid guidelines, was the new
vocabulary. Instead of calling things what they are C/SCSC used their own expressions and
abbreviations. For example earned value was called “Budgeted Cost of Work Performed”,
BCWP, or simply “p”. Overrun was called OTB which stood for “Over Target Baseline”.
Some of those expressions are still used today by some people, although there are simpler
expressions meaning the same thing.
In 1995 the National Defense Industrial Association, NDIA, accepted the task of rewriting
DoD’s formal earned value criteria. The objective of the rewriting was to make the criteria
more practicable for the private industry. The 35 criteria became 32 and terms as Budgeted
Cost Work Performed (BCWP) and Budgeted Cost Work Scheduled (BCWS) where changed
to Earned Value and Planned Value. But the main change was in the attitude of all parties.
The private company was now adopting the technique since it had proven to be best-practice
instead of something that where forced from the government to their contractors. The
restriction of EVM to DoD was removed and in 1998 the new technique became accepted as
an ANSI/EIA document. 12
3.2.2 This is EVM
The main concept of Earned Value Management is to add the amount of work actually
performed when measuring the project. The work actual performed provide the managers with
more accurate information on the status of the project which leads to better forecasting and
corrective actions.13 If a project e.g. is overspending, the traditional project management
measures will indicate that the project is performing badly. If the actual cost is higher than
budget the project might be overspending but there is now way to tell unless you measure
12
Fleming and Koppelman (2005)
13
Christensen David S (1998)
24
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
what has actually been performed. If the actual work is in the same magnitude as the actual
cost, the project is ahead of schedule and not really overspending. In the same way a project
that is spending less money than budget might not be as successful as it seems. If the actual
work performed also is less than planned the project is in fact behind schedule and not only
cost effective as traditional project measurement would say.
Let us look at an easy example to make things even clearer. We have a project extent over ten
months (see Figure 3) with a total budget of $1,000,000. Each month has a budget of
$100,000 and distinct milestones for each budgeted $100,000. Once a milestone is
accomplished the project will earn $100,000. The budget is labeled the “Planned Value”
which consists of two elements, the work scheduled and the budget for that work.
1200
Planned Value
1000
1000$
800
600
400
Actual Cost
Earned Value
200
0
jan
feb
mar
apr
maj
jun
jul
aug
sep
okt
nov
Date
Figure 3 Earned Value Example
After four months the project has spent the planned $400,000 and would by traditional project
management be performing well. The Actual Cost is therefore $400,000. When also looking
at the work performed, as in EVM, you notice that only three of the four milestones have been
achieved and the Earned Value is only $300,000. One could immediately see that the project
25
Vestberg, Rasmus
Chapter 3 – Frame of Reference
is running behind and have a negative Schedule Variance at $100,000 (Schedule Variance =
Earned Value – Planned Value, see Figure 4).
Also shown in Figure 3 is the Actual Cost, $400,000. The Actual Cost is greater than the
Earned Value which tells us that the project is not cost effective. The Cost Variance is defined
as the Earned Value – Actual Cost (see Figure 4).
Earned Value is recommended to help managing company’s different projects and act as a
comparison between a company’s different projects. (Also other company’s projects using
Earned Value can be compared)14 Although some of the basics and theories behind Earned
Value can be used for continues business it is not recommended to use the entire Earned
Value Management method for that purpose.15
Voices have been raised against the Earned Value method, saying it is one of the larger cost
drivers in a project.16 So why should one use Earned Value? There are several benefits with
employing Earned Value and among others it can be said that; it provides reliable data. It
provides early warning signals. It makes reliable predictions possible etcetera.17 Also there is
a general phobia about Earned Value complexity which undermines the acceptance.18
The core basics of Earned Value is the measurement of actual work performed compared to a
baseline plan.19 Perhaps the most advantageous thing about Earned Value is the, through
measurement of performance, provided possibility of reliable prediction of the project. Some
people would say that this measurement requires too much effort compared to the
advantageous with the system.20 Those people are most likely just prejudice based on old
times complexity in Earned Value. Those doubters tell us that there are reasons to keep
14
Averstad, Dan (2003)
15
Fleming and Koppelman (2005)
16
Fleming and Koppelman (2005)
17
Christensen David S (1998)
18
Kondur, Mohan (2007)
19
Christensen David S (1999)
20
Fleming and Koppelman (2005)
26
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Earned Value as simple as possible.21 To keep Earned Value simple should not be a problem.
The concept of Earned Value is often used as a daily routine. For example, when shopping
grocery many people look at the price per kilo. Concerns about what you get for the money
could in some extent be equal to Earned Value. Shopping based only at the total cost is more
similar to the traditional project management.
The concept of Earned Value requires a performance plan, the Planned Value, which
combined with the physical Earned Value and the Actual Cost provides the cost performance.
3.2.2.1 SPI and CPI
The three-dimensional way of looking at the project gives us further possibilities to evaluate
the project. We have earlier spoken about the Cost Variance and the Schedule Variance (see
Figure 4). Those two measurements could be used in a slightly different way. We call them
Schedule Performance Index, SPI, and Cost Performance Index, CPI. The Schedule
Performance Index is calculated as the Earned Value divided with the Planned Value. The
Cost Performance Index is calculated as the Earned Value divided with the Actual Cost. If
looking at a similar example as the one above (PV and EV is the same but AC is $500,000
instead of $400,000) that gives us an SPI of 0.75 (EV $300,000 / PV $400,000 = SPI 0.75)
and a CPI of 0.6 (EV $300,000 / AC $500,000 = CPI 0.6).
21
Kondur, Mohan (2007)
27
Vestberg, Rasmus
Chapter 3 – Frame of Reference
1200
Planned Value
1000
1000$
800
600
Actual Cost
Cost Variance
400
Schedule Variance
Earned Value
200
0
jan
feb
mar
apr
maj
jun
jul
aug
sep
okt
nov
Date
Figure 4 Earned Value Basics
The SPI and CPI can then be used to predict the rest of the project. With an SPI of only 0.75
you could say that for every dollar of physical work this project had planned to achieve only
$0.75 was accomplished. With an original budget of $1,000,000 this project will take
$1,333,333 to finish. That is more than three months of extra work. Note that the schedule is
measured in money but can be directly transformed to man hours or time in Gant-chart. This
is at first sight at bit confusing but will after a while feel perfectly naturally.
The CPI in our example is 0.6 which in the same way as the SPI can tell us something about
the future of the project. For every dollar spent we have only got a value of $0.6 which gives
us the final cost of $1,666.666 for the project.
CV = AC − EV ,
SV = PV − EV ,
CPI =
EV
,
AC
SPI =
EV
PV
Formula 3 a-d
With these figures you can make the correct decisions about the project. It is not likely that
the estimated total cost of the project is possible to improve, unless you change the scope.
However the duration of the project can perhaps be improved by adding extra funds. Research
28
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
from hundreds of projects within the DOD, shows that the performance of a project at 20% of
completion is very accurate for the entire project. Very seldom a projects performance
improves after the 20% completion. If the performance changes it often gets worse and not
better.22 Therefore management no longer must wait until the total funds have been spent to
determine that additional budget is needed.
3.2.2.2 Baseline - Scope the Project
The baseline or the Planned Value is the guiding value for the project. Therefore it is very
important to carefully develop the baseline. Actually you can not use the Earned Value
technique if there is not any Planned Value. Knowing what is included and, perhaps more
important, what is not included is essential for managing of the project.23 The generated
Performance Measurement Baseline (PMB) is the total time-phased budget for the project. It
is important to update this baseline when the scope of the project is changed. Otherwise the
measurement will take place against an invalid reference.24
A useful tool for scooping the project is the Work Breakdown Structure (WBS). A WBS can
be developed in different kind of ways and take different shapes; the important thing is that it
helps grasping the scope.25 The type of WBS discussed further will be the one described in
PMBOK.26
A WBS looks like an organizational chart but it is important to realize that it is not. However
the WBS is for the project leader what the organizational chart is for the company executive.27
The WBS represents the project work to be done and the deliverables of the project. The
WBS is separated into different products or work in several levels. Either one chose to do a
22
Fleming and Koppelman (2005)
23
A Guide to the Project Management Body of Knowledge (2005)
24
ANSI/EIA-748-A-1998
25
Wenell, Torbjörn (2004)
26
A Guide to the Project Management Body of Knowledge (2005)
27
Fleming and Koppelman (2005)
29
Vestberg, Rasmus
Chapter 3 – Frame of Reference
product or a work oriented WBS (se Figure 5 and Figure 6).28 A success factor is to let the
WBS reflect the way the manager actually plans to manage the project.29
Figure 5 Work oriented WBS
Figure 6 Product oriented WBS
At the top levels the product or work is described very briefly, called planning packages. It is
not uncommon that the owner of the project specifies the three top levels of the WBS. The
lower levels are extended by the project leader to contain work, costs and schedule, the work
packages. It is first at work package level that one can accurately estimate cost and schedule
and it is also at this level the Earned Value is computed.30 In very large projects it can be hard
28
Wenell, Torbjörn (2004)
29
Fleming and Koppelman (2005)
30
ANSI/EIA-748-A-1998
30
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
to plan everything on work package level from the beginning. The manager then has to do a
rolling planning bye and bye.31 It is also common that the WBS is reassessed from time to
time when the knowledge gets wider or when requirements changes.32 Each company must
determine its own standard for the conversion of planning package into work packages.33
Different deliverables need different levels of breakdown to clearly define the work. More
detailed breakdown gives better opportunities to control and manage the work. In the same
time too detailed breakdown will inhibit the productivity and lead to ineffective work. It is of
most importance for the project manager to keep the breakdown at a balanced level. 34
For the breakdown of the WBS-element it takes that different subprojects and deliverables is
split into its basic elements. Each element must be clearly defined and assigned to a specific
unit taking the responsibility for the task. When work has been defined it is possible to assign
the tasks to personnel through an Organization Breakdown Structure, OBS. The lowest level
of the WBS is expanded with more details into a WBS-dictionary. This dictionary is the
Statement of Work (SOW) for the project.
Of course there are other techniques used than the WBS to scope the work. The important
thing is naturally not which technique one chooses to use, but the result when grasping the
project. Since WBS is the superior most used technique and also the one used at Naval
System Division there is no reason to study other techniques further.
3.2.2.3 Scheduling
Independent of which management techniques are being used all projects has to schedule the
work, so also the ones applying Earned Value. Perhaps the ones using Earned Value
especially needs it.35 Larger projects may have multiple schedules correlated to each other
31
A Guide to the Project Management Body of Knowledge (2005)
32
ANSI/EIA-748-A-1998
33
ANSI/EIA-748-A-1998
34
A Guide to the Project Management Body of Knowledge (2005)
35
Kondur Mohan (2007)
31
Vestberg, Rasmus
Chapter 3 – Frame of Reference
with a formal scheduling system. But also the smallest project needs at least a project master
schedule, PMS, containing the most important tasks to be performed.36
To implement Earned Value Management there are two basic needs. First, as earlier
discussed, one must define the project and all the work which has to be done. Second, the
work has to be put into a time frame so that performance can be measured against it.37 The
time based schedule together with the WBS is evidently the spinal column of the projects
baseline.
The Earned Value Project Management implies that there is no other way to establish a time
schedule except the Critical Path Method (CPM).38 The Critical Path Method is a technique
used for analyzing web plans. There are some different kinds of web planning methods
although they are all very similar. The concept of them all is to display the tasks in the project
in a web, showing the connections between the different tasks.39 The CPM and the
dependencies between the tasks are then used to show the shortest possible duration of the
project.
The difference between cost variance and schedule variance takes some extra attention except
what has been told earlier. A negative cost variance is always a serious matter telling us that
the project is not performing as planned. A negative schedule variance on the other hand does
not have to be a big issue. The schedule variance only tells us how much of the planned work
we have accomplished. That could of course be a big problem if we for example have a tight
schedule and an important deadline to keep. But it could also be the result of a late start, either
on some of the work packages or on the entire project. If the negative schedule variance is a
consequence of some single work packages starting late, and those not being on the critical
path, the variance in fact could be insignificant.40
36
Fleming and Koppelman (2005)
37
Fleming and Koppelman (2005)
38
Fleming and Koppelman (2005)
39
Project Management Body of Knowledge (2005)
40
Fleming and Koppelman (2005)
32
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Most of the time schedule variance can also be compensated. In the best case it is enough to
reorganize the schedule to get back on track. Otherwise one can ad extra money to speed up
the work. A negative cost variance is seldom to compensate for. The basic reason for this
difference in cost and schedule variance is that the scope is almost never dependent on what
date the work is being done.
3.2.2.4 Performance Measurement Techniques
The Earned Value is based on measuring the performance of the project. To measure the
performance a couple of different techniques have been developed. Below we will take a
careful look at the pros and cons with following techniques.
•
Weighted Milestones
•
Fixed Formula
•
Percent Complete Estimates
•
Percent Complete and Milestones
•
Equivalent Completed Units
•
Apportioned Relationship
•
Level of Effort
Different techniques might suite different projects. The choosing of technique, to make it fit
the projects various work packages, is up to the project leader. One does not have to use the
same technique over the entire project. The important thing is to decide at planning stage
which technique to use and be consistent to that.41
3.2.2.4.1 Weighted Milestones
Weighted milestones are as it sounds a method where different milestones in the project are
assigned a value. When the milestone has been achieved the project earns the value of that
milestone. The milestones can be set to be the entire work packages if they are small,
otherwise one would like to have several milestones in each work package. If the work
41
Fleming and Koppelman (2005)
33
Vestberg, Rasmus
Chapter 3 – Frame of Reference
packages are divided into several different milestones one must distribute the planned value
of that work package among the milestones. This could be done in a simple method where
you divide the planned value equal over all milestones. Since a milestone is unique and
measurable, for example a specific task being finished, the equal division might not fit. It is
therefore ok to divide the planned value uneven between the milestones. The important thing
is that the assigned value for each milestone reflects the actual planned work for achieving
that specific milestone.42
This method is preferred when looking at the performance measurement.43 However it is the
most difficult method when initially planning the project. It takes close cooperation between
scheduling people, resource-estimating functions and work packages managers to get the
milestones meaningful and correctly weighted.
3.2.2.4.2 Fixed Formula
The idea of this method is that one earns a certain value when the work package starts and the
rest when it finishes. The most commonly used values are 25/75, 50/50 and 75/25 meaning
that the percent of the planned value is earned when starting and completing a task
respectively. There is nothing to keep you from choosing different relations as long as they ad
up to 100%. Worth noting is, that if one uses the same values on all work packages and they
are equivalent in size and ff most of the work packages then are closely scheduled afterwards,
this method will work as if you always earn all the planned value when completing a work
package, only with a slight displacement. Simply because you earn the start and end value at
the same time, although not from the same work package.
This method will act well, as long as the work packages are short in duration. If the work
packages span over several report periods, this method will give a lag in the Earned Value. On
the positive side it is one of the simplest methods both to plan and measure. A suited
application for this method is purchased materials, earning 25% of the value when order has
been done and 75% when the material has been delivered.
42
ANSI/EIA-748-A-1998
43
Fleming and Koppelman (2005)
34
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
3.2.2.4.3 Percent Complete Estimates
The Percent Complete method is a subjective judgment of the completed work. The person in
charge of the work packages simply estimate the percent of work completed against the full
planned value of that work package. Sometimes the estimation can be limited with a
“ceiling”. The ceiling means that the person making the subjective estimate only is authorized
to award up to for example 80%. The rest of the value is earned when the work package is
100% completed.44
This method is the easiest of them all to administer which is probably why it have received
increasingly wide acceptance in the private industry.45 But it also has obvious disadvantages.
If one would like to play games with the Earned Value this method is the most vulnerable.
The person in charge of each work package is obviously able to put a gloss on the completed
work. Especially when the pressure is high from the senior managers, the estimates tend to be
a little bit on the large side. To minimize this phenomenon some companies have developed
procedures to incorporate some objectivity into the subjective estimate. Another disadvantage
is that it can sometimes be difficult to actually know how much work has been accomplished.
Of course the situation could be the other way around. The responsible manager
underestimates the performance since they know their reputation might be at stake. It is
always easier to defend modesty than exaggerations. Also, underestimating the progress will
give a buffer for the next report period.
3.2.2.4.4 Percent Complete and Milestones
Combining two techniques, Percent Complete Estimates and Weighted Milestones, provide
yet another way to measure the performance. The value is earned based on the subjective
estimates of the person in charge of the work package. Added to this there is one or several
milestones, or perhaps they should be looked at as toll gates. The manager can only award up
to the predefined value of the next milestone. Only when the milestone has been achieved the
project can earn more value.
44
ANSI/EIA-748-A-1998
45
Fleming and Koppelman (2005)
35
Vestberg, Rasmus
Chapter 3 – Frame of Reference
This method is said to be the best out of two methods, the ease of Percent Complete and the
assured Milestones.46 This seems to work well in any industry and any type of project.
Especially it is a suited method for project with large work packages.
3.2.2.4.5 Equivalent Completed Units
This method can be used when the project consists of several similar units to be delivered.
The value is earned for a full unit and can be rewarded when either a full unit is completed or
when fractions adding up to a full unit is completed. For example if you have ten similar work
packages and you have finished 10% of each of them. Then the sum of the work earned is
equivalent to a full work package which is earned.
This method is often used in construction industry to make progress payment.
3.2.2.4.6 Apportioned Relationship
Using this method on a task means that the value is earned directly related to another work
packages earned value. The other work package is called the measurement base. The
measurement technique of the measurement base could be any of the ones mentioned above.47
This technique could be used when two tasks are closely related to each other. For example if
a work package (the measurement base) consists of code design, then the continuous code
testing could be a separate work package measured with Apportioned Relationship. This
means that the code testing has a separate budget but is rewarded with Earned Value related to
code design. Of course the Earned Value is related in percent. So if the code design has
earned 35% of its Planned Value the code testing has also earned 35%, but of its own Planned
Value. If the apportioned task takes place somewhat after the measurement base, for example
the code testing is done after the code design, then it is ok to let the apportioned task trail the
base task with one reporting period.
The schedule position for the apportioned task and the base task will always be the same
(except for the possible trail). This is natural since any positive or negative schedule variance
46
Fleming and Koppelman (2005)
47
ANSI/EIA-748-A-1998
36
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
will be similar in the apportioned task. However, the cost variance is individual for the related
tasks, since the cost is incurred for the apportioned task itself.
3.2.2.4.7 Level of Effort
Level of Effort (LOE) is used to measure those tasks that are necessary for the project but do
not have any deliverables or performance easy to measure. Examples could be project
management, support functions, security guards etcetera. The value is always earned
automatically according to the Planned Value in Level of Effort measuring.48
Using Level of Effort means that there could be no Schedule Variance, ever. But, since the
cost is still measured the same way as in all measurement techniques, the Cost Variance could
be very big. This is not good at all and it is doubtful that this even qualifies as Earned Value.49
In every project using Level of Effort on some work package, one should quantify the value
of Level of Effort. The quantification is made by dividing the value of Level of Effort with
the value of the total baseline. If the value of Level of Effort tasks exceeds 5-10% the project
is measuring the passage of time instead of true performance.50
A way of dealing with this is to “quarantine” the Level of Effort tasks. That simply means you
remove all task measured with Level of Effort from the baseline and place them in a separate
budget. Those tasks should not be bothered to measure, since they reflect nothing but the
passage of time. 51
One could also replace the Level of Effort with subjective Percent Complete measurements.
The Percent Complete method has its disadvantages as well, but it is still preferable to Level
of Effort52
48
ANSI/EIA-748-A-1998
49
Fleming and Koppelman (2005)
50
Fleming and Koppelman (2005)
51
Fleming and Koppelman (2005)
52
Fleming and Koppelman (2005)
37
Vestberg, Rasmus
Chapter 3 – Frame of Reference
3.2.2.5 Forecasting
The Earned Value Management is a very helpful tool when predicting the outcome of the
project. To make accurate predictions, it is very important that the plan is of good quality. If
the plan is not credible the same thing will be evident for the predictions. Therefore the
managers really have to develop a plan that they do believe is possible to keep up with.
If the measurement of a project in progress shows a cost overrun or a schedule variance the
manager has to take aggressive actions. Unfortunately managers often do not like or accept
the final forecast results and choose not to take necessary action. If the managers did believe
in the plan they created, they should also believe in the predictions given to them concerning
their plan and statistic figures.
In traditional project management an Estimate at Completion (EAC) is calculated as the sum
of Cumulated Actual Cost and a new Estimate to Complete (ETC). The new ETC is prepared
by the project manager based on the experience from the project in progress. This gives a
quite accurate image of the projects future and is probably the most reliable forecasting
method. However, there is one disadvantage with this approach. The work to develop a new
ETC takes time and is most likely not in the scope of the project. That is why the use of
Earned Value Managements easy forecasting is so favorable.53 When using any of the three
formulas below, one uses the EV, AC and BAC at the same level as the forecast is made at.
For example, if one would like to forecast the entire project, the EV, AC and BAC for the
entire project are used.
3.2.2.5.1 Mathematical EAC – Best Case Scenario
This forecasting formula is very simple and easy to understand. The EAC is calculated as the
Actual Cost, plus the Budget at Completion, less the Earned Value. That simply means the
Actual Cost today, plus the rest of the budgeted work.
Estimate at Completion = Actual Cost + ( Budget at Completion − Earned Value)
Formula 4
53
Fleming and Koppelman (2005)
38
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
This formula can be looked upon as a best case scenario. The project has incurred an overrun
which not likely will go away. Assuming that the rest of the project will follow the plan, this
gives the accurate estimate. Almost never the project will underrun in the future and
compensate the early overrun.54
3.2.2.5.2 Cumulative Cost Performance Index – Most Likely
This Formula is based on the experience of the project given by the CPI (se 3.2.2.1). When
calculating the EAC one assumes that the remaining work will be done with the same
efficiency as the performed work.
Estimate at Completion = Actual Cost +
Budget at Completion − Earned Value
Cost Performance Index
Formula 5
To smoothen out the varieties of the project performance the used CPI is the cumulative to
date. This will not give the project team a perfect picture of their recent work, but it will
provide an accurate long-term forecasting. The Formula 5 can be simplified as follows.
Estimate at Completion =
Budget at Completion
Cost Performance Index
Formula 6
This method is often referred to as the “most likely” outcome of the project.55 Studies of
several projects show that the CPI at 20% completion will be stable for the rest of the
project.56
3.2.2.5.3 Cumulative CPI times SPI – Worst Case Scenario
As the heading implies this method uses both the CPI and the SPI.
54
Fleming and Koppelman (2005)
55
Fleming and Koppelman (2005)
56
Christensen and Heise (1993)
39
Vestberg, Rasmus
Estimate at Completion = Actual Cost +
Chapter 3 – Frame of Reference
Budget at Completion − Earned Value
Cost Performance Index * Schedule Performance Index
Formula 7
Since it is very common that project team strive to be on track with the schedule, this formula
is appropriate. If a project is behind schedule, it will take extra effort and probably also
money to get it on track again. By multiplying the CPI and SPI one gets a good performance
factor. If both the SPI and CPI is less than one (1), the outcome will be worse than in ordinary
cumulative CPI forecasting. This is perfectly natural, since it will take extra funds to
compensate the schedule slippage. On the other hand, if one of CPI or SPI is above one (1),
the performance factor indicates that the better of the two compensates the worse.
This calculation method is often referred to as the worst case scenario. Although, there are
some people claiming this to be the most likely outcome of the project.57
3.2.2.5.4 Predicting completion date
Formula 4 and Formula 6 can also be used to predict the final completion dates of the project.
Instead of using CPI one uses SPI to get the forecast in aspect of schedule.
There is no statistic data provided showing this to be a valid forecasting method. Actually
some would say this is a to simplified way of predicting the completion date and that it takes
careful Critical Path work to do this properly. The SPI only indicates the status of the project
based on time and is not sufficient to forecast the completion date itself. Analyzing the
Critical Path combined with SPI calculation is the recommended method.58
3.3 Earned Scheduling
As earlier written, the schedule performance is measured in money. That might be a little bit
difficult to grasp in the beginning, but will eventually make perfectly sense. This is easy to
57
Fleming and Koppelman (2005)
58
Fleming and Koppelman (2005)
40
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
measure and it correlates well with the cost performance which naturally also is measured in
money. However, this way of measuring the schedule performance has disadvantages.
Since the Schedule Variance and the Schedule Performance Index is calculated based on the
planned value, some false results will occur. When the project is supposed to be finished the
Planned Value will not change any more.59 This means that no matter how late the project
actually finishes, the Schedule Variance will approach zero and the Schedule Performance
Index approaches one. This is simply because the Earned Value always equals the Planned
Value at completion of a project; otherwise the project would not be completed.60
Because of this flaw, Earned Value Management is sometime labeled as a cost management
system.61
The remedy for this is to use Earned Scheduling. This is the same thing as Earned Value but
the Schedule Performance is measured in time instead of in money. The Earned Schedule, ES,
is reflecting the time at when the achieved work should have been done according to plan.
The Actual Time, AT, is the time of today. The planning of the project is exactly the same as
with Earned Value and one prepares a baseline to measure against. The only difference is that
one calculates the Schedule Variance as the horizontal difference against the baseline, instead
of a vertical difference against the baseline (see Figure 7). That is, the vertical axis represents
money while the horizontal axis represents schedule time. That will give us the following
formula for calculating SV(t) and SPI(t).62
Schedule Variance(t ) = ES − AT
Schedule Performance Index(t ) =
ES
AT
Formula 8 a and b
The classic Schedule Variance, SV($), shows the difference between the amount of achieved
work and planned work. The SV(t) shows the difference in calendar time between today’s
59
Corovic Radenko (2007)
60
Lipke Walt (2003)
61
Lipke Walt (2006)
62
Lipke Walt (2003)
41
Vestberg, Rasmus
Chapter 3 – Frame of Reference
date and the date when one should have accomplished the amount of work being finished of
today (se Figure 7).63
1600
Budget
1400
1200
SV($)
SV(t)
Cost
1000$
1000
800
600
400
200
0
jan
feb
mar
apr
maj
jun
jul
aug
sep
okt
nov
dec
Date
Figure 7 SV(t) and SV($)
Even if analyzing the Critical Path is probably the best way of forecasting the completion
date, this extension to Earned Schedule provides a tool suited also for time prediction. The
benefit with Earned Schedule against analyzing the Critical Path is the simplicity. In large
projects the network schedule is often of very high complexity, therefore a simple analyzing
tool as Earned Schedule suits well.64
Studies have been made, although not as extensive as DoD’s EVM-study, showing that this
new approach correlates well with the Earned Value basics. Not only do they work together
with EVM’s cost analyze, they also behave correctly over the entire project.65 This makes it
63
Abba Wayne (2002)
64
Henderson Kym (2005)
65
Henderson Kym (2003)
42
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
possible to forecast the outcome of the project, as well on schedule basis, without analyzing
the critical path.66
If one agrees that Earned Schedule provides the true picture of the Schedule Performance, we
can use it to see how incorrect the SV($) can be. All practitioners agree in that SV($) give
erroneous figures after the completion date. But, one can show that SV($) is unreliable over
the entire project when compared to SV(t). This is in the case when the project has a non
linear baseline (see Figure 8).
1600
Budget
1400
Earned Value
1200
1000$
1000
SV($)2
800
600
SVt)2
400
200
SV($)1
SV(t)1
0
1
2
3
4
5
6
7
8
9
10
11
12
Date
Figure 8 Erroneous behaviour in a non linear baseline
If looking at a project with perfect performance, only starting one month late, the problem
becomes obvious. The SV(t) is constant over the entire project (SV(t)1 = SV(t)2), while the
SV($) changes radically (SV($)1 ≠ SV($)2). A project with perfect performance would have
the same Schedule Variance through the entire project. If the baseline is perfectly linear, the
two different measurement techniques will provide the same data.67
66
Henderson Kym (2004)
67
Corovic Radenko (2007)
43
Vestberg, Rasmus
Chapter 3 – Frame of Reference
3.4 Gold Practice
The Data & Analysis Center for Software, DACS, is a Department of Defense in USA. DACS
is collecting data from different software development projects to compile the “best practice”
which they have chosen to call Gold Practice. Gold Practice instead of Best Practice is simply
because there can not be only one right way for every company. The intent of DACS Gold
Practice Community is to share experience with each other. The following part will present
the recommendations made in Gold Practice website regarding Earned Value Management.68
This section has been added based on a wish from the Manager at Naval Operations Project
Office.
Defining the work and developing a Work Breakdown Structure, WBS, is essential for the
successful use of EVM according to DACS. Each work package in the WBS must have a
short duration or be divided into measurable milestones. Further more it is important to keep
the WBS on the right level. To detail planning will create an overload of data which is hard to
track while lack of details may mask important information. The Gold Practice suggests 3-4
levels in the WBS for regular projects and up to 6 levels for very large and complex projects.
The Earned Value is suggested to be calculated as either 0% or 100% where 100% is given
when the task is completed and validated. The Gold Practice also suggests that one should
establish specific exit criteria for each task to make the measurement easier. Doing subjective
assessment to gain the earned value is strongly not recommended.
Later in the project when the work breakdown is done, the baseline is set according to that
specific planning package. Then you can not change the baseline and as a consequence not
either the budget of that working package. That means that the breakdown and detail planning
comprise splitting the budget on different work packages.
68
www.goldpractices.com
44
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
45
Vestberg, Rasmus
Chapter 4 – Empirical Study
4 Empirical Study
This chapter will present the results from the analyzed work packages and the mapping of
how project work is conducted at NSDS. The figures used to compute the data is shown in
Appendix 1. All results are presented in graphs to increase the visibility.
4.1 EVM at Naval Systems Division
Earned Value Management was adopted in the beginning of 2006. This part will describe the
situation at Naval Systems Division regarding the project organization and the Earned Value
Management.
4.1.1 Project Organization
All development at Naval Systems Division is made in project organization, although the
project organization is not really the traditional one. In traditional terms a project is supposed
to have a limited extension in time, a fixed budget and temporary staff.69 At Naval Systems
Division all project are limited in time even if some of them is extended to up to ten years.
The budget, as well, is often very large but still fixed. So at those issues there is nothing odd
about the project organization, although the projects are extraordinary large. The big
difference in the Naval Systems Divisions project management is in the organization. The
projects consist of only one or two persons, the project manager and sometimes a system
engineering manager. The rest of the resources in the projects are bought from the functional
organization. This would not have been strange if it was the staff that was bought but in this
case it is the component that is bought. That means that the project manager has no
responsibility over the handling of the staff. Instead the project manager goes to a Program
Project Leader, PPL, who has a functional responsibility over one component. They make an
agreement over what is supposed to be delivered and at what time and cost.70 After that it is
up to the PPL to plan and produce the agreed component. That means that it is the PPL that
updates the plan according to the progress. If the work with one component is delayed or
69
Project Management Body of Knowledge (2005)
70
Henrik Olsson
46
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
more expensive than agreed there is not much the project leader can do. This type of project
model is called procurement model.71
The Naval Systems Division has been running projects in a more standardized way earlier but
a couple of benefits with today’s organization led to the changes. First of all, many of the
projects at Naval Systems Division develop or use the same technology or product. Roughly
you could say that the Naval Systems Division has a couple of systems that are developed or
tailor-made for special applications and customers. Therefore there are very much of the work
that is similar or perhaps exactly the same in different projects. The Naval Systems Division
uses something they have chosen to call “the Family”, which is a collection of existing
solutions for different problems. If someone improves something they got from “the Family”
this improvement will be saved to provide others with the latest solution. Hence the existing
organization makes double work less common.72
This type of organization also makes it easier to develop or use new technology since it is
more flagrant where and by whom the technology should be adapted. By the same reason it is
easier to merge organizations since the functions and responsibilities are more obvious.73
Much of the reasons can probably be referred to the size of the projects. The benefits with the
more traditional project organization were not very appreciable since the size made the project
leader loose control anyway. A fact is that the Naval Systems Division has increased its result
considerably with the new organization.74
Since the beginning of this study NSDS has made some changes in the project organizations.
These are not dealt with since they do not affect the purpose of the study and the analyzed
data derives from the described organization.75
71
Projekthandbok, SAAB Systems
72
Henrik Olsson
73
Person A
74
Person A
75
Trochez Michael
47
Vestberg, Rasmus
Chapter 4 – Empirical Study
4.1.2 Earned Value
In the early 2006 the Naval Systems Division started using Earned Value as a measurement
tool for project management. Earlier most of the project was managed without knowing the
exact status of today.76 But the Earned Value concept should not be considered as news for
the Naval Systems Division. The thoughts behind Earned Value have been present since the
beginning of the eighties. 77 At that time the concept was called Performance Measurement
System, PMS, which still can be found referenced in some steering documents at the Naval
Systems Division. In the “Projekthandbok” (project guidelines) at SAAB Systems it is stated
that PMS is a tool recommended to use.78 (The “Projekthandbok” has not been updated since
2002) Since that time the basics of Earned Value has been used by different people at
different levels in the organization, shifting dependent on normal reorganizations and interest.
The main difference today is the manifest throughout the entire organization that the Earned
Value Method should be embraced. The reason for this change is as simple as a customer
demand. The largest customer in Sweden stated in 2003 that Earned Value should be used as
a reporting technique in all their projects. At the same time there was a need in a project to be
able to produce reports in Earned Value figures to the Australian Division. In Australia the
Department of Defense has been demanding their supplier to use Earned Value for quite a
time. Those facts combined with the wish from SAAB Group’s management to get reports
based on Earned Value figures lead to the manifestation of the change.79
4.1.2.1 Scoping the Project at Naval Systems Division
When scoping the project at NSDS the work is planned either in planning packages or work
packages.
4.1.2.1.1 Planning Packages
At the project start the entire project is broken down into a big WBS-chart and WBSdictionary. The WBS is principally product oriented although some activities can be stated as
76
Ulf Karlsson
77
Stefan Larsson
78
Projekthandbok, SAAB Systems
79
Michael Trochez
48
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
work oriented WBS-elements. The work is planned in planning packages assigned to
individual documents stating the work to be done. Those documents constitutes the agreement
between the Project Management and the Functional Organization and defines the scope of
work to be done, the inputs, interdependencies and outputs, key milestones and allocated
budget. The packages should be broken down at a level well suited for delegation.80
4.1.2.1.2 Work Packages
There is no specific rule about how far in advance the planning packages should be converted
into work packages at the Naval Systems Division. The conditions for each activity must be
decisive for the time chosen. The time needed to allocate wanted resources is a good guidance
according to the “Projekthandbok”, which also indicates that there is no reason to plan non
critical activities far in advance.81 The common attitude among the Project Managers at Naval
Systems Division is that approximately six months ahead should be planned at work package
level.
The breakdown into work packages is made as a Bottom-Up planning. This means that the
sum of the work packages does not always equal the corresponding planning package. If the
difference is insignificant, no action is taken at all. If the difference is of major size, relative
the project, the baseline can be changed. If the baseline needs to be changed that decision has
to go through the managerial body.82
4.1.2.2 Performance Measurement Techniques
Two Performance Measurement Techniques are used at NSDS, Percent Complete Estimates
and Level of Effort.
80
Projekthandbok, SAAB Systems
81
Projekthandbok, SAAB Systems
82
Henrik Olsson
49
Vestberg, Rasmus
Chapter 4 – Empirical Study
4.1.2.2.1 Percent Complete Estimates
In Naval Systems Division the earned value is achieved in some automatic kind of way. All
employees working at Naval Systems Division reports their hours into a computer system.
The time reported into each work package generates the earned value based on the time spent
and the planned value. No regard is taken to the earlier cost performance when calculating the
value. The system has limitation to prevent the Earned Value to exceed 100% in case the time
reported would be more than the time planned.83
Periodically, at least once a month, the person responsible for each work package updates the
plan and the planned cost for the work package. To make Earned Value Management work
changes in the original baseline is not recommended, SAAB uses therefore several baselines.
The original baseline is named “Baseline 10” and can only be changed if there is a change of
scope, the changing baseline will simply be denoted as “baseline”. If the work package has
used 50% of its time planned in the baseline the work package has earned 50% of the
baseline10. This means that as long as the plan is updated and the baseline is the latest
estimated cost and duration for each work package, the Earned Value will be automatically
generated based on the reported hours.
Earned Value =
ActualWork
* Baseline10
Actual Work + Estimated Remaining Work
Formula 9
If the “Baseline 10” is budgeted as 500 hours, the actual work is 300 hours and the updated
baseline is set to 600 hours, then the Earned Value will be 250 hours. The actual work
represents 50% of the latest estimate of that specific work package, this means that the project
earns 50% of the original baseline; the “Baseline 10”.
4.1.2.2.2 Level of Effort
The performance measurement method Level of Effort is used at the Naval Systems Division
for work packages in which it is difficult to measure the progress. Example could be Project
83
Yngve Henriks
50
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Management, Configuration Management or Quality Management. At the Naval Systems
Division the proportion of Level of Effort varies between 9.4% and 46.8% depending on
which project one looks at. The average proportion of Level of Effort is 19.7% at Naval
Systems Division.84
4.1.2.3 Scheduling
The planning group has the responsibility to schedule projects tasks. As the base of the
scheduling the planning group uses Microsoft Project Server, where all task are gathered and
connected with interdependencies. This Microsoft Project schedule contains for example
information about start and finish dates, budgeted costs and apportioned resources. This data,
as the core of Earned Value, is exported to another system handling the Earned Value data
and metrics.
Since the Naval Systems Division project organization is put together the way it is, one of the
major tasks for the planning group is to manage resources amongst the different projects. This
means keeping track on the covering of staff. Making sure each project gets its required staff
and keeping individuals from being over scheduled. The constant updating of the project
plans is necessary to schedule the resources associated with each work package. This
information is not provided when using performance indexes.85
4.1.2.4 Forecasting and Predicting Completion Date
Neither the Cost Performance Index nor the Schedule Performance Index is used to forecast a
project. Instead of using the formulas described above (3.2.2.5) the forecasting at Naval
Systems Division is based upon the updated baseline as described above (4.1.2.2.1). This
makes the forecasting very simple concerning both time and money since the plan is updated
with the latest revised estimates. That is since the plan has already been updated based on the
experience and the subjective judgment of the responsible person for each work package.86
84
Michael Trochez
85
Yngve Henriks
86
Björn Lindberg
51
Vestberg, Rasmus
Chapter 4 – Empirical Study
Estimate at Completion = Actual Work + Estimate to Complete
Formula 10
The Managers at Naval Systems Division believe that using the CPI and SPI when predicting
the projects outcome is not justifying. A late start will have to large impact on the Estimate at
Completion when using the traditional Earned Value Management forecasting methods. The
experience and subjective judgment from the Project Leader is a much more reliable way of
predicting the projects future.87
4.2 Forecasting Accuracy at NSDS
As earlier mentioned the forecasting at Naval Systems Division is based on the subjective
judgement from the people responsible for each work package. The following graphs
represents the accurateness of those prediction based on the final outcome of the work
package. Worth noting is that when there has been a change of scope this has been filtered to
still show the accurateness of the predictions. That means, bad predictions can not be
explained by change of scope.
4.2.1 Accurateness Relative Entire Work Package
The easiest and most obvious approach to show the accuracy of the prediction is to compare
the final outcome of the work package with the estimated outcome at each time. The
difference between the prediction and the true outcome is the searched value. This is
displayed in the following graphs relative the final entire work for each work package.
Forecasting Accuracy =
Forecast time of measurement − Actual Cost at completion
Actual Cost at completion
Formula 11
87
Yngve Henriks
52
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
In Figure 9 each line represents one work package. The x-axis represents the timeline in
months based on the true dates for the studied work packages. The y-axis represents the
Forecasting Accuracy in percent of the entire work package. The dots on each curve
represents the time when each work package finished. The finish date is set when the Earned
Value has reached 100%. Some of the work packages have smaller changes after the finish
date, which is a consequence from the use of Level of Effort (see 6.1.3).
400,00%
200,00%
100,00%
jun-0
7
maj07
7
apr-0
7
mar-0
feb-0
7
jan-0
7
6
6
okt-0
dec-0
6
sep-0
nov06
6
aug-0
jul-06
jun-0
6
maj06
6
0,00%
apr-0
Forecast relative Actual at Completion
300,00%
-100,00%
-200,00%
Date
Figure 9 Difference in Forecast and Actual at Completion relative entire work package, true finish
dates
53
Vestberg, Rasmus
Chapter 4 – Empirical Study
In Figure 10 the work packages has been horizontally displaced so that all work packages end
in June 2007. This displacement is made to make the analysing of forecasting accuracy trends
easier. As seen in Figure 10 the different curves start at different points. Those points are not
the true start date of the work packages; it is only the start date of the available historical data.
400%
200%
100%
jun-0
7
maj07
7
apr-0
07
mar-
feb-0
7
jan-0
7
6
dec-0
6
okt-0
6
6
sep-0
nov-0
6
aug-0
jul-06
jun-0
6
maj06
6
0%
apr-0
Forecast relative Actual at Completion
300%
-100%
-200%
Date
Figure 10 Difference in Forecast and Actual at Completion relative entire work package, finish
dates displaced to jun-07
54
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Figure 11 is a more detailed through zoom example of Figure 10 to further increase the clarity
of the graphs. In Figure 11 two broken lines have also been inserted at ±10%. Before the
study took place the principal stated that when the forecasting is within 10% we can say that
the knowledge of the outcome is acceptable.
30%
10%
7
jun-0
maj07
apr-0
7
7
mar-0
7
jan-0
feb-0
7
6
dec0
n o v06
6
okt-0
6
sep-0
aug06
jul-0
6
6
jun-0
maj06
0%
apr-0
6
Forecast relative Actual at Completion
20%
-10%
-20%
-30%
Date
Figure 11 Difference in Forecast and Actual at Completion relative entire work package, finish
dates displaced to jun-07, zoomed in
4.2.2 Accurateness Relative Remaining Work
A second approach when looking at the accuracy of the predictions is to measure the
difference between the forecasted and the actual values based on the remaining work. The
only difference in this approach is that the quota is based on the remaining work instead of the
total work at each time.
Forecasting AccuracyRe lative remaining work =
Actual Cost at Completion − Estimated Cost at Completiontime of measurement
Actual Cost time of measurement − Actual Cost at Completion
Formula 12
55
Vestberg, Rasmus
Chapter 4 – Empirical Study
In Figure 12 each line represents one work package. The x-axis represents the timeline in
months based on the true dates for the studied work packages. The y-axis represents the
Forecasting Accuracy in percent of the real remaining work. The work packages have been
horizontal displaced making them all end in June 2007.
1000%
600%
400%
200%
jun-0
7
maj07
7
apr-0
7
mar-0
feb-0
7
jan-0
7
6
dec-0
nov06
6
okt-0
sep06
aug06
6
jul-0
jun-0
6
maj06
6
0%
apr-0
Forecast relative Actual at Completion
800%
-200%
Date
Figure 12 Difference in Forecast and Actual at Completion relative remaining work, finish dates
displaced to jun-07
56
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Figure 13 is a more detailed through zoom example of Figure 12 with the broken lines
inserted to highlight the ±10% area.
100%
80%
40%
20%
7
jun-0
maj07
apr-0
7
7
jan-0
mar07
6
dec-0
feb-0
7
6
nov-0
6
okt-0
6
sep-0
aug-0
6
jul-06
6
jun-0
-20%
maj06
0%
apr-0
6
Forecast relative Actual at Completion
60%
-40%
-60%
-80%
-100%
Date
Figure 13 Difference in Forecast and Actual at Completion relative remaining work, finish dates
displaced to jun-07, zoomed in
4.3 NSDS Forecasting vs. Cumulative CPI Forecasting
The available historical data, used to present the accuracy of the forecasting at Naval Systems
Division, can be used to generate Cumulative CPI Forecasting for the same work packages.
The reports providing us with historical data also contain the Actual Cost at each time. The
Cumulative CPI is calculated with the following formula.
Estimate at Completion =
Budget at Completion
Budget at Completion
=
Cost Performance Index Earned Value Actual Cost
Formula 13
Provided those data we can compare the accuracy of the subjective method used at NSDS and
the mathematical CPI method.
57
Vestberg, Rasmus
Chapter 4 – Empirical Study
4.3.1 Relative Entire Work Package
In Figure 14 to Figure 21 the graphs display the accuracy relative the entire Work Package as
in 4.2.1. Each figure represents one work package forecasted in the two different ways. The
red line represents the CPI method while the black line represents the subjective method used
today at NSDS. In each graph the average inaccuracy for each work package and method is
displayed. The following graphs are meant to show trends and should not be analyzed in
detail one by one.
12%
10%
8%
6%
4%
Forecast using NSDS method
Average: 2%
2%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
Forecast using CPI
Average: 3%
-2%
Date
Figure 14 Work package 1, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
58
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
40%
30%
10%
Forecast using NSDS method
Average: 9%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
20%
-10%
-20%
-30%
-40%
Forecast using CPI
Average: 19%
-50%
-60%
Date
Figure 15 Work package 2, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
60%
20%
Forecast using NSDS method
Average: 18%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
40%
-20%
-40%
-60%
Forecast using CPI
Average: 39%
-80%
Date
Figure 16 Work package 3, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
59
Vestberg, Rasmus
Chapter 4 – Empirical Study
10%
Forecast relative Actual at Completion
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0%
-10%
-20%
Forecast using CPI
Average: 14%
-30%
-40%
-50%
Forecast using NSDS method
Average: 16%
-60%
Date
Figure 17 Work package 4, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
90%
Forecast using NSDS method
Average: 75%
80%
Forecast using CPI
Average: 61%
60%
50%
40%
30%
20%
10%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
70%
Date
Figure 18 Work package 5, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
60
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
80%
Forecast using NSDS method
Average: 12%
70%
Forecast relative Actual at Completion
60%
50%
40%
30%
20%
Forecast using CPI
Average: 5%
10%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0%
-10%
-20%
Date
Figure 19 Work package 6, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
100%
Forecast using NSDS method
Average: 25%
60%
40%
20%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
80%
-20%
Forecast using CPI
Average: 10%
-40%
Date
Figure 20 Work package 7, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
61
Vestberg, Rasmus
Chapter 4 – Empirical Study
40%
30%
10%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
20%
-10%
-20%
-30%
Forecast using NSDS method
Average: 22%
-40%
-50%
Forecast using CPI
Average: 16%
-60%
Date
Figure 21 Work package 8, Forecast using CPI vs. Forecast using NSDS method, relative entire
work package
For the eight work packages represented in the graphs above, the total average inaccuracy is
22% respectively 21% for the subjective NSDS’s method and the CPI method.
4.3.2 Relative Remaining Work
In Figure 22 to Figure 29 the graphs display the accuracy relative the remaining work as in
4.2.2. Each figure represents one work package forecasted in the two different ways. The red
line represents the CPI method while the black line represents the subjective method used
today at NSDS. In each graph the average error for each work package and method is
displayed. The following graphs are meant to show trends and should not be analyzed in
detail one by one.
62
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
1000%
Forecast relative Actual at Completion
800%
600%
400%
200%
Forecast using CPI
Average: 170%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0%
Forecast using NSDS method
Average: 247%
-200%
Date
Figure 22 Work package 1, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
80%
60%
20%
Forecast using NSDS method
Average: 30%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
40%
-20%
-40%
Forecast using CPI
Average: 35%
-60%
-80%
-100%
Date
Figure 23 Work package 2, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
63
Vestberg, Rasmus
Chapter 4 – Empirical Study
300%
250%
Forecast relative Actual at Completion
200%
150%
100%
50%
Forecast using NSDS method
Average: 67%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0%
-50%
Forecast using CPI
Average: 62%
-100%
Date
Figure 24 Work package 3, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
40%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
20%
-20%
-40%
Forecast using CPI
Average: 43%
-60%
-80%
Forecast using NSDS method
Average: 42%
-100%
Date
Figure 25 Work package 4, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
64
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
600%
Forecast relative Actual at Completion
500%
400%
300%
200%
Forecast using NSDS method
Average: 236%
Forecast using CPI
Average: 156%
100%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
1
0%
Date
Figure 26 Work package 5, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
400%
350%
Forecast using NSDS method
Average: 102%
250%
200%
150%
100%
Forecast using CPI
Average: 59%
50%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
300%
-50%
-100%
Date
Figure 27 Work package 6, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
65
Vestberg, Rasmus
Chapter 4 – Empirical Study
250%
150%
Forecast using NSDS method
Average: 73%
100%
50%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
200%
-50%
Forecast using CPI
Average: 37%
-100%
Date
Figure 28 Work package 7, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
150%
50%
Forecast using CPI
Average: 28%
-50%
15
14
13
12
11
10
9
8
7
6
5
4
3
2
0%
1
Forecast relative Actual at Completion
100%
Forecast using NSDS method
Average: 42%
-100%
Date
Figure 29 Work package 8, Forecast using CPI vs. Forecast using NSDS method, relative remaining
work
66
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
For the eight work packages represented in the graphs above, the total average inaccuracy is
102% respectively 72% for the subjective NSDS’s method and the CPI method.
67
Vestberg, Rasmus
Chapter 5 – Analysis
5 Analysis
This chapter contains the analyze of the result of the empirical study and the frame of
reference concerning Naval Systems Division. Deeper discussion about the findings will be
done in chapter 6 Discussion.
5.1 Planning package conversion
The purpose of studying the accuracy of the forecasting at Naval Systems Division was to
provide statistic data telling us how far in advance we can make plans at work package level.
The attempt was to find a trend showing when the forecasting is stabile within 10% from the
final outcome. Through the way of measuring performance at NSDS the forecasting could be
used for this purpose.
In the empirical study there have been two different approaches of how the forecasting
accuracy should be measured. Either one measures the accuracy relative the entire work
(Figure 9-Figure 11) or relative the actual remaining work (Figure 12 and Figure 13).
5.1.1 Relative Entire Work
When measuring the forecasting accuracy relative the entire work package the size of the
work package will have great influence on the result. In a large work package there can be a
significant amount of work left, having great impact on the entire project, still constitute only
a smaller part of the work package. This means that bad predictions are excused thanks to the
size of the work package.
This way of measuring will always approach perfect prediction. The remaining work and
therefore also the prediction offset, approaches zero during the progress of the work package.
Since the entire work eventually will be much larger than the prediction offset the relative
offset will approach zero.
This approach could be looked upon as the measure of how well one could predict the entire
work package at different points.
68
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Figure 9 displays the work packages in an authentic timeline. This is a little bit roundabout to
analyze. Therefore we look at Figure 10 and Figure 11 instead. Those display the same data
only displaced in time to ease analyzing. Figure 11 is a more detailed through zoom example
of Figure 10.
If starting to look at Figure 10 one can see that one year before completion, seven out of
eleven work packages are close to or more than 100% wrong. This means that the estimation
is less than half or more than twice the actual. At some point between seven and nine months
before completion the curves starts to close up on the zero line although they are not within
the 10% limit. Worth noticing is that it is more common with overestimates than
underestimates.
It is most likely that the close up depends on the relative entire work instead of the better
knowledge. Since the curves relative remaining work do not show the same trend at the same
point of time (see 5.1.2). It is more likely that the trend shows that seven to nine months
before completion there is not much work left on the work packages closing up on the final
outcome, instead of showing that the predictions are more valid within this time period.
If looking at the zoomed in Figure 11 we can see that more than a third of the work packages
do not enter within the 10% limit until the work is completed. For the rest of the work
packages they are within the 10% area between 3.5 and 9 months before completion with a
median of 5 months. The work packages entering within the limit at completion is probably
subject for trick based on individuals wish to always under run.
5.1.2 Relative Remaining Work
When measuring the forecasting accuracy relative the remaining work the finished work is
ignored. This makes this method equal the forecasting of a small work package with the same
size as the remaining work.
In the beginning of the work package this approach will give almost the same results as
measuring against the entire work package. This is because the remaining work is close to the
69
Vestberg, Rasmus
Chapter 5 – Analysis
entire work. When work precedes the difference between the entire work and the remaining
work will increase, consequently the methods will give different results.
One should be aware of the possibility of extreme data in the very end of the project or when
there is very small amount of work left. If the remaining work is very small, insignificant
forecasting errors could give large turns on the relative value. Because of this one might not
pay to much attention to the figures provided by this method in the very last phase of the
projects.
This approach could be looked upon as the measure of how well one could predict the
remaining work at different times.
Figure 12 and Figure 13 displays the work packages relative the remaining actual work. The
figures are displaced in time to ease analyzing and Figure 13 is a detailed through zoom
example of Figure 12.
If looking at Figure 12 we can see that some work packages are very poor forecasted. At
worst one is more than eight times overestimated. One can also see the phenomena in the last
two months mentioned above.
If ignoring the last two months and looking at Figure 13 we would hopefully be able to
distinguish some trends suggesting the time frame for planning at work package level.
Unfortunately it is impossible to tell from the curves that the forecasting is getting better in a
certain time frame. In fact all work packages are only occasionally inside the 10% which
might as well be luck as good predictions.
5.2 PMT
One of the purposes of studying literature about Performance Measurement Techniques was
to provide suggestions of improvement within this area at Naval Systems Division.
Combining the information from the literature and the situation at NSDS provides the
fundament for later given suggestions. Some of those suggestions can be supported by some
comparing made in the empirical study.
70
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
5.2.1 Analyze based on Frame of Reference
If the plans are not running updated the whole Earned Value Concept at Naval Systems
Division will fall apart. Since the generating of Earned Value is based on the updated plan
this must work. If not, the measurement becomes in practice the same as in traditional project
management; the Earned Value equals the money spent. The way of measuring equals the
Percent Complete method with subjective judgement. The only difference is that if the
judgements are not made the NSDS method will earn the planned value while the Percent
Complete method will earn no value at all. Earning no value at all will most likely force the
manager to update their plans. Earning perfect value as a consequence of not doing anything
at all does not encourage correct behavior.
The Gold Practices suggest that one should use Fixed Formula with the value 0% and 100%,
earning 100% when the work package is finished. This method is in practice the same thing as
Weighted Milestones. In Weighted milestones the work packages are a little bit bigger and
one chooses to split the work into measurable parts with the different milestones. In the Fixed
Formula method one splits the work at the milestones into separate small work packages. In
the Gold Practices there is strongly recommend not to use the subjective percent complete
method.
The use of Level of Effort is not recommended based on the non real value quality. If the
Level of Effort exceeds 5-10% of the work packages one measures the passage of time instead
of true performance. At NSDS the average of all work measured using LOE is 19.7%. In the
worst project the amount of LOE is 46.8%, that is almost half the project. Those figures truly
indicate that one is not using EVM at its best capacity at Naval Systems Division.
5.2.2 Analyze based on Empirical Study
Since the forecasting and calculation of Earned Value is closely related at Naval Systems
Division, in fact the precise same thing, we can use the forecasting method to evaluate the
used Performance Measurement Technique. Since the forecasting is, as stated above, very
inaccurate the Earned Value must be as well. This implies that we must consider changing the
Performance Measurement Technique to get the correct Earned Value.
71
Vestberg, Rasmus
Chapter 5 – Analysis
One of the great advantages with Earned Value is the easy forecasting provided by the
performance indexes. If NSDS changes the Performance Measurement Technique they will
no longer have the forecasting provided “for free”. They could of course continue with the
subjective judgments to predict to future work packages, this is in literature stated as probably
the best forecasting method. However, this method takes a lot of time (see 3.2.2.5). Instead of
sticking to the old forecasting one could start using the simple index based method
recommended by PMI. The method mentioned as the “most likely” is the Cumulative CPI
method (see 3.2.2.5.2). Therefore this method has been compared with NSDS forecasting
method in Figure 15 to Figure 29. Figure 15 to Figure 21 shows the forecasting error relative
entire work package while Figure 22 to Figure 29 shows the error relative remaining work.
The end points for all the curves are set to the time when all the work packages have reported
100% Earned Value. If first looking at the figures displaying relative entire work, we can see
the following things: The endpoint for the NSDS method does not always equal zero. This is
because the Earned Value has been reported as 100% although the estimates still indicating
that there is some work left. The reason for this is the problem behind Level of Effort.
Although the Level of Effort package has been displaced in time and the manager estimate the
work to be not finished, the value will be earned according to the original baseline. This is
why the NSDS method does not always equal zero in the end, as it is supposed to do with the
relative entire work approach. The CPI method always equals the actual at completion when
100% value has been earned. Besides this there is no larger difference in the precision of the
two methods. The average error for the CPI respectively the NSDS method is 21% and 22%,
which shows an insignificant difference.
If looking at the figures showing the errors relative the remaining work the differences
becomes more evident. The NSDS method is no longer expected to end at a zero error (see
5.1.2) but the CPI method still does. This makes the CPI method more stabile and the average
error for the CPI respectively the NSDS method is 72% and 102%. The CPI method is then
proven to be more valid in this case.
Important to notice is that this is only valid for this specific Division or perhaps only this
specific case. One also has to keep in mind that the CPI is based on the Earned Value. This
gives a circle relation in this case since the Earned Value is related to the forecasting. The
72
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
Earned Value has already been condemned why also the CPI method will be flawed.
However, if the Earned Value were more valid the CPI method would also provide better
accuracy.
73
Vestberg, Rasmus
Chapter 6 – Discussion
6 Discussion
This chapter will discuss the findings made in the Frame of Reference and the Empirical
study. In this chapter I will also present my own ideas and conclusions based on the earlier
presented information and data.
6.1 PMT
This section will discuss findings concerning Performance Measurement Techniques.
6.1.1 Automatic Earned Value
At first sight it might seem to be a good idea to make the earned value automatic in some
way. For example one could use the time reported on each work package and transfer the
figures to earned value. If one uses the cost performance index to date, and assume that the
index is the same for this work package, the estimated earned value would most likely be
pretty accurate. To prevent the possibility that one receives more than 100% earned value of
planned value one could perhaps say that the last 20% only can be awarded by the project
manager or at completion of a task as in Percent Complete Estimates with a ceiling.
However, on second thought, one might ask oneself, is this really earned value? Is this not in
fact only the traditional project follow-up dressed in earned value? If we do not measure the
value but instead calculate the value based on work performed we can not really tell anything
about the status of that particular work package. The only thing this will give us is the planned
value updated based on the projects performance so far. This is in fact the estimated budgeted
cost for work scheduled. The usage of cost performance index in this example does not help
as much as one might think. Since the cost performance index originate from the previous
earned value and the previous actual cost we will have a problem since the earned value was
not really earned value. Only the fully completed work packages will help with providing a
correct cost
74
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
6.1.2 NSDS subjective Measurement Technique
This measurement between “Baseline 10” and baseline means that one can lose already
earned value. Let us say the work package’s “Baseline 10” has been apportioned 800 hours,
the baseline has not been changed and the persons working with the work package has spent
400 hours. Then we have an earned value of 50% of the baseline which gives 400 hours from
the “Baseline 10”. When updating the plan the manager realizes that the work package in fact
will spend 1000 hours. The change of the baseline can originate from many things e.g. bad
plan, unexpected problems, errors etc. Since there has not been any change in scope the
“Baseline 10” is unchanged. This means that we have only done 40% of the baseline which in
turn means that we can only receive 40% earned value from the “Baseline 10”, 320 hours. So
the earned value has decreased from 400 to 320 hours.
The way of measuring the Earned Value at Naval Systems Division is in reality the same as
Percent Complete Estimates. Instead of estimating how much of the work already been done,
one estimates the amount of work still remaining. This gives the exact same result and is only
different in the approach of using the Performance Measurement Technique.
In both approaches it is possible to “lose” already earned value; however, I believe that it
would be greater probability for this to happen with the Naval Systems Divisions approach. It
would be contradictory for a manager to state one week that we have accomplished 40% of
the planned work and the next week decrease it to 35%. In the Naval Systems Divisions
approach, it is not very contradictory to state that one has realized that there is more work left
than what was expected the week earlier, although it has the exact same affect on the Earned
Value.
Is it good that the measurement technique makes it possible to lose Earned Value? On the one
hand it must be a good thing. Since the Earned Value obviously was not achieved one should
not be able to profit what one does not have. On the other hand the reports will be uncertain if
the Earned Value can fluctuate. Also, it must be annoying for the personnel to work in a
project where the progress is not certain. Further, if the value really has been achieved, then it
might be tragic that one can loose it because of complications in the remaining work. The
already earned value might in fact be valid and should not change at all.
75
Vestberg, Rasmus
Chapter 6 – Discussion
The measurement technique used at NSDS is very vulnerable to old plans. If the plan is not
running updated the method will equal traditional project management without EVM. In
traditional Percent Complete estimates bad updates will result in no Earned Value, why the
incentive to update the plans are probably higher in this method than in NSDS’s method. The
way of measuring the Earned Value has its advantages concerning scheduling of resources,
but I still do not think it is true Earned Value. Also, if the forecast is as bad as the Empirical
Study shows, then it is pointless to use it for resource allocations.
These circumstances emphasize the importance of having a measurement technique that one
can truly trust. And I believe, based on the flaws with the different Performance Measurement
Techniques, that the only ones with true Earned Value is Weighted Milestones and Fixed
Formula. If the work is done and can be measured, then the value is earned.
6.1.3 Level of Effort
Level of Effort makes it possible to have earned all value although there is still work
remaining. In fact the value can be earned before the work package even started, if the work
package has been moved in time. This is simply because at NSDS the “Baseline 10” does not
change even if the work does. It is only the baseline used for estimates that is affected of
increases or delays in the work package measured with Level of Effort. This means that it can
look like a work package is finished but there is still work to be done. This is obviously not
good.
The simple solution to this is to change the “Baseline 10” every time the LOE work package
is changed. Since LOE always should perform well this is the only solution if one wants to
use this measurement technique.
In the curves in Figure 10 some effects of NSDS’s wide use of Level of Effort is evident. Not
even when the work packages have earned 100% of its value the forecast is correct. This is
simply caused by the corruption made by Level of Effort measurement.
The average project at NSDS measures 19.7% of the work with LOE. The LOE work
packages are made up by overhead costs, such as support, quality, management, planning etc.
76
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
To reduce the amount of level of effort to be able to use EVM for real one could separate
those work packages and handle them as business overhead cost. This might not be the best
solution since different project might use those functions differently. Another way of dealing
with it is to use Apportioned Relationship or Percent Complete. The Percent Complete is not
recommended in ordinary cases, but it is still much more preferable than LOE. The
apportioned relationship is well suited as long as one could find related work packages to
connect the measurement to.
Note! Instead of only changing PMT for today’s LOE tasks the amount of LOE might work as
a warning signal. Is it really healthy to use 46.8% (the worst case at NSDS) of the time in a
project to management and support? Possibly the entire project resource allocation needs
revisiting.
6.2 Planning Package Conversion
It is a common fact that there is not possible to plan every work packages from the beginning
in a large scaled project. The gathered literature agrees that one should do the planning on
higher level and bye and bye do what is called a rolling planning. However there is no one
mentioning in what timeframe the detailed breakdown should be done. And perhaps most
important there is no discussion what so ever about how this rolling planning influence the
Earned Value.
6.2.1 Forecasting
The forecasting at Naval Systems Division is very simple thanks to the way of using Earned
Value calculation. In fact, forecasting using a revised plan is mentioned to be the most
reliable forecasting method of them all (3.2.2.5). However, this means that the Naval Systems
Division is using the forecasting method used in traditional project management instead of the
easy mathematical method provided by Earned Value Management. Since this feature is one
of the greatest advantages with Earned Value Management one loses some of the merits in
EVM by using this approach.
77
Vestberg, Rasmus
Chapter 6 – Discussion
It might be that the estimate from a Project Leader or a Program Project Leader is more valid
than the estimate made using performance indexes. However, as earlier spoken, the way of
measuring Earned Value at Naval Systems Division is not real Earned Value. The
Performance Measurement Technique will need to be revised. If the constant updating of the
plan is no longer needed to generate the Earned Value this work would only be done based on
forecasting needs. That means we are back in the choice between the time-consuming and
accurate classic forecasting method and the almost as reliable but very easy EVM forecasting
method (more reliable in our case). Instead of as today, when one gets the classic forecasting
method on top of the not fully implemented Earned Value Management System.
In the empirical study the forecasting relative the entire work package is far better than the
one relative the remaining work. Which one should we then pay attention to? It would be the
same thing to predict the entire work package as predicting the remaining of the work
package. However I believe that the mind can be tricked to do different predictions based on
the way the question is put. No matter if the results would be different or not the fact remains;
at Naval Systems Division the question is always “How much work remains?” This means
that there is no excuse for not looking at the predictions based on the remaining work. Hence
we can conclude that the predictions of today are very poor.
If we use this approach to determine the time frame for conversion of planning packages, it is
important that the remaining of a work package can be equalled with a smaller work package.
One could argue that the remaining should not be equalled with a smaller work package, since
there could be things that one states as finished although their not. That would mean that there
is more work left to do than expected. There are two obvious counter-arguments for this
statement. First of all, if we are not certain on the Earned Value achieved we can seriously
question the entire method. Second, the same thing could be valid for smaller work packages,
the inputs from earlier work packages are not correct and there will be more work than
expected.
The reason for using NSDS forecasting method is based on the necessity to have updated
plans in order to make correct resource allocation. One might ask one self this question; is it
likely that the Project Leader or the Program Project Leader will perform better in updating
the far term plans than they did with the nearby work packages? If not, the index of how the
78
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
nearby work packages fall out might be at least as good as the subjective judgment. This line
of argument must also support the idea that it is possible to do long term resource planning
based on performance index. If the index indicates that one needs 1.25 hours for every
planned hour then that would be enough information for the resource planning. At least as
long as we agree in that the performance indexes are good enough to replace the updated
plans concerning estimates and long term planning.
The big difference in the Empirical Study between the forecast and the Actual at Completion
in some work packages can be explained in several ways. It could be that the person in charge
has very bad control over the entire work package. More likely is that some individuals prefer
to purposely overestimate the remaining work to be able to under run in the end. For them it is
more important to look good in the end than to have a realistic plan during the work. This is
confirmed likely based on the curves in Figure 11. All the work packages that first at
completion have a realistic picture are subject for under runs. Large overestimates are a larger
problem than one might think. This leads to higher prize to the customer and perhaps loss of
business.
It is interesting to see that the easy CPI method provides better accuracy than the strenuous
method used at NSDS today. It should also be mentioned at this point that the figures used to
generate the Cumulative CPI forecast is not true Earned Value as stated earlier. The fact that
the subjective judgment provides insecure figures strengthens the theory that there is not real
Earned Value measured when not using measurable defined work.
6.2.2 Rolling planning’s affect on EVM
The “Projekthandbok” at NSDS indicates that there is no reason to do detail planning far in
advance. Then they are not taking into consideration the affect that detailed plan might have
on the EV curves. Neither do one care about the correct figures that might be needed for
decisions concerning the project.
The baseline for the entire project (the core of EV) must be set in the beginning, although the
later part of the project is only planned briefly. As said in the Earned Value Project
79
Vestberg, Rasmus
Chapter 6 – Discussion
Management88 the work package is what gives the baseline. How can one then determine the
later baseline when one has not done the breakdown into work packages yet? The answer is at
first sight pretty simple, one lets the planning packages generate the baseline! However, at
second thought one realizes this way of solving the problem has a flaw. The first part of the
baseline is composed by the work packages in detailed plan, but the second part originates in
les detailed plans and assumptions. Can one then really use the CPI and SPI that has been
calculated based on one type of figures and then apply it to a different kind of figures?
According to DoD and their massive research this seams to work.
To solve this problem I would suggest NSDS to do planning at work package level for the
entire project from the beginning. This is in somewhat controversial but can be supported by
some basic arguments.
First of all the empirical study does not show any signs indicating that people at NSDS do
better plans concerning the near future than far term work packages. Approximately one third
of the studied work packages are probably flawed by deliberate overestimates. However, the
rest of the studied work packages do not show any better trends.
Breakdown of a planning package tends to consist of adjusting the work packages to sum up
the original planning package. This means that the later planned work packages are of the
same granulate as the respective planning package.
Planning on a higher level is likely to provide overestimates because of the safety margin put
in the plan.
Most of the work in the project at NSDS is not research and development why one should
have pretty good knowledge about what to do. If you know how to do it, why not plan it. If
one chooses to do detailed plan one might not need to schedule exactly when it should be
done. But what and at what cost should not be a problem. The difference in the EVM-curves
will then be only according time and not money. That is, the CPI is still ok but the SPI might
show bad figures.
88
Fleming and Koppelman (2005)
80
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
If one anyway chooses to do rolling planning I believe it might be a good idea to be a bit more
flexible about the baseline. The beginning of the baseline is made up by work packages. The
beginning of the baseline is what makes the CPI and SPI which is why those also are based on
figures from work package level. If the later part of the baseline is made up by planning
packages one would combine data from different origin. This is why I suggest that one should
not be too afraid of changing baseline when planning packages is broken down.
6.3 Miscellaneous
This part will discuss findings outside the original problem.
6.3.1 Earned Scheduling
The problem with SV($), shown in Figure 8, further support the idea that the SV($) is not to
recommend. The case in Figure 8 is based on a perfect performance only starting one month
late. It is obvious that the schedule variance should be one month over the projects entire life
cycle. One can argue that the slippage is not constant based on the amount of money needed
to retain the original schedule. As it looks in Figure 8 there would be much less money
needed to catch up with the baseline in the beginning and in the end of the project. This would
indicate that the Schedule Variance should not be constant anyway, and that the SV($) is the
right way of measuring Schedule Variance. However, there is a reason why most projects
baseline is S-shaped. In the beginning of the project the activities consists of planning,
scoping, specifying requirements and so on. In the end of the project the task consists of
testing, delivery, installation etcetera. In neither of the case it would be possible to double the
output by doubling the resource. As an example, a meeting with the buyer is not likely to take
half the time if one doubles the people participating.
SV(t) can if wanted be recalculated to $ again. It might sound a little odd to first replace the
measurement based on money with a measurement based on time, and then convert the time
into money again. Still it will not be the same thing. The SV($) is the vertical difference
between baseline and performance while the SV(t) is the horizontal difference. Converting the
SV(t) into money will not change this difference between the two approaches, only display
the variance in another unit. I would not recommend this conversion since it only complicates
81
Vestberg, Rasmus
Chapter 6 – Discussion
a well working easy technique. The reason for doing this anyway would be if one must
display the figures in the same unit.
The more intuitive approach of the Schedule Variance combined with the less defective result
compared to the SV($) makes this method an obvious choice. Changing into Earned
Scheduling would not mean any revision of the plans or anything. The changes do only
comprise the way of calculating the variance when computing the Earned Value figures.
Unfortunately the system used at NSDS to manage Earned Value does not support Earned
Scheduling. However, all the data needed to calculate Earned Schedule is available in the
system and the only thing needed is the implementation of a new formula. Based on the
insignificant change in the system (probably only some few lines of code) and the size of
NSDS as a customer to the system provider, would make it possible to do this implementation
instantly.
6.3.2 Earned Value Acceptance
The mutual understanding at NSDS is that Earned Value Management is something bright
new, which it is not, not even for NSDS. The main concept of EVM has been present at
NSDS since the beginning of the eighties, only under different names. The name of the
concept is insignificant, the important thing is that people at NSDS do not seem to like it or
know anything about it. Perhaps it is not even necessary to call the measurement technique by
some specific name. The important thing is the figure provided from the system. By using a
simple method as Milestone Measurement and the basics of Earned Scheduling the functions
are well covered.
Using several milestones in each work package makes it easier to really determine the status
of the project, in the same time they serve the purpose of engaging the staff. Having frequent
milestone will visualize the progress of the work.
82
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
83
Vestberg, Rasmus
Chapter 7 – Conclusions
7 Conclusions
This chapter will in condensed form present the conclusions and suggestions for adaptations
at NSDS. The Empirical study is because of the quantity not statistical in any way. However,
the suggestions would be valid anyway based only on the Frame of Reference. The small
quantity of research does nevertheless support all conclusions.
7.1 Planning Package Conversion
The empirical study can be interpreted in two ways; either one do not have to plan at all or we
can plan the entire work at work package level from the beginning. Since the first option is
unthinkable this must mean that we can plan all work packages in the start of the project. For
those cases where there are impossible to plan at a more detailed level than planning packages
the later breakdown must be followed by a consideration of a baseline change. This
consideration of baseline change is direct opposite to recommendations made by PMI.
However I think it is wrong to measure against a baseline consisting of careless high level
plans made perhaps several years ago. If one is more admissive with changes in the baseline it
is important that the changes are made as early as possible. If the changes are made to late the
affect will be perfect performance.
7.2 PMT
The only true Earned Value is the objective measurable value. Therefore the Measurement
Technique to be used must be Weighted Milestones or Fixed Formula. This should be used
with very small work packages or larger work packages divided into several measurable
milestones.
All Level of Effort task must be separated from the project baseline and measured separately.
In cases where separation is not possible the work most preferably should be measured with
Apportioned Relationship and secondary Percent Complete. This will provide a measurement
on those tasks which is not present at all today.
84
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
7.3 Forecasting
The forecasting should be made as described in 3.2.2.5 with some of the mathematical
approaches preferably using all three together as described. The method is easy and provides
data good enough (and some times even better than today). Changing this routine is easy and
could be made instantly without any revision of the plans.
7.4 Miscellaneous
Today I would say that NSDS is not using Earned Value at all. This is mainly based on the
very high level of LOE measurement. Also the shocking reliability in the subjective Earned
Value measurement supports that point of view. One would probably provide more accurate
Earned Value by guessing. Even if other projects than the measured would be more accurate I
still would not say that NSDS is measuring true Earned Value. Using a subjective method will
in my mind never provide assured information. Thinking about it, measuring the performance
with subjective method is what has always been done in traditional project management. The
only difference is that in this case it is taken to a lower level and displayed in graphs.
By adapting this recommendations there will most likely be several benefits. The work with
Earned Value Management will be less. The data provided will be more reliable and therefore
useful. Valid data will most likely yield a much wider acceptance for the system. Detailed
measurable milestones will make it more difficult to strongly overestimate the work. If there
still are large overestimates they will be noticed early when a couple of milestones has been
achieved.
The cause of the extreme portion of LOE should be further investigated. Not necessarily, but
it could indicate bad use of resources. If half of the LOE tasks are misuse of resources (the
amount of LOE is twice the maximum recommended) there is 10% of the total project budget
to save.
Doing the changes in forecasting would mean that one could no longer use the same data as
above to determine the time frame for planning package breakdown. Instead, if adopting all
purposed changes, one could measure the accuracy against the original plan, since all work
85
Vestberg, Rasmus
Chapter 7 – Conclusions
packages is recommended to be planned right away at project start. If there is a special time
frame telling how far in advance it is possible to plan, this will be obvious.
Measuring the accuracy in forecasting should be made running in the same way as in this
report. This provides a good indication of how well the used techniques work, both for
planning and measuring of Earned Value.
86
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
87
Vestberg, Rasmus
Chapter 8 – References
8 References
Books
Cassell, C. and G. Symon (ed.) (2004) Essential Guide to Qualitative Methods in
Organizational Research. London. SAGE Publications Ltd
Fleming Q. and Koppelman J. (2005), Earned Value Project Management, third edition,
Pennsylvania, Project Management Institute, Inc.
Lekvall, P. snd Wahlbin, C. (2001). Information för marknadsföringsbeslut – 4:e
upplagan. Göteborg: IHM Publishing.
Patel R. and Davidsson B. (2003), Forskningsmetodikens grunder, 2nd edition,
Studentlitteratur, Lund
Wenell T (2004), Wenell om projekt, Uppsala, Uppsala Publishing House AB
(2005), A Guide to the Project Management Body of Knowledge: PMBOK guide. – 3rd ed.,
Pennsylvania, Project Management Institute, Inc.
Articles
Abba Wayne (2002), June 2002, Emerging Ideas, Relating EVM to “Real” Schedules,
unpublished presentation
National Contract Management Journal (1993), Spring 1993, Cost Performance Index
Stability, Christensen, D. S., Heise, S.
Acquisition Review Quarterly (1998), Fall 1998, The Cost and Benefits of the Earned Value
Management Process, Christensen David S. Ph.D.
Acquisition Review Quarterly (1999), Summer 1999, Using The Earned Value Cost
Management Report to Evaluate the Contractor’s Estimate at Completion, Christensen David
S, Ph.D.
88
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
The Measurable News (2007), Winter 2006-2007, Why EVM Is Not Good for Schedule
Performance Analyses, Corovic Radenko
The Measurable News (2003), Summer 2003, A Breakthrough Extension to Earned Value
Theory?, Henderson Kym
The Measurable News (2004), Spring 2004, Further Developments in Earned Schedule,
Henderson Kym
The Measurable News (2005), Spring 2005, Earned Schedule in Action, Henderson Kym
The Measurable News (2007), Summer 2007, Earned Value Business Solutions:
Implementation, Lessons Learned and Best Practices, Kondur Mohan
The Measurable News (2003), Mars 2003, Schedule Is Different, Lipke Walt
Proceedings of the 3rd International Conference on Project Management (2006), September
2006, Earned Schedule Leads to Improved Forecasting, Lipke Walt
Interviews
Person A, Manager Naval Operations Project Office
Henriks Yngve, Manager Project Management Office, Naval Systems Division Sweden
Karlsson Ulf, Project Planer, Naval Systems Division Sweden
Larsson Stefan, Senior Project Planner, Naval Systems Division Sweden
Lindberg Björn, Controller, Naval Systems Division Sweden
Olsson Henrik, Project Manager, Naval Systems Division Sweden
89
Vestberg, Rasmus
Chapter 8 – References
Trochez Michael, Head for the Planning and Control Function, Naval Systems Division
Sweden
Internet
http://saabnet.saabgroup.com/SaabSystems/About/index_sv.htm?NavigationType=SiteNaviga
tionDataSource, Accessed June 2007
https://www.goldpractices.com/practices/tev/index.php, Accessed September 2007
Others
ANSI/EIA-748-A-1998, Arlington, Electronic Industries Alliance
Averstad D (2003), Projektstyrning med Earned Value – en metod under FMV
uppdragsledningsprocess, Försvarets Materialverk
Projekthandbok, SAAB Systems, serie av interna dokument rörande projektmetodik,
Uppdaterad 2002-2003
90
Earned Value Management at
Vestberg, Rasmus
Saab Naval Systems Division Sweden
91
Vestberg, Rasmus
Chapter 9 – Appendices
9 Appendices
9.1 Appendix 1
The chart displays the analyzed work package’s monthly figures.
92