Leverage Application Development Metrics to
Gauge Progress & Increase Success
Move forward with the lights on!
Info-Tech Research Group, Inc. Is a global leader in providing IT research and advice.
Info-Tech’s products and services combine actionable insight and relevant advice with
Info-Tech's products
and services
combine
actionable
insight
with ready-to-use tools
ready-to-use
tools and
templates
that cover
theand
full relevant
spectrumadvice
of IT concerns.
and templates that cover the full spectrum©of
IT concerns.©
1997Research
- 2012 Info-Tech
Research Group
1997-2012
Info-Tech
Group Inc.
Info-Tech Research Group
1
Introduction
Set goals & measure trends with effective metrics.
There is an old maxim that says, you can only manage what you can measure, and this directly applies to
application development exercises.
Metrics are necessary in order to understand & improve: product performance, project performance, and
process performance.
Use this research to identify appropriate development metrics for your enterprise. It will help you create a
meaningful set of metrics and introduce how these metrics promote improvements in your development
process while maintaining alignment with both your business and technical goals.
Our goal is to prepare you for reporting with meaningful and useful development metrics that you
will be able to collect.
This Research Is Designed For:
This Research Will Help You:
 Development leaders and development team
 Introduce key metrics.
members.
 Executives and line-of-business managers
tasked with achieving goals and improving
processes.
 Identify how different metrics address
trending and process improvements.
 Introduce practical metrics, implementation
methods, benefits, and risks.
Info-Tech Research Group
2
Executive Summary
Common Wisdom says:
• Tracking various aspects of development, including product, project, and even resources is important
and should be done.
Common Practice suggests:
• Development metrics are a large waste of time and effort. They are too difficult to collect and too
subjective to be useful.
• Project related metrics in the form of estimate vs. actual will provide learning, which will enable teams
to adjust for future projects.
• Tracking of project deliverables as they hit dates and budgets and conducting post project reviews,
results in acceptable learning being achieved.
• Peer assessments and close working environments provide sufficient insight to evaluate, improve,
assign, and review development resources.
We Believe:
• That it is long overdue and time to measure app dev & maintenance in the same manner as the rest
of the business.
• That common practice is wrong!
• That common practice produces little to no real change.
• That common practice produces information that is transient and quickly forgotten.
• Tracking and recording metrics do produce productivity gains and benefits. Trends can be set and
measured, decisions can be made based on tangible results, and resources can be allocated to
projects where their talents and skills can be of best use through effective metrics collection.
Info-Tech Research Group
3
Metrics have strategic value to Application
Development Shops
What’s in this Section:
•
•
•
•
•
Understanding the strategy.
Collecting & using metrics.
People: the weakest link.
Holistically thinking.
Continuous improvement.
Sections:
Metrics have strategic value to
Application Development Shops.
Many App Dev Shops struggle
balancing the costs & benefits of
metrics collection.
Optimize for quality &
performance via your metrics.
App Dev metrics provide a basis
for the validation of development
exercises and processes.
Info-Tech Research Group
4
Just like the rest of the business, development metrics are
needed to identify how much is done & how well it is done
Checking code to see if it works as expected is only a single measurement.
Understanding & knowing:
•
•
•
•
How easy it will be to maintain over time.
How flexible it will be to add to, or change as the business requires.
How easily and quickly new developers can get up to speed with the existing code and begin working
on it.
How easily the software can be tested.
All are just some of the measurements needed to truly understand the quality of work being done by
development teams.
When these measurements are combined with other methods of counting the productivity of the
development team, development leaders can begin to identify trends and formulate a true and objective
understanding of how well the development team is performing. This allows the business and stakeholders
to finally see a return for their business investment.
All of the measurements in this solution can be used for both in-house, as well as outsourced
development work.
Info-Tech Research Group
5
Development measurement typically revolves around
superficial & irrelevant metrics, yet the output is key to the
business
It stands to reason then that this group should be managed, controlled, and
measured like a core business process and be accountable to the highest
corporate standards regardless of where the work is done!
Without meaningful metrics, estimation of release dates and the general quality of the
product delivered will continue to be highly subjective.
Common Practice looks at:
• past project deliveries
• requirements met with each release
• defect counts
Does nothing to tell your business
how hard it is to make a future
change, or what the real internal
code quality is.
Common Practice assumes that:
• Measuring development is difficult and time consuming and just can’t be done, however…
In order to assist development organizations, numerous international bodies have been studying methods for objectively
measuring software quality and quantity in an attempt to make clear the methods and procedures for tracking & measuring
development. These groups include (to name a few):
• Software Engineering Institute (SEI)
• International Standards Organization (ISO)
• Institute of Electrical and Electronics Engineers (IEEE)
Common practice amounts to educated guessing by development managers which leaves executives
wondering about what their investment is truly returning to the business.
Info-Tech Research Group
6
Application Development groups should take a close look at
baseball to understand the benefits of tracking metrics
If you ever watched a baseball game, even once, you have heard the terms batting average
and earned run average. Statistics are used by professional baseball teams to prepare for
games by developing a plan for both pitching to, and hitting against, their opponents.
Managers use statistics about their own players during a game to decide the order
of the lineup, what hitters to use, or which pitcher to bring in, and the general
managers and scouts use statistics to make decisions on the abilities of players,
who to draft, and who to release.
The statistics available in baseball are a consolidated collection of all the
observations as opposed to the observations that may be available from a single
source. This provides a much larger collection of data from which to draw conclusions.
Similarly
Application development groups need metrics (aka statistics) to help determine exactly
the same things:
•
•
•
•
Who is best for a given project?
Who is most likely to succeed in a given situation?
Is the business producing the best it can, efficiently?
What skill sets does the business require to ensure a successful outcome?
Objective, collective metrics provide meaningful and useful knowledge to decision maker.
Info-Tech Research Group
7
Management 101 says: You can’t manage what you can’t
measure & you can’t measure what you can’t see
Metrics that capture information about overall health and what is happening
in development, provide the visibility needed to better manage & measure
application development
Complexity in application development cannot be avoided. Development groups must have a goal to reduce, or
limit the complexity of the code they write, wherever possible. This is where the tracking and measuring of
development metrics and the overall application health come into play.
Development Metrics
•
•
•
•
•
•
•
•
•
•
•
•
Industry standards
Coding specific standards
Language specific standards
Previous research/metrics
Business best practices
Naming conventions
Code complexity
Functions
Inheritances
Comments/documentation
Procedures
and more…
• Use the actual development metrics to add value on their own to
the individuals architecting and building the applications.
• Gain more real power by using the technical information gathered
to determine application health factors (see section 3 of this
solution).
• Use the application health factors to provide a roll-up of
development metrics that look at how the code will affect the
business.
• Report these factors with different weights to provide upper
management a way to measure what the teams have done to the
applications and give them an understanding on what will happen
in the future.
• Remember you cannot have the application health factors without
the development metrics as the development metrics are used to
determine the application health factors.
Info-Tech Research Group
8
The problem is not with the metrics, the problem is with
the time, the people, and the process
“My developers know more than I do, so I
trust what they tell me. My clients don’t
always like what I have to tell them, but
what else can I do?”
Software Development Manager– Manufacturing Sector
What we learned talking to clients:
• Many development shops either do no
tracking, or fail to collect meaningful metrics
opting for the easier more superficial
tracking of on-time, on-budget, and within
scope.
• Project & Development managers rely on
“the walkabout”: moving from team member
to team member asking “how’s it goin’?”
Avoid questionable and superficial metrics. They lead to more questions than answers. It takes courage
to slow down and collect meaningful metrics, but well worth it.
Info-Tech Research Group
9
Pick a number between 1 & 10… ok Mr. Executive,
this is how well we are doing in development… or not
Common Practice creates a problem…
• Many development shops base their “statistics” on personal
observation, perception, or gut-feel instinct. While this may be
correct, it will likely be more coincidental than objectively true.
• Without an impartial analysis about the capacity and
performance of the development group, management is
unable to make informed decisions for the business.
• Instinct, gut-feel, and perception makes for precarious
business decisions; running a business cannot be done on
emotion.
• Business executives, owners, investors, and other
stakeholders must make decisions based on real facts. Your
instinct, or gut-feel just won’t stand up to scrutiny. Executives
may not share your technical knowledge and expertise, and
will not accept, that while you believe you have the “right”
people, their project is late because the work is hard!
“I know how well they
are doing. I know
which developers are
better or worse
because I stay involved
and have my finger on
what they are doing
plus I get feedback
from their peers when
necessary.”
CIO – Service Industry
Info-Tech Research Group
10
Four rules, three levels; metrics are all about making sense of
the numbers, but look at them holistically
In order to establish a good metrics program,
follow Info-Tech’s 4 cardinal rules:
1.
2.
3.
4.
Keep them simple.
Keep them to a minimum.
Base them on business objectives.
Keep them practical (i.e. focus on Time, Cost,
Performance, and Resources).
Collect metrics at three levels:
1.
2.
3.
Pre-project (sometimes called portfolio metrics).
During (in-flight).
Post-project (at the end of the project).
Additional tips:
• When just starting out with metrics, avoid collecting so much data that it will overwhelm the still
elementary capacity to organize and analyze it (3-5 metrics is a good starting point).
• Focus your use of metrics on positive improvement and not negative or punitive action. If staff get a
hint that metrics are being used against them, they will circumvent and hinder the program.
Individually, metrics are of little to no use (to the business), but when combined and viewed holistically,
the leaders can gain objective visibility on the development health.
Info-Tech Research Group
11
Info-Tech Research Group Helps IT Professionals To:



Quickly get up to speed
with new technologies

Manage business expectations

Justify IT spending and
prove the value of IT

Train IT staff and effectively
manage an IT department
Make the right technology
purchasing decisions – fast
Deliver critical IT
projects, on time and
within budget
Sign up for free trial membership to get practical
solutions for your IT challenges
“Info-Tech helps me to be proactive instead of reactive –
a cardinal rule in a stable and leading edge IT environment.
•
- ARCS Commercial Mortgage Co., LP
Toll Free: 1-888-670-8889
www.infotech.com
Info-Tech Research Group
12