T15
Concurrent Session
Thursday 10/02/2008 1:30 PM – 2:30 PM
Test Management for Very Large Programs:
A Survival Kit
Presented by:
Graham Thomas
Independent Consultant
Presented at:
STARWEST 2008
September 29 – October 3, 2008, Anaheim, CA, USA
330 Corporate Way, Suite 300, Orange Park, FL 32043
888-268-8770  904-278-0524  [email protected]  www.sqe.com
Graham Thomas
Graham Thomas wrote his first computer program at college in 1978 and started working in IT in the early 1980s as a programmer,
and discovered software testing in the early 1990s. He has formal qualifications in programming, analysis and design, project
management, and software testing. He worked for a large consultancy, several smaller management consultancies, and also for a
systems house, as well as various end users. He has a wide ranging experience of IT, development and software testing, covering
the public sector, retail, finance, banking insurance, and treasury. Currently he works as either a program test manager or
implementing testing change. Prior to this he worked as a test manager.
Test Management for Very Large Programs
A Survival Kit
Graham Thomas
Independent Software Testing Consultant
STARWEST, Anahiem, October 2008
ABSTRACT
In large organizations with multiple, simultaneous, and related projects, how do you coordinate testing efforts for better utilization and higher quality? Some organizations have opened Program Test Management offices to oversee the multiple streams of testing projects and activities, each with its own test manager. Should the Program Test Manager be an über‐manager in control of everything, or is this office more of an aggregation and reporting function? Graham Thomas examines the spectrum of possible duties and powers of this position. He also shares the critical factors for successful program test management, including oversight of the testing products and deliverables; matrix management of test managers; stakeholder, milestone, resource, and dependency management; and the softer but vital skills of influence and negotiation with very senior managers. Relating experience gained on several large testing programs, Graham shares a practical model—covering the key test management areas of organization, people, process, tools, and metrics—that your organization can adapt for its needs.
© Graham Thomas 2008
2
AGENDA
y
y
y
y
y
Introduction
Definitions
Role
Attributes & Skill Set
Survival Kit
o
o
o
o
o
Organisation
People
Process
Tools
Metrics
y Critical Success Factors
y Lessons Learnt
y Summary
© Graham Thomas 2008
3
INTRODUCTION
y Why Program Test Management?
o Large Organisations
o Big Programs of work
o Many inter‐related projects and programs
y Why Program Test Managers?
o To represent testing at the program level
o To manage multiple streams of testing projects and activities
o At a level which is outside the sphere of influence for individual projects or testing activities
© Graham Thomas 2008
4
DEFINITION
TEST MANAGEMENT
“The direct management of testing activities”
• planning
• execution
• analysis
• validation
• design
• reporting
• preparation
© Graham Thomas 2008
5
DEFINITION
PROGRAM MANAGEMENT
y What is a Program?
“A broad effort encompassing a number of projects and/or functional activities with a common purpose.”
Association for Project Managers - Web site Definitions
y What is Program Management?
“The management of multiple streams of development activities (including testing), each with their own teams carrying out detailed planning, management and execution activities.”
© Graham Thomas 2008
6
DEFINITION
PROGRAM TEST MANAGEMENT
“The management of multiple streams of testing, each with their own test management in place which is responsible for detailed planning, management and control activities.”
© Graham Thomas 2008
7
PTM ROLE CONTINUUM
y Über Test Manager
o
o
o
o
o
Micro‐management
Disempowering
Decision making bottleneck
Stifle creativity
Perpetuates hero culture
y Aggregating View
o
o
o
o
o
Detached and remote
Spy in the camp
Collation and aggregation
Viewed as overhead
No ‘test’ value add
© Graham Thomas 2008
8
IDEAL PTM ROLE RESPONSIBILITIES
y PTMs are not responsible for managing the testing activities. That is what we have Test Managers for!
y So, what are PTMs responsible for if they are not managing the testing?
y Responsibilities vary from program to program:
♦
♦
♦
Testing oversight
Analogous to Program Management
Looking for gaps across the program
♦
♦
© Graham Thomas 2008
Getting the development groups to work together with the testers
Defining and managing the program testing approach
9
ATTRIBUTES OF A GOOD PTM
y Honesty, Integrity, Experience, Credibility
y Pro‐active, (Dynamic ‐ if you think rushing round helps without causing panic)
y Politically aware and astute
y Focussed on the program and organisational goals
y Independence
© Graham Thomas 2008
10
ESSENTIAL SKILL SET
y In addition to:
o Good test management
o Good project management skills
y Stakeholder management
o Influence & negotiation skills
y Program Management
o
o
o
o
Milestone management
Resource & Dependency management
Delivery through others
Oversight and awareness
y Motivate and enthuse
© Graham Thomas 2008
11
SURVIVAL TOOLKIT
y 3 Good Things
y Swiss Army Knife ‐ That tool that no‐one knows what it is for!
y What to avoid doing, is as important to know, as what to do
© Graham Thomas 2008
12
CIO
ORGANISATION
PO
Bus.
Dev.
Test
BEST PRACTISE
n Define clear test organisation structure with matrixed
relationships
o Clear and agreed interface with stakeholders and sponsors
p Maintain independent reporting for testing
ANTI‐PRACTISE
y Testing only reports into delivery
y Testing perceived as secondary activity
© Graham Thomas 2008
13
PEOPLE
BEST PRACTISE
n Ideally want people who want to be testers
o Identify your best people and give them the right opportunities
p Create a collaborative environment to share test knowledge and experience
ANTI‐PRACTISE
y Unclear testing roles
y Lack of training and investment in testers
© Graham Thomas 2008
14
PROCESS
BEST PRACTISE
n Align development and testing activities
o Consistent processes ‐ standards, methods and approach
p Fit for purpose, flexible and effective
ANTI‐PRACTISE
y Testing in silos – UT, ST, SIT, UAT, OAT etc.
y Quality police
© Graham Thomas 2008
15
TOOLS
BEST PRACTISE
n You will get as big a benefit from common process
o Match tools and processes e.g. Test Planning
p Not just automation ‐ but Config, Release & Change Mgmt
ANTI‐PRACTISE
y Developers get tools, testers don’t
y You have Quality Centre, what more can you want?
y No Silver bullets
© Graham Thomas 2008
16
METRICS
BEST PRACTISE
n Align your measures and metrics to your stakeholders and sponsors needs o Keep your measures and metrics consistent
p Trend measures are very powerful
ANTI‐PRACTISE
y If it moves measure it!
y Inappropriate use of metrics can drive blame culture
© Graham Thomas 2008
17
CRITICAL SUCCESS FACTORS
y To ensure that you put in place a testing organisation that meets the testing needs of the program
y To ensure that you have an efficient and effective testing process that supports and integrates with your development approach
y To ensure that your stakeholders and sponsors clearly understand what testing is doing for them
y To be able to articulate the program testing message
© Graham Thomas 2008
18
SUMMARY & QUESTIONS
y
y
y
y
y
Defined the Term Program Test Management
Examined the PTM role continuum
Looked at responsibilities, attributes and skills
Worked through a model for Program Test Management
Identified Critical Success Factors
y Any Questions © Graham Thomas 2008
19
CONTACT DETAILS
Graham Thomas
Independent Software
Testing Consultant
[email protected]
+44 7973 387 853
Þ www.badgerscroft.com
© Graham Thomas 2008
20