Visual Studio Agile Deployment Assessment
Deliverable Template
Developer Tools Deployment Planning Services
Page i
Visual Studio Agility Plan Template, Version 1, 2013.10
1
Getting the Most from Your Agile Plan ....................................................................... 2
1.1
2
Give Microsoft Feedback ........................................................................................................ 2
Executive Summary ................................................................................................... 3
2.1
Background ............................................................................................................................. 4
2.2
Current Situation ..................................................................................................................... 5
2.3
Bottlenecks ............................................................................................................................. 5
2.4
Recommendations .................................................................................................................. 5
2.5
Agile Practices Summary........................................................................................................ 6
2.6
Existing Best Practices ........................................................................................................... 7
2.7
Existing Hindering Practices ................................................................................................... 7
2.8
Key Areas for Improvement .................................................................................................... 8
2.8.1
Current State – Urgent Issues ............................................................................................ 8
2.8.2
Current State - Additional Issues ........................................................................................ 9
3
Roadmap to Agile Maturity ...................................................................................... 10
4
Agile Maturity Model............................................................................................... 11
5
Current vs. Ideal State.............................................................................................. 12
6
Roadmap ................................................................................................................. 13
First Iteration.................................................................................................................................. 13
Second iteration ............................................................................................................................ 15
7
Detailed findings on Agile Planning .......................................................................... 17
8
Resources ................................................................................................................ 22
9
Conclusion ............................................................................................................... 24
Page ii
Visual Studio Agility Plan Template, Version 1, 2013.10
1
Getting the Most from Your Agile Plan
Guidance: This template includes guidance blocks and wording examples. Prior to handing over the
document, remove the guidance blocks (like this one) and replace any highlighted sample text in
<brackets> with your findings and recommendations.
Our recommendations for optimizing the agile practices and tools in your environment are detailed
within this document. Please take your time to review the findings and ask any follow-up questions
necessary. Depending on the capabilities of your teams and organization, you may elect to try the Agile
improvements in-house or contract with an outside consultant. In either case, this plan should be given
to the party responsible for the work and used as an implementation guide.
1.1
Give Microsoft Feedback
This Planning Service has been provided as part of your Microsoft Software Assurance benefits. Please
use the link below to tell Microsoft about your experience with the engagement and any improvements
you would like to see made to it. The results of the survey will only be viewed by the Planning Services
team at Microsoft.
http://www.surveymonkey.com/s/dtdps_cs
Page ii
Visual Studio Agility Plan Template, Version 1, 2013.10
2
Executive Summary
Guidance: The audience for this section will be interested in being able to read and digest this
quickly. Keep the text in this section concise.
At the request of <Customer Name>, <Partner name> conducted Agile Planning Services with the
following objectives:








Understand the current software development process
Elicit the desired objectives for an agile implementation
Document existing Application Lifecycle Management (ALM) topology
Create a baseline measurement of the current Agile tools and capability
Uncover opportunities for improvement
Identify the most impactful areas to the business
Document ideal end-state for teams.
Generate and present a roadmap to <implement> <improve> Agile process using the Team
Foundation Server agile tools
The Application Lifecycle Management (ALM) model was used as a framework to develop a vision and
sustainable approach by which <Customer Name> can prioritize business investments that fuel business
growth. The engagement focused on understanding existing development and best agile processes and
recommending improvements. Technology, practices and people/knowledge requirements were then
identified to support the process.
Page iii
Visual Studio Agility Plan Template, Version 1, 2013.10
2.1
Background
The following issues with the current development capability were articulated at the start of the
assessment.
Guidance: https://www.scrum.org/Agility-Path





Quality issues
Team Communication
Project Visibility
Projects delivered cycles
Feature Requirements
The following business priorities were articulated at the start of the assessment.
Guidance: Agile Development
Application
Lifecycle
 Improve
qualityManagement with Visual Studio and Team Foundation Server
 Improve customer satisfaction
 Improve cycle time
The following Tools and Processes (Strategy) were articulated at the start of the assessment
Guidance: Application Lifecycle Management with Visual Studio and Team Foundation Server
Application
Lifecycle
Management
with Visual Studio and Team Foundation Server
 Usages
of current
agile tools
 Challenges that exist in the agile tools
 Knowledge of the available Team Foundation tools and practices
Page iv
Visual Studio Agility Plan Template, Version 1, 2013.10
2.2
Current Situation
Guidance: Briefly summarize the organization’s current situation and associated top issues. Identify a
high-level solution.
<Customer Name> is experiencing growth with new projects and teams. The main challenges they are
facing are due to inconsistences within the software development cycle. Having a reliable and proven
workflow will help <Customer Name> reduce the following issues that were captured during the
interview process.



2.3
There are inconsistences and unreliable software in place to meet customer needs
There is not a clear and understandable workflow between teams
There is unclear transparency and requirements
Bottlenecks
Guidance: Call out what bottleneck(s) might hinder an adoption of agile practices
During the interview process at <Customer Name> a few bottlenecks arose that would impact an Agile
adoption. Some of these bottlenecks are due to <Customer Name> requirements and external
constraints, while others are due to team or organizational issue that can be addressed in the following
ways:



2.4
<Customer Name> needs to have a high security model and workflow as the ready code moves
through the environment. This requirement however does slow the delivery and testing
practices as teams have a challenge accessing the environment.
<Customer Name> has to wait for external vendors for feedback. This slows the team’s abilities
to complete the story.
<Customer Name> does not have an automated software delivery pipeline to reduce the
manual and error prone deployment.
Recommendations
There are three main areas focus for the team to improve their software development practice. Please
see the Roadmap section below for a more detailed outline.



Improve Agile practices across all teams and business owners
Improve quality by defining the story, task, and definition of done
Improve team cycle time to meet the customer needs
Page v
Visual Studio Agility Plan Template, Version 1, 2013.10
2.5
Agile Practices Summary
Guidance: Call out what is the main practices that is being used. Scrum, Lean, Kanban, etc…
Software development becomes more complex to manage and create with an increasing number of
uncertainties. More companies are moving to empirical models to improve the economics of software
s
development cycles. All of these models have a similar theme, called the Agile Conesus.
 Flow of value, where the customer defines value
 Continual reduction of waste impeding flow
Application
Lifecycle enabling
Management
Visual to
Studio
and Team
Foundation
Server
 Transparency,
team’swith
members
continually
improve
the above
two
Agile is a practice that helps organizations reach their full potential. It fosters Individuals and team
interactions, delivering working software, customer collaboration, and responding to changes. The 12
principles outlined in the Manifesto for Agile Software Development guide display the core values of
Agile practices.
There is not a “one size fits all” to Agile practices. The Agile practice does use many frameworks to
achieve the core principles and to improve the ROI of software development. An organization can use
one or a combination of frameworks to meet software development demands. For example, Scrum may
be used for development team members while Kanban is being used for operations team members on
the development team.
Below is a quick summary of the main Agile practices typically used.
Scrum is a process that relies on transparency and trust to achieve tight feedback loops in order
to deliver the highest value software to the business. The Scrum framework provides a reliable
delivery cycle that allows business leaders and customers to forecast when features will become
available. This optimizes business investments by delivering working software to meet customer
and market needs. *the recommendations in this document are focused on Scrum.
Lean is a process that has fundamental values that are easy to remember but not always easy to
achieve. The main principals of Lean are Identify Value Streams, Reduction of Waste, and
Increase Delivery Flow. These three pillars of Lean provide a team and business with a more
agile framework. Having a streamlined “lean” process enables teams to react to changes and
requirements to meet the business and customer needs
Kanban is a workflow that relies on visual signals to manage “Work In Progress” (WIP) for
teams. The simple approach of Kanban provides the visual aid and framework that some teams
like operation and other department need to manage the flow of work. In conjunction with a
Scrum or Agile practice, Kanban can provide missing workflow for the whole team (Development
to Operations) to becoming more agile.
Page vi
Visual Studio Agility Plan Template, Version 1, 2013.10
We have found that <Customer Name> is mainly using a <Type of Agile Practice> model.
Fundamentally, <Customer Name> is heading in the positive direction towards agile practices. However
<Customer Name> are running into issues running <Type of Process> within a predominantly <Type of
Agile Practice>. The work that <Customer Name> has done to move towards an <Type of Process> is
exemplary and <Customer Name> is getting value from the new process.
2.6
Existing Best Practices
Our interviews surfaced the following Best Practices that are being used by teams at <Customer Name>.
These practices are:




Business Analysis: The client uses <XYZ tools> to manage all project requirements.
Requirements Management: By utilizing the capabilities of TFS, particularly the alerting
mechanism, which effectively and efficiently keeps all team up to date.
Test Team Organization: The use of virtualization for test environments.
Test Planning: Test plans are well defined and linked to project plans.
We recommend that these practices continue to be employed and are continuously evaluated and
improved in order to promote process optimization.
2.7
Existing Hindering Practices
Guidance: The purpose of the existing hindering practices are to point out the pain-points the
team is experiencing from the lack of Agile practices. The following are some examples.
Teams Dynamics:
s
There is currently very little communication between team members and a complete lack of selforganization exists. These in themselves are however not the root cause of the apathy that is
surrounding the teams. That is caused by the lethargy of the business towards the software and is
bleeding
over Lifecycle
to the teams
and is reflected
in the Studio
qualityand
of the
software
that it Server
produced.
Application
Management
with Visual
Team
Foundation
In order to move forward effectively you need to identify who you’re Product Owner really is and where
that role should sit. I would recommend that you appoint the most capable Project Coordinator and
make them the Product Owner for the business with all other Product Coordinators being the primary
stakeholders and filters for the rest of the business.
Page vii
Visual Studio Agility Plan Template, Version 1, 2013.10
Current State and Approach:
Your business is currently not accepting of agile process. You have a Release Manager that is providing a
protective bubble so that your teams do not need to concern themselves with the organizations
reporting and documentation needs. If you removed this person, it would be unlikely that you would be
able to deliver software without a major overhaul of the wider organizations business processes. As the
current trend is to increase the red-tape and not reduce it you must continue to utilize this role.
An overhaul of your existing business practices and a solution to the current disinterested attitude of
your business to IT and software development specifically would allow you to pop the agile bubble that
is surrounding your teams and roll agile and lean thinking out to the wider organization.
2.8
Key Areas for Improvement
2.8.1
Current State – Urgent Issues
During our onsite interviews, we uncovered the following practices that should be considered critical
and essential to improving the development capability in relation to their impact on the business. These
practices were:
Guidance: This will be the main points for the call to action
s


Lack of consistent quality: The current unpredictability in requirements and testing are the
leading causes of inconsistent quality.
Allowing the Development Team to select the stories: Managers are directing the work the
teams stories to be worked on in each sprint
 Definition
of Done
–The Agilewith
Planning
should
have
a short, measurable
checklist that
Application
Lifecycle
Management
VisualTemplate
Studio and
Team
Foundation
Server
mirrors shippable and is applied at least every Sprint. Start with where you currently are and
increase this as you identify quality improvement opportunities at each team’s retrospective.
Page viii
Visual Studio Agility Plan Template, Version 1, 2013.10

2.8.2
Actionable backlog – Use the principle of jidoka (“automation with a human touch”) to enable
development teams to refuse backlog items that do not meet a minimum standard for delivery.
Current State - Additional Issues
During the course of the assessment, we were able to identify additional issues that we believe are
having a material impact on Application Lifecycle Management within <Customer Name>. These include:

Requirements Management
o Requirements Analysis/UAT: By ensuring the UATs are made available early in the
project, the test team can incorporate them into the testing lifecycle. This should ensure
a smooth UAT process.
o Storyboarding: By providing clear and understandable requirement of the features
requested, defects are found early in the lifecycle and are cheaper to rectify than those
found later.
o Portfolio Management: Allowing the Product Owner to track project backlog items
across multiple teams provides the transparency that is required to ensure the project
are released at the correct time in production.

Technical Debt
o Technical Debt Analysis: Technical debit includes items that build silently overtime. The
cost of technical debt is not known in most organizations. The first step is to capturing
that data is to analyze the cost and priories the stories to ensure they get into the
Product backlog.
o Technical Debt Payback: Once the items have be identified and prioritized by the
Product Owner, the team will need to add these stories and task to the work planned.
Page ix
Visual Studio Agility Plan Template, Version 1, 2013.10
3
Roadmap to Agile Maturity
Guidance: This section will set the stage and workflow on how a company can move to an
advanced maturity model. The purpose of this section is to explain that the growth and road is not
a direct one and that we recommend the organizations have to have additional support as they
move into different states or levels.
The level of Agile maturity varies greatly between companies and the type of software being developed.
The goals outlined in the maturity level are not “one size fit all” solutions for companies and teams,
s
rather,
they provide a direction for maturing your software development cycle to meet the needs of the
businesses and customer.
The growth of a mature Agile practice is not a direct one. As the team becomes more efficient,
requirements
become refined
and testing
more
transparent,
andServer
there will be new areas
Application Lifecycle
Management
withresults
Visual become
Studio and
Team
Foundation
of concern that will emerge.
As Agile maturity levels move through the next stages, (for example, Doing Agile to Being Agile), some
common “growing pains” will begin to emerge. These common “growing pains” will be the best choices
for additional assessments and guidance to help assist with the transition for the business, teams, and
individuals.
Page x
Visual Studio Agility Plan Template, Version 1, 2013.10
4
Agile Maturity Model
The model below shows the different maturity levels of Agile at a board scale. Each company will show
different levels of Agile maturity as they perform their team’s daily workflow. Enhancing the workflow to
achieve the best possible Agile process throughout the enterprise should be the end goal of the team
and individual effort. This falls into the basic Agile census principals - Transparency, Reduction of Waste,
and Flow of Value - from the individual to the team and the business.
Level 1
Level 2
Level 3
Level 4
Ad-Hoc
Doing Agile
Being Agile
Culturally Agile
Sporadic
developments
cycles and lose Agile
practices
Agile practices are
established but lack
of common
approach and
consistency
between team and
initiatives
Following Agile
principles
repeatedly across all
initiatives to achieve
business and
customer goals
Adopting Agile
approaches at a
strategic Enterprise
level
Page xi
Visual Studio Agility Plan Template, Version 1, 2013.10
5
Current vs. Ideal State
Guidance: This section will give an overview of the current state of the organization and how it
matches with the maturity model and the organization’s Ideal State.
s
<Customer
Name> is currently at a Agile Maturity level of 2, “Doing Agile”. The team has some good
practices in place and the enthusiasm to achieve the “Being Agile” level.
Current Practices [Doing Agile]:
Application Lifecycle Management with Visual Studio and Team Foundation Server
 Agile practices are established, however there is a lack of common approaches and consistency
between team and initiatives
 Requirements are created but not defined in a digestible size for the team to complete in a 2 to
4 week timeframe
 Software delivery is still a manual process in most environments
 Testing is not consistent or automated
Ideal State [Being Agile]:




Repeatedly follow Agile principles across all initiatives to achieve business and customer goals
Have defined requirements from the business and customer, that are sized well, and prioritized
into a backlog that the teams can pull from to complete within a 2-4 week timeframe
Have an automated software delivery pipeline to all environments
Have an automated and consistent testing approach in all environments
Page xii
Visual Studio Agility Plan Template, Version 1, 2013.10
6
Roadmap
Based on our observations and discussions, we recommend that the following iterative roadmap be
implemented in order to understand and instill Agile best practices within the teams.
Please note that the areas for improvement mentioned in the prior section, which are marked as urgent,
may not be addressed immediately. In some cases, the foundations for improving a particular service
area will not be in place in the first or second iteration.
First Iteration
In the first iteration, the improvement areas listed below are set as the first items to be addressed first.
The main goal of the first iteration is to identify and review the actionable items for the team and
consult to review and plan on the course that would be beneficial.
1. Solve the lack of consistent quality
During the interview process, management echoed the need to improve the quality of the software
being delivered. In order to improve the quality, the business and technical team need to understand
the root cause of the lack of consistent quality. The key areas identified during the assessment were the
lack of a well-groomed backlog (making the backlog actionable) and requirements.
The Product Owner needs to take time to constantly maintain a well-groomed backlog for the next 3
sprints. This will allow the team to estimate those stories and give them time to see if anything needs
breaking down.
Current issue(s)
1.
2.
3.
4.
The team often defines stories during Sprint Planning
Many stories straddle many sprints making it impossible to attain a sustainable pace
The product owner is unable to use velocity to determine delivery schedules
The Team fails to deliver a Sprint Goal
Recommendation(s)
The Scrum Master should make sure that the stories in the backlog are ready just-in-time for Sprint
Planning and make sure that the Product Owner has the knowledge and tools in Team Foundation
Server necessary to provide an effective backlog. The Product Owner should spend at least 4-8 hours
grooming (making the backlog actionable) during the sprint.
2. Allow the Development Team to select the stories:
The Stories that are given and estimated by the Product Owner is a direct impact on quality and poor
communication within the team.
Page xiii
Visual Studio Agility Plan Template, Version 1, 2013.10
The content of the backlog is currently inconsistently formed and is not well understood by the team. If
the team poorly understands it then the Product Owner likely poorly understands it as well.
Current issue(s)
1.
2.
3.
4.
The teams are unable to estimate stories accurately
In sprint is when teams figure out that items are too big
The Team fails to deliver a Product Backlog Item or a Sprint Goal
The Team sees that Scrum is fundamentally not working and elects to abandon the process
Recommendation(s)

Use Storyboarding tool in Team Foundation Server to identify User Stories more effectively

Use INVEST to gauge the size and complexity of user stories

Write stories in the “As <usertype> I want <requirement> so that <reason>” format to allow
ease of understanding

Write acceptance criteria in the “GIVEN <truth> WHEN <criteria> THEN <result>” model to help
focus on the important aspects of the stories

If you end up with more than 3-4 Acceptance Criteria on a user story then it is probably too big

Have all Product Owners, Dev Leads and Managers attend an Agile Requirements Workshop
Development teams of 6+-3
In order to facilitate communication between team members the team size should be between 3 and 9
people, and you are currently within this threshold. (Too many people make it less likely for each team
member to remember what everyone is working on.)
3. Have a Definition of Done
The teams currently have very little understanding of Scrum, particularly the Definition of Done.
Current issue(s)
1.
Teams are not creating “working” software each sprint
2.
There is an exponential increase in the cost of fixing a problem
3.
Confusion and lack of commination increases within the teams
Recommendation(s)
Have all of the Team Members, Scrum masters, and Product Owners read the Scrum Guide and pass the
free Scrum Open assessment. This will demonstrate a basic knowledge of the rules.
Explaining the tools that are in Team Foundation Server to support the transparency of work.
Page xiv
Visual Studio Agility Plan Template, Version 1, 2013.10
Have the team and the Product Owner work together to define the Definition of Done. Have this
agreement in a visible place for all team members.
Second iteration
Once the critical issues are in progress, we recommended having a continuous improvement progress
check-in. After the first iteration, there is potential for the team to be overwhelmed with the number of
new process, tools, and changes. We thus recommend time for the team to “work on your own” after
the first iteration.
In the second iteration there may be additional items added to the backlog, as the team find additional
pain points as they continue to work through the new process.
1. Requirements Management
Environment availability and the requirement for the software development will need to be further
refined as to keep with the Agile Census model of Flow of Value, Continual Reduction of Waste, and
Transparency. This will also increase the ALM Maturity model by improving the core characteristics.
Current issue(s)



The team is not able to gain fast feedback on bugs
The team does not have a consistent delivery model
There is an increase in queue times (silent project killers) for the software development team
Recommendation(s)
The requirements management has been called out in the first iteration as a “Lack of consistent quality”.
In iteration two, the focus will be to highlight the tool(s) in Team Foundation Server that will assist in
creating fast feedback loops and decreasing the queue time created by manual processes.
Training on Storyboarding and the Portfolio Management tools and best practices will also assist the
Product Owner and the team in increasing transparency and the overall health of a project.
Training on Business Driven Development (BDD) tools and practices could also assist the Product Owner
and the team by bridging the requirements communication of a project.
Further, the build and deployment model is the backbone of continuous integration and delivery. By
bringing build and deployment into the Agile process, the team can then be extended to have a
consistent delivery model. The build and delivery can increate quality and decrease the costly and errorprone manual process.
Page xv
Visual Studio Agility Plan Template, Version 1, 2013.10
2. Technical Debt
Technical debt has a number of impacts on a team and organization. It is one of the leading causes of
paralysis within a team. There is a loss of connection and functionality resulting from poor code quality,
and a number of workaround costs which impact the whole team.
Current issue(s)



An increase in bugs and poor code base
Missed opportunities on current and improved architectural design
Team frustration and performance
Recommendation(s)
Having a plan to identifying the root cause of the technical debt is the first steps on preventing the team
on accruing more. As the new practices are being implemented, so should a way to deal with the
technical debt.
Implementing Team Foundation Server tools and providing training on them can help teams understand
how to gather information on technical debt in the software development cycle.
Page xvi
Visual Studio Agility Plan Template, Version 1, 2013.10
7
Detailed findings on Agile Planning
The detailed findings will indicate the maturity levels of each of the following discipline areas. The
findings provide a general estimate of the maturity level and the impact it is causing.
Requirements Engineering & User Experience Summary :
Elicitation
Maturity Level Rating:
Level 2
Impact Level Rating:
Medium
Requirements Analysis
Maturity Observations:
The proficiencies relative to any formal requirements analysis are
minimal but seemingly evolving. As the requirements group matures
and moves from a UI driven approach to more comprehensive
approach to requirements management, this should continue to
improve
Maturity Level Rating:
Level 2
Impact Level Rating:
Medium
Best Practices:
There are many emerging capabilities relative to this capability as the
resulting of the implementation of this new group and although the
requirements efforts are focused on elicitation, the feasibility to
implement this practice is well positioned for successful adoption
Maturity Observations:
There is minimal traceability present within the development lifecycle.
The QA group has established an initial level of traceability, but the
drivers of this should be clear so any commitment to this should be
value added for both the business and IT.
Maturity Level Rating:
Level 1
Impact Level Rating:
Medium
Traceability
Development Summary :
Development Understanding Requirements
Maturity Observations:
Clear requirement provides a more efficient method for teams to
estimate the effort of or the work requested.
Maturity Level Rating:
Level 2
Impact Level Rating:
High
Impact Observations:
The inconsistency in requirements influences the team ability to
estimate correctly and to complete stories and task in a given sprint.
The teams are spending up to 30% of their time have to clarify the
requirements for the software development
Lack of requirements is impacting the ability to truly measure code
quality in development and through-out the lifecycle.
Impact Benefits:
Team are highly encouraged to take ensure the project requirements
are well understood and defined before starting a sprint.
Better defined requirements will enable the team to become more
proficient, reliable, and increase quality of the software being delivered
Page xvii
Visual Studio Agility Plan Template, Version 1, 2013.10
Sprint Cycles
Maturity Observations:
There was minimal discovery of formal consistence sprints within the
development team. The team tries to have a 4-week sprint cycle but
slips as stories are not completed. The team decides to extend the
sprint out of the extra week/s to complete stories.
Maturity Level Rating:
Level 1
Impact Level Rating:
Medium
Impact Benefits:
Having a set sprint cycle provides the team, Product Owner, and
business a predicable flow of value.
The business can then measure the rate the development team can
delivery working software for marketing and customer request
Definition of Done
Maturity Observations:
None of these are currently in place
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Impact Benefits:
By having an agreed on definition of done by all team members
provides a solid understanding of when a tasks or stories is completed
of a sprint. The activity reduces the waste by having some team
member do more than they need to and other’s not enough to meet
the objectives
Software Configuration Management Summary :
Collaborative Development
Maturity Observations:
Collaborative Development is accomplished today by a complex set of
labels in the branching solution. Although concurrent development is
recommended, it usually is associated with a cost and a learning curve.
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Impact Observations:
Currently the customer is suffering; They are manually promoting code
from one environment to the next. To enable concurrent development
they have used a complex label process. Fixing this process will have
impact across the board in the release process.
Impact Benefits:
By simplify how concurrent development is accomplish, <Customer
name> can simplify the life of development and allow Release
Management more time to monitor highly valued aspects of the release
process
Version Control Repository
Maturity Observations:
Labels are currently being employed as a branching approach. This is
further complicated by the multiple and various CM systems (Version
Manager and VSS) The build management is done in source safe and it
is transferred via zip file to the VM system.
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Impact Observations:
There is a significant opportunity to be attained in this area. These
manual processes can be improved through automation through the
implementation of TFS.
Page xviii
Visual Studio Agility Plan Template, Version 1, 2013.10
Impact Benefits:
Unified version control with a code promotion process supported by the
tools will ultimately deliver significant value to the team. This includes
time saved in the day to day process and few mistakes that impact the
entire team.
Release Management
Maturity Observations:
A root cause analysis of the many quality challenges can be traced to
the manual involvement of the release process. Mistakes are known to
occur between labeling and the release of the code. This combined
with the lack of formal ownership of the release itself can create
potential chaos. No one person appears to be accountable for the
management of the release
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Build Management
Maturity Level Rating:
Level 2
Impact Level Rating:
High
Deployment & Operations Summary :
Deployment
Maturity Observations:
There are apparent gaps that further put quality at risk within the
deployment space. This is present with the management of the custom
application code. Practices such as not applying formal QA to custom
code or the practice of a custom team folding code back over a code
baseline significantly compromises quality.
The QA and Production environment do not use the same deployment
mechanism and therefore testing is not actually testing deployment
Maturity Level Rating:
Level 2
Impact Level Rating:
High
Impact Observations:
Allowing the deployment process to be tested in the natural course of
testing with provide further assurance that deployment will be a
smooth process.
Impact Benefits:
Reduced the deployment errors found in production by testing the
deployment process in QA.
Environment Management
Maturity Observations:
The multiple environments are managed through heroic efforts. The
testing and staging environments are not automatically synchronized,
and this creates risk in bug definition and tracking. Equally it creates
significant challenges in the management of the numerous PTF's
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Project Planning & Management Summary :
Project Monitoring and Control
Maturity Observations:
With the reactive management of the ongoing PTF fixes as well as the
seemingly lack of ownership of a release, there appears to be strong
opportunity to improve the project control of these efforts. The
introduction of formalized and standardized reporting should be a start
to the significant changes requisite for improvement
Page xix
Visual Studio Agility Plan Template, Version 1, 2013.10
Maturity Level Rating:
Level 1
Impact Level Rating:
Medium
Testing and Quality Assurance Summary :
Test Team Organization
Maturity Observations:
The test organization is independent and well-staffed.
Although the test leads are involved in the early stages of the
product, they do not have any input into the estimation.
Maturity Level Rating:
Level 2
Impact Level Rating:
Low
Impact Observations:
Virtualization is used in all test environments
Impact Benefits:
Reduced the deployment errors found in production by testing the
deployment process in QA.
Maturity Observations:
Consideration to the various functional and non-functional aspects of
testing are considered and catered for within test plans. Each test
phase has an associated test plan.
Although the time allocated to the stabilization period at the start of
projects is sufficient this is often reduced actuality if development slips.
The release dates are not often moved, rather testing is squeezed.
Maturity Level Rating:
Level 2
Impact Level Rating:
Low
Impact Observations:
The perceived level of quality is a big pain issue for the customer.
When stabilization windows are reduced the number of bugs found in
production increases and undermines the confidence of the customer's
client.
Impact Benefits:
Test plans are well defined and linked to project plans.
Maturity Observations:
There are varying levels of maturity being displayed. Of particular
significance are the lack of linkage between test cases and
requirements, the lack of an integrated defect management tool and
the lack of test case prioritization.
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Impact Observations:
Test case prioritization - This is essentially to ensure that the most
important tests are executed first. This takes on additional importance
when stabilization windows are reduced.
Requirements linking - The customers client is very critical of the lack
of visibility when it comes to which requirements are being tested.
Defect Management - The time expended using tools that are
integrated into the process is impacted both the testing and
development areas
Impact Benefits:
Test case prioritization - All test cases should be prioritized during the
test analysis phase, so that they accurately reflect the customer's
needs.
Test-Requirement Linking - Linking test cases to requirements is an
essential enabler of test/project tracking progress.
Defect Management - Using an integrated defect management system
will reduce unnecessary level of complexity and duplication
Test Planning
Test Management
Page xx
Visual Studio Agility Plan Template, Version 1, 2013.10
Defect management - Utilizing the capabilities within Microsoft Test
Manager and TFS would greatly reduce unnecessary duplication of
effort and complexity
Test Types
Maturity Observations:
Functional testing is performed at the most basic level.
Maturity Level Rating:
Level 1
Impact Level Rating:
High
Impact Observations:
The client has significant issues with the level of functional testing
currently performed.
Impact Benefits:
Improved functional testing will improve the overall quality and
increase the client's confidence with the customer. Defect management
- Utilizing the capabilities within Microsoft Test Manager and TFS would
greatly reduce unnecessary duplication of effort and complexity
Page xxi
Visual Studio Agility Plan Template, Version 1, 2013.10
8
Resources
The following is a list of very useful resources readily available to help deepen your knowledge and
understanding.
Testing with Visual Studio and TFS

ALM Rangers Guide – Test Release Management:
http://vsartestreleaseguide.codeplex.com/
This Visual Studio ALM Ranger project has the primary goal of delivering scenario based and handson guidance for managing Microsoft Test Manager Test plans.

ALM Rangers Guide – Visual Studio Test Tooling Guidance:
http://vsartesttoolingguide.codeplex.com/
This umbrella project delivers a range of practical and scenario based guidance for Visual Studio test
features, such as Coded UI, Microsoft Test Manager, IntelliTrace, and Microsoft Fakes.
Automated Build-Deploy-Test

Setting Up Automated Build-Deploy-Test Workflows:
http://msdn.microsoft.com/en-us/library/hh191495.aspx
You can use a build-deploy-test workflow to deploy and test your application when you run a build.
This lets you schedule and run the build, deployment, and testing of your application with one build
process. Build-deploy-test workflows work with Lab Management to deploy your applications to a
lab environment and run tests on them as part of the build process.

Visual Studio Team Foundation Build Customization Guidance:
http://vsarbuildguide.codeplex.com/
This Visual Studio ALM Ranger project has the primary goal of delivering scenario based and handson lab guidance for the customization and deployment of Team Foundation Build.
General ALM Resources

What’s new in Visual Studio 2013:
http://www.microsoft.com/visualstudio/eng/visual-studio-2013

Getting started with Application Lifecycle Management:
http://msdn.microsoft.com/en-US/library/vstudio/dd286491(v=vs.110)

Technical Articles for Visual Studio Application Lifecycle Management
http://msdn.microsoft.com/en-us/library/ee889983.aspx
Page xxii
Visual Studio Agility Plan Template, Version 1, 2013.10

Visual Studio ALM + Team Foundation Server Blog:
http://blogs.msdn.com/b/visualstudioalm/

Brian Harry’s Blog: Everything you want to know about Visual Studio ALM
http://blogs.msdn.com/b/bharry/

Microsoft Visual Studio Virtual Labs:
http://msdn.microsoft.com/en-US/vstudio/ff640662
Technical ALM Resources

Visual Studio 2013 Application Lifecycle Management Virtual Machine and Hands-on-Labs /
Demo Scripts:
http://blogs.msdn.com/b/briankel/archive/2013/08/02/visual-studio-2013-application-lifecyclemanagement-virtual-machine-and-hands-on-labs-demo-scripts.aspx

Team Foundation Server Migration and Integration Solutions:
http://msdn.microsoft.com/en-us/vstudio/bb840033

Visual Studio Team Foundation Server Planning Guide:
http://vsarplanningguide.codeplex.com/

Administer Team Foundation Server:
http://msdn.microsoft.com/en-us/library/ms181758.aspx

Visual Studio and MSDN Licensing White Paper:
http://www.microsoft.com/en-us/download/details.aspx?id=13350

TFS Update List of Feature in each versions:
http://tfs.visualstudio.com/news/release-archive

Visual Studio Capabilities Comparison:
http://www.microsoft.com/visualstudio/eng/products/compare

Visual Studio Forums:
http://go.microsoft.com/fwlink/?LinkId=294362

Visual Studio Developer Center:
http://msdn.microsoft.com/en-US/vstudio
Page xxiii
Visual Studio Agility Plan Template, Version 1, 2013.10
9
Conclusion
We recommend that the implementation of the practices outlined in this document be validated during
the initial deployment and as projects and teams are brought on board the system. Teams constantly
have to adapt and change their processes as the development environment changes around them. The
Team Foundation Server Agile Planning tools allow projects and workflows to be highly customized to
accommodate these changes.
To encompass all of the recommendations in this document, a schedule for all of the relevant tasks
should be created. Complete implementation and customization should be done by <Customer Name>
operations staff.
Page xxiv
Visual Studio Agility Plan Template, Version 1, 2013.10