Develop, Implement and Maintain
a CSV Program Globally – A Case
Study
Michael Reid
Manager, CSV QA
McNeil Consumer Healthcare
One Dream. One Team.
11
Agenda
I. Development and Implementation of the CSV Process
 Risk-based Approach
 Process and Procedures
 Training Program
II. The CSV Process





Planning Phase
Development Phase
Validation Phase
Operational Phase
Retirement Phase
III.Maintenance of the CSV Process
 CSV Governance
 CSV Continuous Improvement
IV.Interactive Discussion
 Open Question and Answer
McNeil Consumer Healthcare
One Dream. One Team.
2
Risk-Based Approach to CSV
 CSV processes must be risk based
– Best use of resources
– Allows focus on critical quality attributes
 System level risk assessment
 Requirements level risk assessment
 Risk assessments must translate into different levels
of effort
McNeil Consumer Healthcare
One Dream. One Team.
3
Procedures
• Clearly defined steps necessary to execute the process
• Critical for process control and compliance
• 3 levels of Procedures
Level I
Level II
Level III
Corporate and Quality Policies
Standards
Local Procedures
McNeil Consumer Healthcare
One Dream. One Team.
4
Characteristics of a Good Procedure
Easy to Understand
 The information should not be vague or capable of
misinterpretation.
Reliable
 Information is accurate and can be relied upon.
Complete
 Provides all the necessary information to achieve the
objective.
Usable
 Easy to use by its intended audience.
Easy to Maintain
 Permit changes to be made easily.
McNeil Consumer Healthcare
One Dream. One Team.
5
McNeil Computer System Validation Process
• Provides a controlled and consistent approach to address
computer system compliance from conception to retirement
• Defines activities, roles, training and documentation requirements
• Allows for a “Risk-based” approach:
– Scope / Strategy of validation effort
– Testing approach
– Consolidation of deliverables
– Resources necessary to conduct validation activities
• Provides governance to ensure sustainability and evolution of the
computer system validation process
McNeil Consumer Healthcare
One Dream. One Team.
66
McNeil CSV Process
Planning
Development
Validation
Operational
Retirement
• Planning - Designed to define the system requirements, the validation strategy and to
define the required deliverables – based on risk
• Development – Designed to create the system design and configuration, build the
system, perform the technical testing and to ensure that the system is ready for
validation
• Validation – Designed to provide the documentation of formal QA testing and the
deliverables necessary to release the system into use in a regulated environment
• Operational – Designed to ensure that the use is maintained in a validated state
• Retirement – Designed to ensure that the system, its data and its documentation are
retired in a controlled manner
McNeil Consumer Healthcare
One Dream. One Team.
7
Roles & Responsibilities
• Responsible for maintaining a validated state
throughout system’s life cycle
Business
/System Owner
• Ensures completion of the validation process
• Participates in requirement definition process
• Ensures system meets intended business
purpose
• Participates in validation testing
• Establishes and maintains business SOPs
• Provides updates to the System Inventory
• Reviews, comments and approves assigned
deliverables
McNeil Consumer Healthcare
One Dream. One Team.
8
Roles & Responsibilities
• Provides technical input to
requirements definition
Technical Unit
• Develops design from approved
requirements
• Develops, tests, debugs, implements
and maintains the application
• Provides technical support
throughout system life cycle
• Creates, reviews, comments and
approves assigned deliverables
McNeil Consumer Healthcare
One Dream. One Team.
9
Roles & Responsibilities
CSV QA
• Ensures compliance with McNeil
Policies, Standards, SOPs and
external regulatory requirements
• Reviews, comments and approves
deliverables as defined in the VPD
and related SOPs
• Participates in the event resolution
process
• Provides coaching and mentoring
McNeil Consumer Healthcare
One Dream. One Team.
10
McNeil CSV Procedures (SOP’s)
• Define the requirements for all
phases of CSV lifecycle
• Build on each other with one
phase following the next
• Identify the activities, deliverables,
roles and responsibilities for each
phase
– Validation Practices
– Computer System
Validation
– Electronic Record and
Electronic Signatures
• Individual SOP for each phase of
the life cycle
• Traceable to requirements defined
in McNeil CSV Standards
McNeil Consumer Healthcare
One Dream. One Team.
1
1
11
McNeil CSV Standards & Procedures apply to:
All computerized systems and
software supporting GMP
processes
• Manufacturing systems
• Laboratory systems
• IT systems
• Spreadsheets
McNeil Consumer Healthcare
One Dream. One Team.
1
2
12
McNeil CSV Templates
 Specific templates have been developed for every CSV
deliverable
 Templates are approved documents that correspond to a
governing SOP
 Use of these CSV Templates is required unless another CSV
QA approved equivalent is leveraged
 Provides consistent application of the CSV Process
 Includes “instructional text” to help facilitate completion
McNeil Consumer Healthcare
One Dream. One Team.
13
CSV Training Program
 Training is assigned based on role
– Various curricula developed: Overview, authors,
reviewer/approvers, owners, etc.
 Training program includes multiple methods of delivery
– SOP “Read and Understand” courses with Knowledge
Assessments
– Instructor Led Training (ILT) courses with KA’s
– Computer Based Training (CBT) courses with KA’s
 Training is monitored through (Learning Management
System) LMS
 Outlook mailbox set up to receive feedback on CSV
Training
 Training program continues to evolve
McNeil Consumer Healthcare
One Dream. One Team.
14
CSV Organizational Assessment
 The CSV QA group owns the CSV Process and has overall
responsibility for CSV at McNeil
 A full organizational assessment of the CSV QA group was
performed.
 The assessment was designed to determine if the appropriate
organization was in place to sustain the CSV process
 The assessment took into account the complexity and volume
of the work required in the process by evaluating each activity
defined in the CSV SOP’s and assigning work estimates to each
McNeil Consumer Healthcare
One Dream. One Team.
15
CSV Planning Phase
Planning
Development
Validation
Operational
Retirement
Assessment
McNeil Consumer Healthcare
One Dream. One Team.
16
CSV Planning Phase
 Defines the specific approach used for computer system validation as
it applies to a given system
 Validation strategy determined based on a risk level considering
factors such as system complexity, patient safety, regulatory
implications, and intended use
 A Risk Assessment is performed for each system either before or
during the Planning Phase
 An overall system risk level is determined that is used throughout
system’s life cycle
McNeil Consumer Healthcare
One Dream. One Team.
17
CSV Planning Phase
 The Validation Planning Document (VPD) provides the rationale and
approach to the validation and the required deliverables
 The VPD includes a system overview and the intended use
 The outcome of risk assessment is captured in the VPD
 Assessment of the Vendor is also addressed during this phase and
incorporated in the VPD as a form of risk and helps to shape the
validation effort
McNeil Consumer Healthcare
One Dream. One Team.
18
CSV Planning Phase
 Requirements are also developed as a part of the Planning Phase
 Requirements may be split in multiple documents or contained within
one document depending on the size and complexity of the system
 User Requirements clearly define what the Users wants the system to
do
 Functional requirements describe what the system should do to
satisfy the user requirements.
 The initial draft of the Trace Matrix is also developed during this
phase
McNeil Consumer Healthcare
One Dream. One Team.
19
CSV Planning Phase SOP’s & Templates
Standard Operating Procedures
 Determination Assessment for
Computer and Automated
Systems
 Planning Phase for Computer
and Automated Systems
 Vendor Pre-Audit Assessment
and Questionnaire SOP
Templates
o
System Regulatory Assessment
Template
o
System Exclusion Assessment Template
o
System Risk Assessment Template
o
Validation Planning Document Template
o
Training Plan Template
o
Deployment Plan Template
o
URS Template
o
FRS Template
o
Requirements Traceability Template
o
Vendor Pre-Audit Questionnaire Form
McNeil Consumer Healthcare
One Dream. One Team.
20
20
CSV Planning Phase Quality Gate
 The Planning Phase Quality Gate consists of:
– The Validation Planning Document (VPD) is approved
– Requirement Specifications are approved
– Initial draft of the Traceability Document
 Additional Quality Gate requirements can be identified in the VPD
McNeil Consumer Healthcare
One Dream. One Team.
21
21
CSV Development Phase
Planning
Development
Validation
Operational
Retirement
McNeil Consumer Healthcare
One Dream. One Team.
22
22
CSV Development Phase
 System’s design is defined, documented and developed
 The Development Management and Documentation Management
Strategy documents along with the Technical Specifications set the
scope and depth of the activities in this phase
 Technical Specifications, define in technical terms, how the system
developer will build the system so that its functionality meets the
Requirements defined in the planning phase.
 The development phase is greatly impacted by the type and size of
the system being implemented.
McNeil Consumer Healthcare
One Dream. One Team.
23
CSV Development Phase
 A custom system requires a high degree of rigor in development,
testing, and supporting activities.
 A configured system may not have custom coding, but the
configuration can range from minimal to extensive.
 With a non-configured system there may be no development
activities at all.
 The supporting activities include a traceability confirmation, formal
Design Review, development testing, and code reviews.
 Once the Quality Gate for the development phase is satisfied, the
Validation Phase may commence.
McNeil Consumer Healthcare
One Dream. One Team.
24
CSV Development Phase SOP & Templates
Standard Operating Procedure
 Development Phase for
Computer and Automated
Systems
Templates
o Document Management Strategy
Template
o Technical Specification Template
o Development Strategy Test Plan Template
o Development Test Script Template
o Development Error Report Template
o Development Testing Error Log Template
o Development Testing Summary Template
o Development Phase Summary Report
Template
McNeil Consumer Healthcare
One Dream. One Team.
25
25
Development Phase Quality Gate
 Approved Technical Specification Document
 Documented and approved development testing
 Traceability from Critical Requirements to Tech. Specs.
 Approved Development Phase Summary Report
McNeil Consumer Healthcare
One Dream. One Team.
26
26
CSV Validation Phase
Planning
Development
Validation
Operational
Retirement
27
McNeil Consumer Healthcare
One Dream. One Team.
27
CSV Validation Phase
 The Validation Phase provides documented evidence and satisfies
regulatory requirements of ensuring critical requirements and
associated business process flow(s) are challenged.
 The extent of activities and testing performed during the validation
phase are based on risk, complexity and regulatory relevance as
determined in the Planning Phase.
 The Validation Test Strategy can include installation, functional and
process testing
 Non-testing activities are also part of this phase and include training
development and delivery as well as development and approval of the
operational and maintenance SOPs.
McNeil Consumer Healthcare
One Dream. One Team.
28
CSV Validation Phase
 The Validation Summary Report documents completion of this
phase and closes the VPD.
 Prior to authorization for release to production, required SOPs must
be approved and effective and training for administrators and users
must be provided.
 The Computer System Inventory is updated, per Inventory of GMP
Computer Systems and Software SOP, and the document index gets
created and filed.
 Upon release authorization, as well as satisfying all Quality Gate
criteria, the operational phase of the computer system’s lifecycle
begins.
McNeil Consumer Healthcare
One Dream. One Team.
29
CSV Validation Phase SOP & Templates
Standard Operating Procedure
 Validation Phase for
Computer and Automated
Systems
Templates
o
o
o
o
o
o
o
o
Validation Testing Strategy Template
Validation Test Protocol Template
Validation Test Script Template
Validation Event Report Template
Validation Testing Event Log Template
Validation Test Summary Report Template
Validation Summary Report Template
System Release Authorization Notice
Template
McNeil Consumer Healthcare
One Dream. One Team.
30
30
Validation Phase Quality Gate
– Approved and
documented validation
testing
– All events are closed
– The required SOP’s are
approved and effective
– The validation summary
document is approved
– The release authorization
is approved by the
Business Owner and CSV
QA
McNeil Consumer Healthcare
One Dream. One Team.
31
CSV Operational Phase
Planning
Development
Validation
Operational
Retirement
32
McNeil Consumer Healthcare
One Dream. One Team.
32
CSV Operational Phase
 The Operational Phase ensures that the system operates as intended
and the validated state of the computer system is maintained.
 Operational and maintenance procedures, system security and
administration, change control, training and the maintenance of
validation deliverables are critical components in maintaining a
system in a validated state.
 The key activities during the Operational Phase are Change Control
and Periodic Reviews.
 The Operational Phase is the longest phase of the CSV Lifecycle
beginning when the system is put into service and ending when
system retirement activities commence.
McNeil Consumer Healthcare
One Dream. One Team.
33
CSV Operational Phase SOP’s & Template
Standard Operating Procedures
 Operational Phase for
Computer and Automated
Systems
Template
o Periodic Review Report Template
McNeil Consumer Healthcare
One Dream. One Team.
34
34
CSV Retirement Phase
Planning
Development
Validation
Operational
Retirement
35
McNeil Consumer Healthcare
One Dream. One Team.
35
CSV Retirement Phase
 The Retirement Phase ensures that a Computerized System is
retired from GMP service in such a way as to ensure the
preservation of the electronic records and data generated in
accordance with 21 CFR Part 11 and for the duration of their
required retention periods.
 The disposition of hardware and software is determined.
 All associated documents relative to the validation of the retired
system are properly retained and archived, including, but not
limited to the obsolescence of relevant SOPs and other documents
which may be related to the system being retired.
 The Inventory of GMP systems is updated with the change in
operational status.
 Retirement activities are deemed concluded when the retirement
summary is approved and the associated change control is closed.
McNeil Consumer Healthcare
One Dream. One Team.
36
CSV Retirement Phase SOP & Templates
Standard Operating Procedure
 Retirement Phase for
Computer and Automated
Systems
o Retirement Plan/Strategy Template
Templates
o Retirement Summary Report Template
o Communication Plan - Retirement
Template
McNeil Consumer Healthcare
One Dream. One Team.
37
37
Shared System Validation
 A Shared System is defined as a computerized systems intended for
GMP utilization by McNeil, but owned by another J&J Company.
 Specific responsibilities for system maintenance and quality system
activities are defined within a Quality Agreement.
 The Shared System Management procedure addresses the required
contents and maintenance of the governing Quality Agreement.
 The Shared System Validation procedure defines the McNeil
requirements for validation the McNeil business processes
supported by a shared system.
 Shared System validation can be achieved by either participation in
UAT or separate Business Process Testing.
McNeil Consumer Healthcare
One Dream. One Team.
38
What is a Quality Agreement?
• Agreement between two or more Johnson & Johnson companies
or a Johnson & Johnson company and an external company as to
the implementation and maintenance of computerized systems
or software
• The Quality Agreement identifies:
• System(s) covered by the agreement
• Entity that owns the computerized system and entity(s) that will share
access and GMP use of the computerized system
• Points of contact for each business entity identified within the
agreement
• Quality system activities and system maintenance responsibilities
McNeil Consumer Healthcare
One Dream. One Team.
39
IT Systems Used by McNeil - Comparison
McNeil-Owned
IT System
J&J IT Shared
System
CSV is managed under
direction of McNeil CSV
Quality Assurance
CSV is managed under
the direction of another
J&J division’s CSV
Quality Assurance
Business owner is
within McNeil
Business Owner is
within another J&J
Division
McNeil Business
System Liaison for a
Shared System
McNeil Consumer Healthcare
One Dream. One Team.
40
Shared Systems CSV SOP’s & Templates
Standard Operating Procedures
 Shared Systems Validation
 Shared Systems Management
o Business Process Test Plan
o Business Process Test Summary
Templates
o Business Process Test Script
o Business Process Testing Signature
Log
o Business Process Test Event Log
McNeil Consumer Healthcare
One Dream. One Team.
41
41
Spreadsheet Verification & Validation
 The Spreadsheet Verification and Validation process outlines the approach to
both spreadsheet verification and validation.
 The combination of risk and complexity determines the approach to
spreadsheet compliance.
 The procedure details the process for spreadsheet verification and
spreadsheet validation.
 Two special types of spreadsheets are defined within the procedure and
permit special handling: Single Use Spreadsheets and Type-writer Rule
spreadsheets.
 Prior to a spreadsheet being placed into use, the validation report and SOPs
must be approved and effective.
 The Computer System Inventory must be updated, per the Inventory of GMP
Computer Systems and Software SOP.
 Spreadsheets are maintained in a controlled environment per the
Spreadsheet File Share Repository SOP.
McNeil Consumer Healthcare
One Dream. One Team.
42
Spreadsheet Verification & Validation
SOP’s & Templates
Standard Operating Procedures
 Spreadsheet Verification and
Validation
 Special Use Reporting
o Spreadsheet Verification Plan Template
 Spreadsheet File Share
Repository
o Spreadsheet Verification Summary
Report Template
Templates
o Spreadsheet Validation Plan Template
o Spreadsheet Validation Summary
Report Template
o Spreadsheet URS-FRS Template
o Spreadsheet Technical Specification
Template
o Special Use Reporting Template
McNeil Consumer Healthcare
One Dream. One Team.
43
43
CSV Planning & Remediation Committee
 Goals:
– Measure Health of CSV
Process, SOP’s, Templates,
Training, and Tools
– Drive Continuous
Improvements to the CSV
Process
 Metrics
– RFT CSV Documents
– RFT Test Script Execution
– Break Fix Change Controls
– Periodic Reviews Results
– Audit Results
McNeil Consumer Healthcare
One Dream. One Team.
44
44
CSV Planning & Remediation Committee








Meets Quarterly
Reviews the meeting minutes from previous meeting
Reviews metrics
Reviews Periodic Review recommendations, internal audit
observations, regulatory agency inspection findings, and any other
committee actions
Determines if new Action Plans are necessary and assign the Action
Plan designees
Reviews Action Plan progress updates
Reviews and approves Action Plan summary reports
Discusses any additional agenda topics identified by a committee
member
McNeil Consumer Healthcare
One Dream. One Team.
45
CSV Metrics Definitions
Overall Goal: Monitor & Evaluate the health of McNeil’s CSV processes in place.
The following are the five metrics that will be gathered:
1. Right First Time Errors in Test Scripts: This measurement will determine the Right First Time of CSV Test Scripts by
analyzing the event reports included in the validation package due to errors in test scripts for computer system validations.
2. CSV Documentation: This measurement will determine the Right First Time for CSV Documentation by analyzing the
percentage of documents submitted for approval to Computer System Validation – Quality Assurance (CSV QA) that conform with CSV
SOP requirements without modification.
3. Audit Findings: This measurement will determine the health of a validated GMP computer system.
4. Break Fix Change Controls: This measurement will analyze the number of Break Fix Change Controls raised for a GMP
system to assess the robustness of the process.
5. Periodic Review Findings: This measurement will determine the health of a validated high risk GMP computer system.
McNeil Consumer Healthcare
One Dream. One Team.
46
CSV Metric Collection, Analysis & Reporting Overview
Daily
CSV
CSV
Planning
Planning
& &
Remediation
Remediation
Committee *
Committee
CSV QA Team Members - HO
CSV QA TEAM – Plant 1
Mo
Metric Analyst
CSV Metric
Analyst
AD CSV
QAQA
Head
of CSV
nth
ly
Daily
Mo
nth
ly
Monthly
Monthly
Quarterly
CSV QA
Members
- LP2
CSV
QATeam
TEAM
– Plant
Mo
nth
ly
CSV QA Metrics
Mailbox
Daily
Mo
nth
ly
CSV QA Team Members - LN
CSV
QA TEAM – Plant 3
Daily
CSV QA Team Members - FW
CSV QA TEAM - Plant 4
Collect Metrics
Each CSV
Team Member
sends Metric
Data Sheets on
the first of each
Month to
Departmental
Mailbox
Collect Metric
Data Sheets
and enter in to
CSV Metrics
Tracking
Spreadsheet
Generates
Reports
Interim
Monthly
Report
or
Quarterly
CSV
Metrics
Report
Quarterly
CSV
Metrics
Report
*Head of Quality
- Head
Sr. Director,
QA QA
of CSV
- AD, CSV QA (Chair)
1 Quality
Mgt.MGT
- Plant
FW Manu.
Site Quality
Quality
Mgt.
- Plant
LN Site2Quality
MGT
- Plant
LP Site3Quality
MGT
Quality
Mgt.
- IT CSV MGT
Plant
4
Quality
Mgt.
- IT Risk Assurance MGT
IT
Mgt.
- Tech. Ops. MGT
IT Risk Assurance Mgt.
Tech Ops Mgt.
Page 1
V1.0
McNeil Consumer Healthcare
One Dream. One Team.
47
Metrics Trending Examples – Not Actual Data
Total Document Volume per Month
Right First Time: CSV Documentation
100%
90%
80%
FOR PRESENTATION PURPOSE ONLY: NOT ACTUAL DATA
120
88%
80%
80%
105
97
80
60%
50%
100
80%
70%
FOR PRESENTATION PURPOSE ONLY: NOT ACTUAL DATA
60%
60%
54%
60%
41%
40%
30%
20%
10%
CSV
Documentation
60
Target
40
Trigger Limit
20
Total
Documents
Reviewed
53
0
0%
Oct-2014
Nov-2014
Oct-2014
Dec-2014
Nov-2014
Dec-2014
CSV Documentation: Errors by Type
FOR PRESENTATION PURPOSE ONLY: NOT ACTUAL DATA
80%
70%
60%
50%
40%
30%
20%
10%
0%
Incomplete Contents
Lack of Clarity, Completeness,
Consistency, or Measurability
Lack of Required Signatures
Oct-2014
Nov-2014
Dec-2014
McNeil Consumer Healthcare
One Dream. One Team.
48
CSV Process Continuous Improvement
 Goal is to sustain and continue to evolve the process
 Various mechanisms in place
–
–
–
–
Metrics
CSV Planning & Remediation Committee
CSV Community of Practice
Voice of Customer
 Efforts already underway to simplify targeted areas,
improve other areas but maintain compliance
McNeil Consumer Healthcare
One Dream. One Team.
49
Interactive Discussion
McNeil Consumer Healthcare
One Dream. One Team.
50