Demystifying Risk‐Based Audit Methodology
Instructor Jay Ranade
CISA, CISM, CISSP, ISSAP, CBCP, CGEIT
Ph. 1‐917‐971‐9786
[email protected]
[email protected]
Instructor Introduction
Jay, a certified CISA, CISM, CISSP, ISSAP, and CBCP, is an internationally renowned expert on computers, communications, disaster recovery, IT Security, and IT controls. He has written and published more than 35 IT‐related books on various subjects ranging from networks, security, operating systems, languages, and systems. He also has an imprint with McGraw‐Hill with more than 300 books called “Jay Ranade Series”. He has written and published articles for various computer magazines such as Byte, LAN Magazine, and Enterprise Systems Journal. The New York Times critically acclaimed his book called the “Best of Byte”. He is currently working on a number of books on various subjects such as IT Audit, IT Security, Business Continuity, and IT Risk Management. Jay has consulted and worked for Global and Fortune 500 companies in the US and abroad including American International Group, Time Life, Merrill Lynch, Dreyfus/Mellon Bank, Johnson and Johnson, Unisys, McGraw‐Hill, Mobiltel Bulgaria, and Credit Suisse. He was a member of the ISACA International's Publications Committee (2005‐2007). He teaches graduate‐level class on Information Security Management and Enterprise Risk Management at New York University. He also teaches accounting information systems, IT auditing, and internal auditing at St. John’s University.
Currently, he is the Director of Education for TechnoDyne University, a NJ, USA‐based professional educational services organization specializing in CARGOS (Controls, Auditing, Risk, Governance, Operations, and Security)
He is four times world champion in Arm Wrestling and two times world champion (2002 and 2003) in martial arts breaking. He has appeared on ESPN and ESPN2 numerous times. www.technodyneuniversity.com
2
Instructor Information
• Contact information
– [email protected]
– [email protected]
– USA +1‐917‐971‐9786
• TechnoDyne University
502 Valley Road, Suite 103
Wayne, NJ 07470
USA
Why Risk Based?
Two aspects • Risk‐based IT auditing has 2 aspects
• Risk‐based audit planning
– Annual basis
– CAE accountable for that
– That results in individual audit engagements
• Risk‐based individual engagement planning
– This is for each engagement
– Engagement auditor does that
www.technodyneuniversity.com
5
What is flow of Risk Management?
• Threat exploits a vulnerability
• That’s damages an asset
• That damages a business process – From AIC perspective
• That’s the risk
• You put controls in place to mitigate the risk
• Till (executive) management says it is an acceptable risk
• Note: auditors give assurance on that to the board
www.technodyneuniversity.com
6
Limited Audit Resources
• Limited audit resources in any organization
• Audit’s value proportional to cost element
• Cost of controls must be less than protection to the asset
– CAPEX and OPEX perspective
• Controls cost money and controls impede business
• Controls and value must be balanced
www.technodyneuniversity.com
7
Who Develops Audit Plan?
• Accountability lies with CAE
• Lack of IT knowledge with CAE
• Those who have knowledge of IT lack knowledge of business processes
• Knowledge of business processes and IT is required for an audit plan
• It is revised on an annual basis
• Remember that according to QARs, developing risk‐
based plan is the weakest link in IAA
www.technodyneuniversity.com
8
Organizational Factors • Organizational factors affect audit plans
• What is industry sector
– Financial, pharmaceutical, energy, health care
• Size of the organization
• Business processes
– Unique for each organization
• Geographical locations
www.technodyneuniversity.com
9
Four Steps in Risk‐based Audit Planning
1. Know the business
•
•
•
•
•
What are organizational strategies
What are business objectives
What is the risk profile
How is operations structured
How does IT support business
– Support role
– Is IT business enabler?
www.technodyneuniversity.com
11
2. Know the Audit Universe
•
•
•
•
•
•
•
•
Organization is collection of business processes
Applications support business processes
Infrastructure supports applications
IT Service Management supports infrastructure
Technologies constitute infrastructures
New projects are created continuously
All of them constitute the universe of audit
But you can not audit all……….
www.technodyneuniversity.com
12
3. Perform risk assessment
• Why? Because you audit subset of audit universe
• Risk assessment should be a process
• Risk factors help prioritize audit subjects
• Remember‐ Risk is to business processes, not IT
www.technodyneuniversity.com
13
4. Now, create Audit Plan
• Time to create audit engagements • Audit frequency determined by significance of business process
• Management can also add to engagement subjects
– Assurance and consulting
• Business will validate the plan but audit committee has final say
www.technodyneuniversity.com
14
1. Know the business
Each Organization is Different
•
•
•
•
•
•
•
•
Different mission
Different goals
Different objectives
Different business models
Different market base
Different supply channels
Different product generation or service generation processes
Different delivery mechanisms
• So there is no cookie cutter approach to audit planning
www.technodyneuniversity.com
16
What is the operating Environment?
• How business processes are structured to meet business objectives?
• Documents needed to understand
– Mission statement
– Vision statement
– Strategic plans (4‐5 years horizon)
– Annual business plans (one year)
– Annual reports and supplements
– Regulatory filings
www.technodyneuniversity.com
17
Operating Environment cntd.
• Key processes contributing to success of the entity
• Remember that business processes differ
– For each operating unit (BU)
– For each support functions (IT)
– For each entity‐level project (corporate)
www.technodyneuniversity.com
18
Operating Environment cntd.
• Operating units include core processes to meet objectives
– Manufacturing, sales, distribution, services
• Support functions support core operational functions
– Governance, compliance activities, HR, finance, cash management, treasury, procurement
– Oh yes, IT as well ;)
www.technodyneuniversity.com
19
Operating Environment cntd.
• Now you know the business processes
• A business process has three components
– Manual
– IT
– Third party dependency
www.technodyneuniversity.com
20
Operating Environment cntd.
• Our focus here is IT – Business processes need IT application systems
– Business application systems need infrastructure
•
DB, OS, networks, facilities
– Infrastructure needs supporting IT processes
• SDLC, operations, security, change management, problem management and many more
– And lets throw in compliance activities
• Regulatory, financial reporting
– They all have risks elements and contribute to risk‐based audit planning
www.technodyneuniversity.com
21
www.technodyneuniversity.com
22
2. Know the Audit Universe
Audit Universe
• What is audit universe
– Finite and all encompassing collection of audit areas
– Organizational entities
– Locations related to business functions
• Most comprehensive list of audits if CAE had UNLIMITED RESOURCES and TIME ;)
• It is independent of risk assessment
• There are 2 parallel universe ;)
– IT Audit universe and Business Audit universe
www.technodyneuniversity.com
24
Audit Universe
• You have to know what is possible before you know what is feasible • To know the audit universe you should know
– Organization’s objectives
– Business model
– IT support model
www.technodyneuniversity.com
25
Audit Universe
‐ Business Model
• Business Model
– Organization has business objectives
– Operations units and support functions support those objectives
– And each of them has business processes
• Business processes of sales units, marketing units, • Support functions have their own processes
– IT applications support these processes
– Infrastructure supports applications
www.technodyneuniversity.com
26
Audit Universe
‐ Centralized vs. Decentralized
• Centralized functions good for individual audits
– Network audit, Security admn. audit, DBM audit, server admn. audit, help desk audit etc
– These functions, if centralized, are ideal for individual audits – Audit team can cover a lot with a single audit
– Single GC audit paves way for application audits on a platform
www.technodyneuniversity.com
27
Audit Universe
‐ Centralized vs. Decentralized
• Centralized functions good for individual audits
– Centralized audit functions reviewed at least annually
– In decentralized, each location is a different audit at GC level
– In a decentralized environment, with diverse technologies, multiple number of reviews needed
www.technodyneuniversity.com
28
Audit Universe
‐ IT Support Processes
• IT Support Processes
– Infrastructure supported by support processes
– ITIL is the leader in support processes
– Change management, asset management, configuration management, release management, incident management, problem management
– Their effectiveness determines effectiveness of infrastructure to support applications
– Site audit is about “how they are followed and not effectiveness” because standard processes are always affective www.technodyneuniversity.com
29
Audit Universe
‐ Audit Subject Areas?
• What are audit subject areas
– Goal is to create most affective audits and coverage
– Business risk is NOT evaluated at this stage
– Defining too small audit subject areas hinder audit effort
• Because there is admn. overhead for each audit
– Large (long) audits can hinder client productivity
www.technodyneuniversity.com
30
Audit Universe
‐ Audit Subject Areas?
• What are audit subject areas
– There is no right or wrong way, depends upon organization culture
– 2‐3 IT auditors for 3‐4 weeks is appropriate audit size for a subject area
– Need highly technical people for GC audit and general auditors for AC – Management accountability consideration for grouping audit subjects
• Else resolution of audit issues becomes an issue
– Scope of each audit must be defined properly
www.technodyneuniversity.com
31
Audit Universe
‐ Business Applications
• Business audit universe
• IT audit universe
• Business applications usually audited with business audit universe
• And GCs audited as a separate entity
• ERP applications span many business processes
– So, they are given special consideration
www.technodyneuniversity.com
32
Audit Universe
‐ Now the RA
• Now you do risk assessment
• So that you can create a subset of audit universe
• And that is the basis of annual audit plan
www.technodyneuniversity.com
33
3. Perform Risk Assessment
Why Risk Assessment?
• Objectives are related to (this is what business wants from IT)
– Confidentiality
– Integrity
– Availability
– Reliability
– Efficiency
– Effectiveness
– Compliance
www.technodyneuniversity.com
35
What is RA Process?
• RA based on IT risk
– Likelihood
– Impact on the organization
• Audit based on if adequate controls in place to bring risk down to acceptable level
• Audit plan will be based on selecting a subset of universe based on RA
www.technodyneuniversity.com
36
Perform Risk Ranking (RR)…..
• Risk Ranking
– Impact and likelihood of occurrence
– Each risk may not be significant in the audit universe
– Weight differentiates relative importance over others
– E.g. for SOX compliance, an area directly related to accuracy of financial statements carries a higher weight vis‐à‐vis an area not directly related to financial statements
www.technodyneuniversity.com
37
Three RR Techniques ‐ 1
• Direct probability estimates and expected loss functions. Or application of probability to asset value
– Insurance industry uses this method, IT auditors do not
• Based on ALE = SLE x ARO – where SLE = AV x EF
www.technodyneuniversity.com
38
Three RR Techniques ‐ 2
• Observable or measurable factors to measure risk or class of risk
– Good for macro risk assessment not micro risk assessment
– This approach is OK if all auditable units are homogenous in the audit universe
www.technodyneuniversity.com
39
Three RR Techniques ‐ 3
• Weighted or sorted matrices. • Use of threats vs. component metrics
– Good for micro risk assessment
– Weight of component taken into consideration
• All components are not equal
– E.g. web‐facing applications carry more weight than non web‐facing application
– Used for application level risk assessment www.technodyneuniversity.com
40
Likelihood Scale
Likelihood Scale
H
3
High probability that the risk will occur.
M
2
Medium probability that the risk will occur
L
1
Low probability that the risk will occur
www.technodyneuniversity.com
41
Impact Model
• Impact is to business process and not IT
• Different impact models for different organizations
• Impact can be financial, reputational, asset‐
specific, client‐retention specific
www.technodyneuniversity.com
42
Impact Model Scale
Impact Scale (Financial)
H
3
The potential for material impact on the organization’s earnings, assets, reputation, or stakeholders is high.
M
2
The potential for material impact on the organization’s earnings, assets, reputation, or stakeholders may be significant to the audit unit, but moderate in terms of the
total organization.
L
1
The potential impact on the organization is minor in size or limited in scope.
www.technodyneuniversity.com
43
RR Score Model‐ an example
• Refer to the spreadsheet –Financial Impact
–Quality of internal controls
–Changes in audit unit
–Confidentiality, integrity, availability
www.technodyneuniversity.com
44
Recommended Annual Cycle
Level
Composite Risk
Score Range
Recommended
Annual Cycle
H
35–54
Every 1 to 2 years
M
20–34
Every 2 to 3 years
L
6–19
Every 3 to 5 years
www.technodyneuniversity.com
45
ITG Frameworks
• COBIT
– 4 domains and 34 processes
– 218 Control objectives
– CMM scale maturity level for each IT process
– Good for large organizations
• ITIL v.3
– Service strategy, design, transition, operations, continuous improvement
• ISO 27001/27002
www.technodyneuniversity.com
46
Prioritizing Applications
• Business processes are supported by applications
• So, computer applications form the hub of risk‐based audit plan • So, how do you prioritize applications? www.technodyneuniversity.com
47
3A. Prioritizing Applications
Examples of IT‐AC
• 3‐way match for AP
– PO, vendor invoice, recipient of goods/services
• Depreciation of CAPEX is recorded in the correct period
• Received goods are accrued upon receipt only
• SoD based on job function
– Governed by the principle of CARRE
• Goods procured with approved PO
www.technodyneuniversity.com
49
IT‐AC Transaction Audits
• No one person should...
– Initiate the transaction
– Approve the transaction
– Record the transaction
– Reconcile balances
– Handle assets
– Review reports
• ∙ At least two sets of eyes needed
www.technodyneuniversity.com
50
Facts about GCs and ACs
• IT‐ACs depend upon reliability of IT‐GCs
• If GCs are malfunctioning, ACs don’t have any value
– E.g. if change controls are weak, auditing internal processing of application has no value www.technodyneuniversity.com
51
Complex vs. Non‐Complex IT Environment
• Application controls differ in both
• Complex IT Environment’s characteristics
– Source code is developed in‐house – Customized prepackaged software is adapted to organization’s needs
– Changes made to systems, databases, and applications
– Production deployment of pre‐packaged applications, changes, and code
www.technodyneuniversity.com
52
Complex vs. Less‐Complex IT Environment cntd.
• Less‐Complex IT Environment’s characteristics
– Existing IT environment not changed much
– Pre‐packaged software implementation with no major modifications in current year
– User‐configurable options that do not change application functioning
– Not many IT development projects
www.technodyneuniversity.com
53
Complex vs. Non‐Complex IT Environment cntd.
• Less complex environment = more complex auditing – Because less complex environment does not have many inherent or configurable application controls for risk management
• So, degree of transactional or support application will drive scoping, implementation, effort level, and knowledge to perform application control review
• Auditing is about DE and OE of controls
– Less complex environment does not have many controls
www.technodyneuniversity.com
54
Manual Controls vs. IT‐ACs cntd.
• Risk Factor
– Pre‐packaged application does not allow for code changes
– However, application controls within complex ERP (SAP, PeopleSoft) can be disabled w/o code change
– And Packaged applications are ALL parameter driven for control changes
www.technodyneuniversity.com
55
AC and Risk Assessment
www.technodyneuniversity.com
56
Financial Reporting Risks
• Summary (Very important)
Revenue is from Business Units
Payables, payroll, treasury is corporate
But risks are in business processes
Controls are in processes
Processes can span business units
IT‐applications support business processes
IT‐AC are in IT applications Controls are also in underlying technology which is IT‐GC
– Control weakness in any of them can affect financial statement, so we do end‐to‐end audit
–
–
–
–
–
–
–
–
www.technodyneuniversity.com
57
So, How do You do IT‐AC Risk Assessment?
• Define the universe of – Applications supporting processes
– Databases supporting those applications (GC)
– Technology supporting those applications (GC)
• Remember that 3 associated GCs directly affect applications
– Change management
– Logical security
– Operational controls
• Remember that a table change in an application can eliminate controls thus bypassing change management controls for code changes
www.technodyneuniversity.com
58
So, How do You do IT‐AC Risk Assessment?
• Two methods to do risk assessment
– Qualitative
– Quantitative
• Qualitative is subjective
– Risk (1= low impact, 5= high impact)
– Controls (1= strong control, 5= weak control)
– Determine risk and control weights for each of the 10 factors • Quantitative is objective
– Annual < $100,000 is risk level 1
– Annual > $2,000,000 is risk level 5
www.technodyneuniversity.com
59
So, How do You do IT‐AC Risk Assessment?
• Qualitative is subjective (cntd.)
– Calculate (risk factor rating x current risk weight) or (risk factor rating x current control weight) for all 10 risk factors for an application
– Add score for 10 risk factors
– Calculate for all applications that need assessed
– Sort results in descending order of composite score
– Create audit plan based on higher composite risk score
www.technodyneuniversity.com
60
So, How do You do IT‐AC Risk Assessment?
• 10 factors and their weight for each application RA
–
–
–
–
Application contains primary controls (30)
DE of AC (20)
Complex or Less‐complex application (15)
Application deals with privacy issues (20)
• Depends if affected by EuroSOX, GLBA, HIPAA, Turnbull etc.
– Application supports more than one critical business processes (20)
www.technodyneuniversity.com
61
So, How do You do IT‐AC Risk Assessment?
• 10 factors and their weight for each application RA (cntd.)
– Frequency of application change (15)
– Complexity of application change (20)
– Financial impact of change (25)
– Overall effectiveness of IT‐GCs (25)
– Audit history of controls (10)
• Previous audits discovered serious DE and OE deficiencies
www.technodyneuniversity.com
62
Example‐ Application = A/P
Risk Factor Rating
Rating
Risk/Control Rank (1 to 5)
Risk Score App. Has Primary controls
30
4
120
DE of AC
20
3
60
Complex or non‐complex application
15
3
45
Privacy Issues or confidentiality issues
20
1
20
Support > one critical application
20
4
80
Frequency of application change
15
1
15
Complexity of application change
20
4
80
Financial impact of changes
25
5
125
IT‐GC Effectiveness
25
1
25
Audit History of controls
10
5
50
Cumulative Score
Remarks
600
www.technodyneuniversity.com
63
Next step……..
• Note: Total possible cumulative score is 1000. You may change risk factor rating or risk/control factor based on your subjective judgment
• Sort in the descending order of cumulative score
• Select higher score applications based on audit resource availability
• Important: Irrespective of cumulative score, audit will include evaluation of input, processing, and output controls
www.technodyneuniversity.com
64
AC and GC SoD Principles
• GC SoD follow DOPESS principles • AC SoD follow CARRE principles
• Note: Risk Assessment model depends upon may factors. It depends on your environment. We discussed only 10 in our case. There are some of those factors as well (given in the next foil). www.technodyneuniversity.com
65
17 factors for Application Assessment
• This one has 17 factors for Application Assessment:
•
•
•
•
•
•
•
•
•
Quality of internal controls
Economic conditions‐ fraud increases in bad economy
Recent accounting system changes
Time elapsed since last audit
Operational complexity
Operational environment change
Recent changes in key positions
Time in existence
Competitive environment
www.technodyneuniversity.com
66
17 factors for Application Assessment
•
•
•
•
•
•
•
•
Prior audit results
Assets at risk
Transaction volume
Regulatory agency impact
Staff turnover
Impact of application failure
Sensitivity of transactions
Monetary volume
www.technodyneuniversity.com
67
AC Audit Methods
• Business Process Method
– ACs present in all the systems that support particular business process
– BP may span many BUs
– ERP transactional applications arising out of BPR – In non‐ERP applications, review all applications spanning a BP
– Consider downstream and upstream interfaces (aka inbound and outbound interfaces)
www.technodyneuniversity.com
68
AC Audit Methods
• Single Application Method
– Suitable for non‐ERP and non‐integrated environment
– Not a recommended method for ERP applications
• There could be many data feeds going in and coming out of a module
• Difficult to assess ERP with single application
www.technodyneuniversity.com
69
Business Process Method Auditing An Example www.technodyneuniversity.com
70
Four Types of BP Audits
• Mega Process
– End‐to‐end audit or integrated audit
– E.g. in AP, it is procure‐to‐pay process
– Level 1
• Major Process
– One component of mega process
• E.g. one of the AP components
• procurement or receiving or payment of goods – Level 2
www.technodyneuniversity.com
71
Four Types of BP Audits cntd.
• Minor Process
– Component of major process
• E.g. PR and PO sub‐process of procurement process
– Level 3
• Activity
– System transactions that create, modify or delete data in a sub‐process
– Level 4
– IT auditor’s traditional domain
– But levels 1,2,3 are very important
www.technodyneuniversity.com
72
Example of Mega Process Procure‐to‐pay
Level 2
Level 3
Level 4
Procurement
PR and requisition processing
PO Processing
A, C, D
A, C, D
Receiving
Goods (services) receipt processing
Goods return Processing
A, C, D
A, C, D
Accounts Payable
Vendor management
Invoice processing
Credit memo processing
Process payments
Void payments
A, C, D
A, C, D
A, C, D
A, C, D
A, C, D
www.technodyneuniversity.com
73
Example of Mega Process Procure‐to‐pay cntd.
• Highlighted items in previous slide are called “triple control”
• Level 4 is where IT auditor concentrates
• But, if you don’t know level 1,2, and 3, risk is not mitigated because
– Controls at the lower level (level 4) do not compensate for controls at the higher levels
www.technodyneuniversity.com
74
4. Create the Audit Plan
The End Result
• Audit plan a subset of the audit universe
• It is an outcome of risk assessment
• Additions to audit plan from senior management and audit committee
• Everything must be risk based
www.technodyneuniversity.com
76
The Real Audit Plan
• In risk assessment driver is risk, influencer is resources
• In creating audit plan, driver is resources and influencer is risk
www.technodyneuniversity.com
77
Key Activities
Understand
Risks
Risk Assessment
• Obtain explicit input
Driver = Risks
Influencer = Resources
from stakeholders.
• Identify relevant risks.
• Assess risks.
• Prioritize risks.
Key Activities
Audit Plan
Driver = Resources
Influencer = Risks
Allocate
Resources
• Understand universe of
potential
audits subjects.
• Allocate and rationalize
resources.
• Reconcile and finalize
the audit plan.
Objectives For Risk Assessments And Audit Plans
www.technodyneuniversity.com
78
Requests from Stakeholders
• Stakeholder requests from board, audit committee, senior management, operating managers
• Special audit assurance from stakeholders
• Consulting services requests from stakeholders
• Fraud investigations requests come throughout the year
• Consulting engagements to be included in the audit plan
www.technodyneuniversity.com
79
Audit Frequency
• Multiyear plans presented to audit committee and management
• 3 to 5 years is normal for planning
• May need external resources • Annual plan is a subset of multiyear plan
• Audit frequency established at RA time
www.technodyneuniversity.com
80
Frequency vs. Resource Allocation
Priority
Frequency
Resource Allocation
H
Immediate action, usually within the first year Annual reviews or multiple actions within the cycle High allocation M
Mid‐term action within the audit cycle One or several audit engagements within the cycle; could be postponed Base allocation
L
Limited allocation
Audit engagements At most one audit usually not planned engagement planned within the cycle within the cycle Frequency and resource allocation of audit activities
www.technodyneuniversity.com
81
Audit Plan Contents
• Different types of IT audits
• Integrated business process audits
–
–
–
–
–
IT processes (as in COBIT, ISO, and ITIL)
SDLC reviews
Application controls
Technical infrastructure audits
Network audits
• Financial reviews, operational reviews, compliance reviews
• SoD
• New threats and innovations
www.technodyneuniversity.com
82
Integration of IT Auditing
• Low integration IT audit
– Isolated from non‐IT activities
• Partially Integrated
– Associated with business process reviews
– Application reviews
• Highly integrated
– IT audit part of business process engagement
– Multidisciplinary team
www.technodyneuniversity.com
83
Audit Universe
Low‐integrated
Audit Plan
Partially Integrated
Audit Plan
Highly Integrated
Audit Plan
Business Processes
• Operational
• Financial
• Compliance
Non‐IT audit
Non‐IT audit
Integrated approach
Applications Systems
• Application controls
• IT general controls
IT audit
Integrated approach
Integrated approach
IT Infrastructure Controls
• Databases
• Operating systems
• Network
IT audit
IT audit
Integrated approach
IT auditing and integrated auditing
www.technodyneuniversity.com
84
TARGETED RESULT
T
O
T
A
L
AUDIT RESOURCES
HIGH
A
U
DI
T
U
NI
V
E
R
S
E
Consider
alternative audit
approach (CSA)
LOW
Chart of targeted audit results
www.technodyneuniversity.com
85
Audit Plan –
A Living Document
• New threats and new vulnerabilities evolve
• IT has higher rate of change than non‐IT activities
• New technologies‐ e‐commerce, web applications
• Therefore, audit plan is a living document
www.technodyneuniversity.com
86
Executive buy‐in and Plan Approval • Audit plan presented to audit committee and senior management
• Also discussed with CIO, CTO, IT managers, business application owners
• Client interaction during RA is important • Buy‐in brings cooperation, hence value to the organization
www.technodyneuniversity.com
87
Questions
www.technodyneuniversity.com
88