European Court of Auditors
Experiencies in implementing Audit Quality
Eduardo RUIZ GARCIA
Head of division
Håkan HELLSTRÖM
Principal auditor
Vilnius March, 2006
European Court of Auditors
• Supreme Audit Institution of the European Union
• Audit the budget of the EU, >100.000 Mio €
• “Audit Office type” mandate; no judicial powers
• Collegial body; 25 Members
• Staff +/- 750 (half auditors)
• Financial & Performance Audit
• Annual Report, Statement of Assurance; Special Reports; Opinions
PRESIDENCY
Supervision of the performance of the Court's work, External relations, Legal matters
5 AUDIT GROUPS
I
II
III
IV
Agricultural policies
Structural and internal policies
External actions
Own resources, banking activities, administrative expenditure,
Community institutions and bodies
CEAD GROUP
Coordination, Evaluation, Assurance, Development,
ADAR unit (Audit Development and Reports)
ADMINISTRATIVE COMMITTEE (1 Member per Audit Group)
Administrative matters requiring Court decision
AUDIT COMMITTEE (3 Members and external expert)
Court’s internal auditor reports to Audit Committee
SECRETARIAT-GENERAL
Human Resources, IT and Telecommunications, Finance and Administration,
Translation Service, Court’s secretariat
ADAR (Audit Development And Reports)
ADAR has a horizontal mandate, to assist the Audit
Groups and the Court by:
• Developing audit standards/methods/procedures/tools
• Providing support to Audit Groups and ensuring that
standards, methods and procedures are followed
• Ensuring the publication of quality Court reports and
opinions in a timely manner
• Facilitating liaison with auditees, Budgetary authorities,
Commission DGs (notably DG BUDG) and Internal Audit
service, COCOBU/Budget Committee of Council, OLAF
European Court of Auditors
AREAS FOR ADDING QUALITY
Strategy/Programming
Planning the audit Executing the audit
Resources
Organisation structure
Procedures
Staff
Knowledge management
Information systems
Audit standards
Defining priorities
Allocating resources
Audit objectives
Audit scope
Audit approach
Resources
Obtaining evidence
Testing
Supervision
Reporting
Drafting
Presentation
Communication
Director's Quality
statement
IT SelfAssessment
New Programming Procedure
Intranet project
Review of Audit
Policies &
Standards
ADAR Quality Support
Advice for Audit Programmes: Development of methodology
Performance Audit Manual &
Review of APM
Guidelines
Programming
guidelines
Recruitment
Working environment
Professional training
Staff appraisals
Quality Assurance
(ex-post verification)
ECA Self-Assessment & Peer Review
ADAR Quality Support
Review of reports
European Court of Auditors
AREAS FOR ADDING QUALITY
Strategy/Programming
Planning the audit
Executing the audit
Resources
Organisation structure
Procedures
Staff
Knowledge management
Information systems
Audit standards
Defining priorities
Allocating resources
Audit objectives
Audit scope
Audit approach
Resources
Obtaining evidence
Testing
Supervision
Reporting
Drafting
Presentation
Communication
Director's Quality
statement
IT SelfAssessment
Intranet project
Review of Audit
Policies &
Standards
New Programming
Procedure
Programming
guidelines
ADAR Quality Support
Advice for Audit
Programmes:
Review of APM
Development of
methodology
Performance Audit Manual
&
Guidelines
Recruitment
Working environment
Professional training
Staff appraisals
Quality Assurance
(ex-post verification)
ECA Self-Assessment & Peer Review
ADAR Quality Support
Review of reports
New Programming Procedure
• Why a new procedure:
- consistent application
- rational decision making
-straitforward and “economic”
• What the new procedure looks like...
Audit Strategy
• setting priorities
• organising
• allocating resources
Annual Work Programme
• permanent/recurrent
Portfolio Potential Audit Tasks
• identify tasks
• analyse
• ranks / priority
• alternative proposals
• selected tasks
• allocation of resources
• reporting deadlines
New Programming Procedure
Portfolio of Potential Audit Tasks
Establishment of the Portfolio of Potential Audit Tasks
Analysis of audit field
Identification of topics
Assessement of criteria: risk, materiality, coverage, relevance
Setting of priorities (Low, Medium, High)
Ranking the topics
List of Audit Topics
Ref Topic Priority(L/M/H)
1
(...)
n
Remarks
Audit Proposal (n)
For top priority Audit Topics
Completion of an Audit Proposal:
- Description of the Audit field
- Reasons for the Audit
- Audit organisation
- Milestones
- Other remarks
Audit Proposals (1...n)
Audit Proposal (1)
I- Audit Field
II- Reasons for the audit
III- Audit Organisation
IV- Milestones
V- Other remarks
Elaboration of the Annual Work Programme
Selection of the Audit Proposals to be put forward in the Annual
Work Programme
Annual Work Programme
Audit Proposals selected
Development of Methodology
and Audit Quality Support
Development of
Performance Audit
Manual &
Guidelines
Audit Quality Support
Development of Methodology
and Audit Quality Support
Product definition
Development of
Performance Audit
Manual &
Procedures
GuidelinesQUALITY
Communication
Audit Quality Support
Documentation
Audit Quality Support
Active support
Review of APM
Product
identification
Define together with
“client”
Transparent criteria
Procedures
Set up as tasks
Check FAQ
Internal consultation
Checklist
Internal consultation
Documentation
Work papers
Chronology of events
Data base
Checklist
Minutes
Communication Discussions
Recommendations
Evaluation sheet
Written note
Development of Methodology
Court mission statement
Court Audit Policies and Standards (CAPS)
Performance
audit manual
Financial
audit manual
General
topics
Toolbox:
guidelines on methods, studies, software...
Literature selected from other sources
Development of Methodology
Performance Audit Manual
Chapter 1
Framework (part I)
Chapter 2
Framework (part II)
Chapter 3
Essential Qualities
Chapter 4
Planning
Chapter 5
Examination
Chapter 6
Reporting
Development of Methodology
Toolbox
Structuring
methods
Data collection
methods
Data analysis
methods
Risk analysis
Interviews
Ratio analysis
Developing audit
questions
Case studies
Frequency count
Programme Logic
Model
Surveys
Regression
Analysis
Brainstorming
Documentary review Comparative
analysis
Organisational
mapping
Focus groups
Coding and
abstraction
Quality Assurance
(ex-post verification)
• Objectives of the Quality Assurance function:
provide assurance of the quality of the audit work
contribute to improve audit work
• Main constrains
standards & experience staff
add value
acceptance & communication
• Main outcome
most common findings: documentation, audit trail,
supervision, use of resources
integrate results into audit practice
ECA Self-Assement & Peer review
• Why a Self-Assessment & Peer review
quality
best practice
legitimacy
• The procedure
 the Common Assessment Framework
Self-Assessment Groups
final report
Action plan to be decide by the Court
Peer-review
ECA Self-Assement & Peer review
The CAF Model
ENABLERS
RESULTS
People
results
People
Leadership Strategy &
planning
Partnerships
& resources
Processes
Citizen/
Customer
Results
Society
results
INNOVATION AND LEARNING
Key
performance
results
Conclusions: some quality hints
• Control & supervision is important, but is not the “key” to quality
• More rules, standards & instructions are not always the solution; need
for guidance & support (acceptance & secure).
[Guidelines on Audit Quality]
• Key to Quality: capacity and attitude of the organisation & the staff
• Improving quality, twofold strategy:
macro: general and long-term vision
micro: implement concrete improvements in each possible area
• Support the organisation & the auditors in adding quality