auditing of predetermined objectives

27 November 2015
AUDITING OF PREDETERMINED OBJECTIVES
Reputation promise/mission
The Auditor-General of South Africa has a
constitutional mandate and, as the Supreme
Audit Institution (SAI) of South Africa, it exists to
strengthen our country’s democracy by enabling
oversight, accountability and governance in the
public sector through auditing, thereby building
public confidence.
Topics to discuss
1. Introduction
2. Legislative requirements and framework for performance
management and reporting
3. Audit strategy and approach
4. Findings for local government 2013-14
5. Recommendations and good practices
6. Key controls for local government 2013-14
3
Introduction
Need for reporting on predetermined objectives
• Supreme audit institutions (SAIs) do not only conduct
financial audits. They also provides assurance on the
reporting of predetermined objectives produced by
municipalities, government departments and entities
• Why is this necessary?
Public sector reforms
Improving public reporting
Providing better information on what government is
accomplishing regarding service delivery
Audits performed by AGSA
Annual mandatory audits
• Financial statements
• Report on predetermined objectives
• Compliance with laws and regulations
Discretionary audits
• Performance audits e.g. infrastructure
Difference between auditing of predetermined objectives
(AOPO) and performance auditing
AoPO
Audit of Predetermined Objectives
 Annual audit of reported actual
performance against predetermined
objectives, indicators and targets as
contained in the annual performance
report.

PA
Performance Auditing
•
Individual audits
•
Focused on a specific government policy
or management process
•
Audit criteria are the 3E’s:
Integral part of the annual regularity
audit process, confirming:



compliance with related laws
and regulations
usefulness of performance
information
reliability of performance
reporting
•

Economy

Efficiency

Effectiveness
Done by subject matter experts
The planning, budgeting and reporting cycle
Legislative requirements and framework for
performance management and reporting
Auditing requirements
Sections 20(2)(c) and 28(1)(c) of the Public Audit Act (PAA)
require that:
An audit report must reflect an opinion or conclusion on
the performance of the auditee against predetermined
objectives
Applicable to all spheres of government
10
Legislative requirements and framework
Municipal Finance Management Act (MFMA)
Municipal Systems Act (MSA)
MFMA circulars
LG: Municipal planning and performance management regulations, 2001 –
GNR.796 of 24 August 2001
LG: Municipal performance regulations for municipal managers and managers
directly reporting to municipal managers, 2006 –
GNR.805 of 1 August 2006
NT Framework for managing programme performance information –
issued by the National Treasury in May 2007
This represents the performance management and reporting framework against
which the performance information should be managed and reported.
Principles and requirements from framework are used as a basis for the audit.
Audit strategy and approach
AGSA strategy
2004-05 to 2008-09 2009-10 to 2010-11
 Phased-in approach
 Factual audit findings
reported in both
management and audit
reports
 No audit opinion in
audit reports
 Interaction with
stakeholders (NT,
Presidency, DPSA) to
determine and test
audit approach
 Inputs to drafting of NT
frameworks (FMPPI)
 Completed phased
audit approach
 Audit to the extent
necessary to express
an audit conclusion
 Audit conclusion in
the management
report for all highcapacity
municipalities and
their entities
 Audit reports contain
audit findings – not
audit opinions
2011-12 to 2014-15
 Completed phased
audit approach
 Audit to the extent
necessary to express
an audit conclusion
 Audit conclusion in
the management
report for all
municipalities and
their entities
 Audit reports contain
audit findings – not
audit opinions
 This will continue until
the environment
shows a state of
readiness to provide
an audit conclusion in
the audit report
Audit Approach
• The audit approach for the audit of predetermined objectives is similar to that
of the audit of the financial statements in the following ways:
1
Obtaining an understanding of the entity's information system and related control
activities.
2
Assessing risks of material misstatement to form a basis for designing and
performing further audit procedures.
3
Obtaining sufficient appropriate evidence on which to base the conclusion on the
validity, accuracy, and completeness of reported performance information.
• Auditees need to instil the relevant controls to ensure that the reported
performance for each indicator is valid, accurate and complete in the same way
controls are implemented to ensure line items in the financial statements are
valid, accurate and complete
• Proper record-keeping measures should be in place to ensure that information
is accessible and available to support performance reporting
Audit criteria (derived from performance management and reporting
framework)
Main criteria
Sub-criteria
Definition
Presentation
Performance reporting using the relevant NT
reporting principles
Measurability
Indicators are well defined and verifiable
Targets are specific, measurable and time bound
Relevance
Indicators relate logically and directly to the entity
mandate and realisation of strategic goals and
objectives
Usefulness
Consistency
Objectives, indicators and targets are consistent
between planning and reporting documents.
Validity
Reliability
Accuracy
Completeness
Reported performance has occurred, have been
recorded and reported correctly and completely.
Measurability and relevance
criteria
– further details
Measurability
• Test well defined
of indicators
Measurability
of targets
Relevance of
indicators
• Test verifiability
• Specific
• Measurable
• Time-bound
• Direct and logical link of
indicators and targets
• To mandate, strategic
goals and objectives
What do auditors test when auditing measurability?
Measurability
of indicators
Measurability
of targets
• Test whether indicators are :
• well defined (Does the indicator have a clear definition and is there
technical indicator descriptions for each indicator? )
• verifiable (Is it possible to verify the processes and systems that
produce the indicator?)
• Test whether targets are:
• Specific (Is the nature and required level of performance clearly
identified?)
• Measurable (Can the required performance be measured?)
• Time-bound (Is the time period/deadline for delivery specified?)
What is appropriate audit
evidence?
Relevant
Appropriate
evidence
Reliable
evidence
Examples of relevant and reliable evidence
Relevant
Reliable
• Evidence is relevant to planned target and reported achievement
e.g. Target is to build a dam but evidence provided is a report on a
infrastructure plan to be developed
• The primary sources or origin of the actual performance is available
to the auditor
• Evidence is more reliable when
• In documentary form – e.g. paper, register, electronic or other
media
• Generated through management and information systems
with proper controls
• All source information exist and is complete – e.g. a list of
houses built that can be verified against completion certificates
• Information is valid – e.g. every entry in register can be traced
to a invoice/payment certificate or appropriate record
Audit reporting – management report
An audit conclusion will be expressed for ALL municipalities and municipal
entities in the management report, on –
USEFULNESS of reported performance information for
the selected development priorities/ objectives
RELIABILITY of the reported performance for selected
development priorities/objectives
Audit reporting – audit report
REPORT ON OTHER LEGAL AND REGULATORY REQUIREMENTS
Predetermined objectives
Usefulness of information
Material audit findings focusing on consistency, relevance and measurability
of reported performance information for selected development priorities
and objectives
Reliability of information
Material audit findings focusing on reliability of reported performance
information for the selected development priorities and objectives
Compliance with laws and regulations
Non-compliance findings relevant to the performance management systems
and processes
Findings for local government 2013-14 (278
municipalities and 57 municipal entities)
Three-year trend – findings on annual performance report
(Improvements noted)
38% (126)
59% (199)
31% (103)
69% (230)
24% (79)
76% (251)
3% (10)
2013-14
2012-13
2011-12
With findings
Outstanding audits
* Statistics at 20 January 2015
With no findings
There has been an increase in auditees with no material
findings on the quality of their annual performance reports
when compared to the previous year.
Findings on the lack of performance reports as well as the
usefulness and reliability of annual performance reports
Usefulness
147
20
Reliability
120
38
36
No report
13
6
13
Finding addressed
New finding
51
Repeat finding
Findings on the lack of performance reports as well as the
usefulness and reliability of annual performance reports
• The annual performance reports of 55% (167) of the auditees was not useful
• The most common findings on usefulness were that auditees reported on
indicators that were not well defined (105) or verifiable (87) with targets that
were not measurable (101) or not specific enough (89) to ensure that the
required performance could be measured and reported in a useful manner. A
total of 99 auditees reported information that was not consistent with the
objectives, measures and/or targets in their plans
• Findings on reliability were identified at 52% (158) of the auditees
• The most common challenges on reliability were as follows:
 No supporting documents or insufficient supporting documents was
provided to support the reported targets (Validity)
 The amounts, numbers and other data relating to the actual targets
reported did not agree to the supporting documentation provided
(Accuracy)
 The report has missing information, not everything relating to the
actual results and events was recorded (Completeness)
Root causes of defective service delivery reports
Lack of
monitoring by
leadership
Ineffective
audit
committee
oversight
Inadequate
formal planning
for PDO
reporting
Organisational
structures not
aligned
Lack of
accountability and
consequences for
actions / inaction
Inadequate
internal audit
testing systems
and data
Poorly defined
roles and
responsibilities
Structures not
capacitated
PDO reporting
not included in
performance
agreements
Do not meet regulatory
requirements
Not useful
Not reliable
Recommendations and good practices
Recommendations and good practices
•
•
•
•
•
•
•
•
•
Policies and procedures to report on performance information should be
developed and implemented
A set of performance indicators that are well defined and verifiable should be
developed
Performance targets that comply with the SMART criteria should be developed
Data definitions for all performance indicators should be defined
CoGTA should support local government by developing customised indicators
and targets
The methodologies and systems used for compiling budgets should be improved
to strengthen the link between budgets and performance targets
The correlation between planned and actual performance (performance
implementation) in relation to the budgeted versus actual expenditure (financial
implementation) should be monitored and evaluated
Predetermined objectives, financial reporting and compliance reporting should
be viewed in an integrated fashion as these are inseparable processes
Auditees need to ensure that they have adequate skills and resources to perform
proper strategic planning, performance monitoring and reporting in line with
applicable requirements
Recommendations and good practices
• Formal processes and systems for the collection, collation, verification and
storing of actual performance information should be developed, documented
and approved by municipal manager.
• Auditees should ensure that there is an adequately resourced and functioning
internal audit unit, audit committee and MPAC that provides assurance over the
quality of the quarterly and annual performance reports
• Management should maintain portfolios of evidence to support reported
targets, which are reviewed monthly and audited by the internal auditors.
• There needs to be sufficient oversight and monitoring of performance during the
reporting cycle to ensure that performance targets are reported as planned
• Larger auditees should consider a dedicated strategic planning and/or monitoring
and evaluation unit
• Auditees should prepare an audit action plan based on the audit findings of the
previous financial year, including definite actions to address these audit findings
• Leadership must insist on credible quarterly performance reports that link to key
projects undertaken and progress to date
Key controls for local government 2013-14
(278 municipalities and 57 municipal entities)
Drivers of internal control
31
Basic controls
32
Dashboard report indicators
Significant deficiency is not applicable
Significant progress had been made to address it
Urgent attention to the matter is required
Components of the dashboard report
3 Audit dimensions
Fundamentals
of internal control
Financial
Performance
objectives
Assessment
Leadership
•
Provide effective leadership based on a culture of honesty,
ethical business practices and good governance, protecting
and enhancing the best interests of the entity
•
Exercise oversight responsibility regarding financial and
performance reporting and compliance and related internal
controls
•
Implement effective HR management to ensure that adequate
and sufficiently skilled resources are in place and that
performance is monitored
•
Establish and communicate policies and procedures to enable
and support understanding and execution of internal control
objectives, processes, and responsibilities
•
Develop and monitor the implementation of action plans to
address internal control deficiencies
•
Establish an IT governance framework that supports and
enables the business, delivers value and improves
performance
Complian
ce with
laws and
regulatio
ns
Components of the dashboard report
3 Audit dimensions
Financial
Fundamentals
of internal control
Performance
objectives
Assessment
Financial and performance management
•
Implement proper record keeping in a timely manner to
ensure that complete, relevant and accurate information is
accessible and available to support financial and performance
reporting
•
Implement controls over daily and monthly processing and
reconciling of transactions
•
Prepare regular, accurate and complete financial and
performance reports that are supported and evidenced by
reliable information
•
Review and monitor compliance with applicable laws and
regulations
•
Design and implement formal controls over IT systems to
ensure the reliability of the systems and the availability,
accuracy and protection of information
Compliance
with laws and
regulations
Components of the dashboard report
3 Audit dimensions
Fundamentals
of internal control
Financial
Performance
objectives
Assessment
Governance
•
Implement appropriate risk management activities to
ensure that regular risk assessments, including
consideration of IT risks and fraud prevention, are
conducted and that a risk strategy to address the risks is
developed and monitored
•
Ensure that there is an adequately resourced and
functioning internal audit unit that identifies internal control
deficiencies and recommends corrective action effectively
•
Ensure that the audit committee promotes accountability
and service delivery through evaluating and monitoring
responses to risks and providing oversight over the
effectiveness of the internal control environment including
financial and performance reporting and compliance with
laws and regulations
Compliance
with laws and
regulations
THANK YOU
Questions?
37