The role of the SAI in promoting
IT efficiency in Government
Simon Camilleri
Background
EU member state with a population which is under 0.5 million.
• In 1814, the first Audit Department was set up on the island,
which was then under British rule.
• Following Independence in 1964 and the setting up of a
Republic in 1974, the National Audit Office (NAO) was
established with the current organizational setup as a result
of Constitutional amendments unanimously approved in
1997.
Background (cont)
With a staff compliment of 40 qualified audit staff, the NAO
completed the following audits in 2011 and 2012
(2011)
• 5 Performance audits
• 1 Information Technology (IT) audit
• 3 Investigative audits
• Annual financial audit report
(2012)
• 7 Performance audits
• 2 Information Technology (IT) audit
• 4 Investigative audits
• Annual financial audit report
IT audits carried out by NAO
• NAO executed its first IT audit in 2007 as part of its annual
financial audit and reviewed the IT resources within the
Education Department and the Courts of Justice Division.
• These audits and similar audits in subsequent years included a
general review of the management of the IT systems at the
auditee site.
• In October 2011, NAO published the report of the first stand
alone IT audit which was carried out at the Inland Revenue
Department (IRD).
• Following this audit another two similar standalone IT audits
were carried out at Heritage Malta and the Medicines
Authority whose reports were published in 2012
IRD IT Audit
The IRD IT audit and subsequent IT audits were structured on
the COBIT 4.1 audit framework and would review the following
aspects of IT management:
• organisational setup
• strategy
• business planning
• network infrastructure
• inventory management
• software applications
• security
• business continuity.
Organisational Setup
When reviewing the IT organizational setup, the IT audits would
analyse:
• roles and responsibilities of the auditee’s IT management
team
• possible bottlenecks and dependencies
• quality of process documentation
• management of IT processes through a review of:
–
–
–
–
Systems Development Life Cycle adopted for the implementation of IT systems
management/monitoring of IT maintenance contracts
monitoring of the service levels included in the above maintenance contracts
procedures adopted to procure IT hardware and provision of related hardware
maintenance services.
Strategy
The importance of having a formalised IT strategy with strategic
objectives based on the corporate vision for the organisation
cannot be stressed enough.
The IT audits carried out by the NAO would verify the availability
of such a strategy along with the allocation of management of
resources for the implementation of the IT strategy and the
ongoing monitoring/recalibration required during its life cycle.
Business Planning
In order to review the management of the auditees’ IT business
planning process, NAO IT audit teams would review the
following aspects of the process:
– IT Budget estimation process based on the list of current and new IT
projects/services required for the subsequent year
– Comparison between allocated IT budgets and actual IT expenditure
Network Infrastructure
NAO IT audits would typically include an assessment of:
• Local Area Network (LAN) and Wide Area Network (WAN) setup and
performance which would include a review of:
– WAN connectivity of auditee sites and related redundancy
– LAN Logical diagram to check for:
• Network transfer speed (eg 100Mbps)
• Use of UPSs for networking equipment
• Physical security of networking equipment
– Use of port locking
• Monitoring of service levels within agreements covering the provision of
maintenance services for the network equipment
• Level of LAN monitoring to establish router CPU, link and server disk
utilisation
Inventory Management
IT audits carried out by the NAO would review inventories for
• IT hardware,
• software applications
• software licenses.
Software Applications
Reviews of the software applications currently used by the
auditee and would typically cover the following aspects:
•
•
Clear identification of who is responsible for the operation of the system and the data within
the system
Monitoring of deliverables listed in the related system support and maintenance agreements
covering:
–
•
Review of the system functionality to assess:
–
•
•
•
•
•
•
rollout of system enhancements and upgrades and service levels for the resolution of system bugs and errors
alignment with auditee business processes, user friendliness, limitations, user satisfaction, overall perfomance
Availability of updated user manuals and programming documentation
Management of access controls when assigning user passwords, access levels and third party
access
Secure environment for electronic submissions of forms and on-line payments over the web
Availability of audit trails especially for the critical transactions within the system
Access to a report generator to produce the variety of reports required by management in a
timely fashion and with the required level of quality.
Frequency of scheduled system backups and test restores.
Security
The component of IT audits dealing IT related security covered
controls related to physical access, data security and antivirus
protection. In line with the above NAO IT audits would include a
review of the following IT security related items:
•
•
•
•
•
•
•
•
•
availability of an information classification policy
availability of an information retention and storage policy
Procedures for the disposal of IT equipment
antivirus protection and software patch updates
off-site storage of backup media
fire-fighting and intrusion detection in IT related strategic areas
implementation of a policy to control access to IT related strategic areas by visitors
coverage of CCTV monitoring of sensitive IT areas and handling of CCTV recordings
monitoring of physical access, temperature and humidity to the server rooms at the auditee
site.
Business Continuity
Business continuity is another important aspect of IT operations
which NAO IT audits delve into. The NAO IT audit team would
typically review the following aspects:
•
•
•
•
•
•
•
•
•
•
•
Availability of business continuity plans (BCPs) based on risk assessments which
would include:
Contacts list
List of essential hardware / software
List of essential information
Frequency of BCP updates
Availability of disaster recovery plans (DRPs) which would typically include:
Periodic testing of DRPs
Restore plans
Allocation of access rights following restores
Details for continued operation from an alternative site
Manual fail over process
Role of NAO in promoting IT efficiency
in Government
Building on the experience gained from executing the above
mentioned IT audits, NAO is now looking at widening the current
scope of its IT audits to include the following four IT audit topics:
1. Identification of anticipated benefits reaped from the investment made in
the procurement, implementation and operation of Government IT
infrastructure and systems;
2. Attainment of related targets listed in the auditee’s IT strategy and/or
national IT strategy;
3. Monitoring of key performance indicators (KPI’s) for IT projects and
operations;
4. Quality, reliability and transparency of financial reporting from existing IT
systems.
Identification of anticipated benefits
By far the identification benefits to be reaped from investing in IT systems
should be the building block for any sound business case to be made before
deciding to procure any IT system. The anticipated outcomes from
implementing an IT system should translate itself in a combination of
operational savings, shorter processing times and easier access to a public
service when reviewing e-Government systems.
Anticipated benefits could include savings which could be routed at procuring
the system in the first place ie making the project partially self financing. One
important issue with identification of savings is their segregation from other
funds so that they can be easily measured and used as originally planned.
Attainment of related IT strategic
targets
Anticipated benefits from implementing IT systems should assist
the auditee in attainment of one or more of the targets
identified in the auditee’s IT strategy.
The audit team would need to assess the level congruity
between the system benefits and the strategic targets. Due to
the nature of this study a multidisciplinary approach would be
required involving resources from IT, Performance and possibly
Financial Audit teams.
Monitoring of key performance
indicators (KPI’s)
In order to assess the overall performance of IT projects
undertaken by an auditee, key performance indicators (KPIs)
would need to be established and measured.
Audit teams would need to assess how the KPIs were
established and the type of monitoring being carried out apart
from verifying the actual levels of KPIs being attained for that
project. The performance indicators would need to be
benchmarked with other successful IT projects in that sector.
Quality, reliability and transparency of
financial reporting
Multi-disciplinary audit teams would need to assess the level of
reliability in the data within financial reports extracted from
current IT systems used by auditees.
This would require the input of both IT and financial experts in
order to assess the controls adopted to ensure integrity of the
data and that all the required audit trails and access controls are
in place.
Possible dependencies
Benefits of implementing an IT project to be clearly defined by
the auditee;
IT strategy at departmental level to be drawn up by the auditee,
with clearly defined targets;
Definition of KPI’s by the auditees, for current IT projects and
operations in their responsibility;
Integration of resources from NAO IT, Financial and Performance
Audit Teams to carry out joint audits
Conclusion
It is NAO’s role to bring about a greater awareness of the standards/best
practices on which any Government entities/departments should implement
its IT strategies, policies and governance rules. This can be done through the
IT audit methodology adopted by NAO putting focus on local and
international standards/procedures such as: ISO27000, ISO9001, COBIT, ITIL,
Prince2, Government procurement regulations, Government IT CAPEX and
OPEX business planning procedures .
Apart from the IT audits, NAO can promote IT efficiency in Government with
regular contact with Government Chief Information Officers in order to
increase awareness on related best practices and promote the idea of selfassessment. CIO’s on their part would have the opportunity to keep the NAO
in the loop on the current concerns in the sector.