Local Government Auditing Quarterly
Impactful Audit Practices - Planning
the journal of local government auditing

Fall 2012
LOCAL GOVERNMENT AUDITING QUARTERLY
Volume 26, Number 1
Published four times per year – in September, December, March, and June – by the Association of Local Government
Auditors (ALGA). Membership information appears on the inside back cover.
Association of Local Government Auditors
449 Lewis Hargett Circle, Suite 290
Lexington, KY 40503-3590
(859) 276-0686
algaonline.org
Board of Directors
Drummond Kahn, President
Director of Audit Services
Portland, OR
(503) 823-3536
[email protected]
Bill Greene, President-Elect
Acting City Auditor
Phoenix, AZ
(602) 534-9510
[email protected]
Kymber Waltmunson, Secretary
Senior Principal Management Auditor
King County, WA
(206) 296-0383
[email protected]
Corrie Stokes, Treasurer
Deputy City Auditor
Austin, TX
(512) 974-2468
[email protected]
Ross Tate, Past President
County Auditor
Maricopa County, AZ
(602) 506-1588
[email protected]
Tina Adams
Senior Auditor
Charlotte, NC
(704) 336-7270
[email protected]
Ruthe Holden
Chief Auditor
LA Metropolitan Transportation Authority, CA
(213) 922-1031
[email protected]
Pamela Weipert
Manager
Oakland County, MI
(248) 858-0994
[email protected]
Denny Nester
City Auditor
Colorado Springs, CO
(719) 385-5694
[email protected]
Publications Committee
Chris Constantin
LGAQ Editor
San Diego, CA
Amanda Lamb
Chair
San Diego, CA
Justin Anderson
King County, WA
Emily Jacobson
Denver, CO
Paula Ward
Washoe County School District, NV
Ruth Holden
LA Metro Transportation
Authority, CA
Corrie Stokes
Austin, TX
Margot Wilson
Austin, TX
Kristine Adams-Wannberg
Portland, OR
TABLE OF CONTENTS
From the Editor..............................................................................................................................................
President’s Message........................................................................................................................................
Opportunities for Improvement: How to Plan for the Best Audit Results......................................
AudiTechie: Using Technology to Enhance Audit Planning………....................................................
Fighting Fraud: Fraud Detection in Audit Planning and Fieldwork……...........................................
Professional Issues Committee……….........................................................................................................
Awards Committee……………......................................................................................................................
Did You Know? ALGA’s Member Demographics – Fall 2012……....................................................
Member News……………...............................................................................................................................
Professional Issues Committee: ALGA Perspectives on the New 2012 Draft of the COSO
Internal Control-Integrated Framework.............................................................................................
Audit Shop Profile: Portland, Oregon…………….....................................................................................
Audit Planning: Smart-Scoping: Audit ProgramsA Case Study for Small or
Resource Challenged Audit Groups………...........................................................................................
Audit Planning: Developing an Aviation Annual Audit Plan……………............................................
Audit Planning: From Boundless Surveying to Focused Planning………….......................................
Audit Planning: Creating Significant Impact through the Audit Planning Phase…………............
Audit Planning: Traditional Audit Planning vs. SCRUM………….....................................................
Audit Planning: Planning the Evaluation of the IT General Controls for
Government Entities…….........................................................................................................................
Audit Planning: Toward a Comprehensive Performance Scoping Model…………..........................
Audit Planning: Performance Audits: Success Is in the Design…………............................................
Quarterly Quizzer……………........................................................................................................................
Abstracts of Recently Completed Audits……………...............................................................................
How to Submit Abstracts and Quarterly Content…………...................................................................
1
2
3
4
6
9
12
13
17
19
20
23
24
27
30
33
35
38
43
46
51
52
65
From the Editor
Impactful Audit Practices –
Planning
T
he Publications Committee is excited
to present this edition which starts off
our Impactful Audit Practices Series with
Planning. Years ago, a former City Auditor
once told me “Proper planning prevents
poor performance.” Now ten years later, the
statement rings true.
CHRIS CONSTANTIN
Editor
Assistant City Auditor,
San Diego City Auditor’s
Office (CA)
There is a natural tendency to jump into
doing the “interesting” work such as data
analysis or drawing conclusions from
reviewing key documents. Why not?
When a large dataset arrives in Excel, there
are times I want to drop everything and
jump into analyzing the data. It is as if my
inner child came across a door which
opened into a proverbial candy store.
If we allow ourselves to follow the inner
child, we would find that we enter a
seemingly endless cycle of analysis and
review. When the cycle does end, we may
find we expended all or more of the allotted
audit hours and missed key questions
related to an audit objective. Planning is
the necessary activity meant to focus our
audit effort, uncover effective and efficient
ways to achieve the audit objective, and
ultimately, to tame our inner audit child.
Proper planning allows auditors time to
understand what activities would add the
most value, how to perform analysis which
addresses key audit objectives and reduce
the time required to complete an impactful
audit. In my opinion, the more time spent
planning will result in a better audit with
regards to audit impact and/or efforts
required to address audit objectives.
2
This edition includes many other expert
auditors’ views of audit planning. Take
advantage of their perspective.
Additionally, this edition includes other
content, such as our fact sheet called “Did
You Know.” The factsheet includes some
interesting tidbits about ALGA’s
membership demographics and geographic
dispersion. Take a look and see what
ALGA has to offer.
We hope you enjoy this issue of the
Quarterly. Our Winter 2012 Quarterly
issue will be on Impactful Audit Practices
Fieldwork. Articles are due on November
15th. Submit your articles to editor@
governmentauditors.org. Also, here are the
next two future ALGA Quarterly themes to
give you a head start on writing articles:
•
•
Spring 2013 due February 15:
Impactful Audit Practices – Analytical
techniques
Summer 2013 due May 15: How to
References – Compilation of step-bystep tools, techniques, and activities to
enhance performance auditing
LinkedIn: ALGA – Networking is
important to auditors, so if you haven’t
joined yet, it is time to join ALGA’s
LinkedIn group. We are now at 813
international members and growing!
Regards,
President's Message
Greetings ALGA Friends!
I
DRUMMOND KAHN
ALGA President, 2012-13
Director of Audit Services,
City of Portland (OR)
t is my pleasure to serve as your President,
and I’d like to thank you for your
continued membership and support of our
association and for the promise of local
government auditing. ALGA provides you
with unsurpassed advocacy, training, peer
reviews, publications, on-line resources,
and a global reach of professionalism from
our members across North America and
overseas.
ALGA’s logo shows a capitol building dome
surrounded by a triangle. The triangle is the
Greek letter Delta – and Delta is a symbol
for “change”. One of ALGA’s core purposes
– extending across our association’s 25year history – is to improve government
through auditing. This positive change
comes from our hundreds of member
offices, representing many kinds of local
governments conducting audit work that
improves our governments.
ALGA members work on both sides of the
Equator and both sides of the International
Date Line. We’re in cities, counties,
territories, and commonwealths – working
on tribal lands, in airports and fire stations,
at water districts and school districts.
ALGA committee conference calls can be
complicated to schedule because of the
time zones the call has to span (and this is a
good problem to have, since it reflects one
measure of the breadth of our membership).
ALGA membership means more
than having training and conference
opportunities, although these are critical in
our association’s success. Your membership
provides leverage and a part of the broader,
shared purpose of public-sector auditing.
ALGA is highly involved in standards-
3
setting, with representation on national
boards that set the rules. Where our
members can be impacted, our association
is there to explain ALGA’s position and
to both protect and defend the practice of
local government auditing.
Where public money is spent, ALGA
members conduct audits to ensure that
resources are used effectively and efficiently.
Wherever local government auditing
happens, ALGA is there to support, train,
inform, represent, and advocate for our
profession.
In closing, thank you for the opportunity
to serve as your President. I’m joined
by a highly-qualified Board of Directors
and Committee Chairs, all of whom are
dedicated to providing outstanding service
to our members and to our profession.
Improving government through auditing
is tough work, and we all face challenges.
With our ALGA colleagues, the path
is made clearer and the journey made
easier – from Advocacy and Peer Review
to Professional Issues, On-Line Resources
and Publications, Education’s work on
regional conferences and webinars, to the
Conference Committees’ work to stage our
annual conference, ALGA adds high value
to your operations and to our profession.
I’ll write in future messages about ALGA’s
partnerships and connections to the broader
profession – in the meantime, thank you for
your work to improve government and your
participation in our association.
Best wishes,
Opportunities for Improvement
How to Plan for the Best Audit
Results
S
omething is missing from the Yellow
Book. I consider it one of the more
important aspects of an effective audit, and I
wonder how other audit shops address it. I’m
talking about the preliminary survey.
GARY BLACKMER
Director
Oregon Division of Audits
Let me describe what I really mean by
preliminary survey because I have encountered
other audit shops that do something
different but they call it the same thing. In
my experience, the objective of the survey
is to identify and select the most significant
potential audit findings and gather enough
information to prepare the fieldwork plan. To
accomplish this, the auditors must develop
a good understanding of the organization’s
operations and any problems that adversely
affect its mission.
There are only a couple sentences in the
Yellow Book that talk about risk significance,
which is the closest it comes to preliminary
survey. There is another section on internal
controls that could determine the audit topic,
but I want to challenge your thinking on that
subject and will return to it in a moment. So,
while the Yellow Book commits many words
to every other aspect of the audit, the topics
to be audited seem to arrive spontaneously as
audit objectives.
Here is my theory: many audit organizations
are legislative, and as a result their audit
topics are often generated by the requests
or questions of legislators. The objective is
predetermined, as is the scope, and then the
auditors go out to develop the findings within
that frame. The idea of an auditor going out
"looking for trouble" is a luxury that many
audit organizations don't have. As a result, the
Yellow Book doesn't provide the guidance to
conduct a broad review of an agency to find
the area of greatest potential audit benefit.
I asked Marcia Buchanan, the GAO's Yellow
Book Maven, a couple months ago about this
gap and she said that the preliminary survey
is an 'audit technique', which can be addressed
in an organization's procedures but techniques
4
are not included in the Yellow Book. That
leaves many organizations on their own to
figure out how to choose an audit topic.
I've always worked for an elected auditor, or
been one, so the decision of what to audit was
determined by the audit organization itself,
though we considered legislative requests for
audits that would aid decision-making. That's
the luxury of independence, but it also comes
with the burden to choose the topic wisely.
But which agencies do you choose to audit,
and what areas do you choose to focus on?
Unless you have a crystal ball, or insider's
knowledge, it can be daunting to find those
potential audit needles in the agency haystack.
Of course you need a plan to organize your
search, but you always need to conduct some
basic steps.
Talk to people. Ask them open-ended
questions. Ask about what's working and
what isn't. Most importantly, get out on the
front lines and observe and ask the staff what
makes their jobs difficult. Often they suffer
from unfriendly computer systems, stupid
rules, lack of direction, poor communications,
bottlenecks in work processes, lack of training,
poor priority-setting, inadequate feedback to
employees about performance, unnecessary
paperwork, duplicative and excessive signoffs, and…I'm just describing some of the
conditions we've found in audits by observing
and asking.
These were living, breathing problems that ate
time and quality and costs. There is something
else embedded in these problems. They
all spring from a lack of internal controls.
Not compliance controls, but management
controls. Here is the sentence at the end of the
definition in the Yellow Book:
2.11 b. Internal control includes the
processes and procedures for planning,
organizing, directing, and controlling
program operations, and management’s
system for measuring, reporting, and
monitoring program performance.
Rather than plan, organize, direct, etc., I
prefer the simpler, time-tested performance
Opportunities for Improvement
(continued)
management framework for describing the elements
of management: planning, budgeting, managing, and
reporting. Lack of direction is poor planning and
objective-setting. Poor priority-setting is a budgeting
matter because resources do not get allocated to
the most important duties. Poor communications,
unnecessary paperwork, and a few others are just
poor management of processes and procedures.
Inadequate feedback and measures of results are the
reporting weakness.
organization has a breakdown in one of those
elements, it may produce adverse outcomes for the
public, which is a much more persuasive argument
for change. Sorry, the mere risk of an adverse
outcome is not enough in performance auditing.
There are hundreds of possible management controls
and far fewer resources to apply them, so unless you
can show a manager how some additional control
effort will reap greater rewards for the public, you're
just wasting everyone's time.
These are controls: It's management's responsibility
to set clear objectives and expectations for the
organization and each employee. They must also
allocate resources to conduct the work that best
achieves those objectives. They must establish work
processes and support systems to efficiently conduct
the work. And they must develop measures to track
results, as well as identify and diagnose problems.
Controls.
If you've successfully gathered the weaknesses about
the program you're auditing you should be able to
develop one or more potential audit findings. Prepare
simple descriptions of the elements of each finding,
and consider the risks and benefits of auditing each
finding. Choose the best one and begin developing
your audit objective, and your fieldwork plan.
Document why you chose it, and you have satisfied
several Yellow Book standards about risk and
significance. And that standard that says you should
consider internal controls as they relate to the audit
objective? You're actually conducting an audit of
those internal controls.
So, besides talking to the frontlines, talk to the
supervisors and managers to ask what kinds of
controls they have, but don't use that wordit's auditspeak, not manager-speak. "How do you track your
efforts and accomplishments? How do you manage
your workload? How do you assign staff to work
assignments, schedules, district offices? Can we get a
copy of your procedures manual? What are the most
critical duties/process you perform? What kind of
training do you provide to new employees?"
These questions help you put the pieces together
in that management context, to understand the
program risks. Talk to peer agencies, experts, and
partner agencies; study the documentation about the
program, expert literature, performance measures
from elsewhere; think about the interactions and
develop a diagram of cause-condition-effect when
you see problems.
How does it look from the client's perspective? Are
there patterns among the failures? Where do people
fall through the cracks? Are there failures when
people are handed off from one agency to the next?
Step back and think differently: if you were to design
this program anew, is this how it would look?
I want to emphasize that you don't have a finding
if you determine that an agency doesn't apply
performance management. The elements of this
framework are not criteria, they are causes. If an
5
Notice that the audit objective isn't developed in this
case until you've completed your survey, unlike the
legislative-driven audit topics. Rather than aiming
the audit at a target/objective you've had placed in
front of you, you get to draw the target around the
findings you plan to audit. At the end of the survey
you're also more likely to have a thorough, well
thought-out work plan that covers all the elements so
your fieldwork can be much more efficient.
Sometimes the survey comes up with several
potential audits. We have developed fieldwork plans
for each and chosen one to begin with then picked
up the other at a later date. Two or more audits from
one survey.
Here's one final argument for a survey: my worst
auditor worry is, just a couple weeks after an
audit is released, reading a newspaper headline
describing some big, ugly problem that we
didn't see because we had a tight scope from the
beginning. Sure I can say it was "outside the scope
of our audit" but that really sounds so pathetic.
A good survey increases your insurance against
missing that embarrassing finding. 
________________________________________
AudiTechie
Using Technology to Enhance
Audit Planning
supplement their efforts. Here are a few to
start with.
Automated tools are being developed
daily. Many can easily be adopted to
enhance audit planning.
Online Alerts
T
SCOTT JARRETT
Senior Law Enforcement
Auditor
Maricopa County
City of Phoenix (AZ)
he planning phase of a performance
audit can be challenging. Auditors are
tasked with gathering and reviewing useful
information, determining risk, identifying
internal controls, and focusing the scope of
a potential audit with an audit program that
will satisfy all stakeholders. The following
scenario is common for most performance
auditors:
You are assigned an audit of a department.
At first, you are very excited because you have
heard rumors or read articles about problems
that have troubled the department over the
years. Your coworkers are envious because
they know you have the next Knighton
award winner. You return to your desk and
reality sets in. You have no idea what the
department actually does, or why they do it.
You visit the department website, but find
only outdated pages and little insight into the
department’s operations.
ROSS TATE
County Auditor
Maricopa County
City of Phoenix (AZ)
As auditors begin their routine planning
steps (such as conducting interviews and
reviewing documents), technological tools
should be considered to enhance and
The days of searching the web for
information are gone. Now, the specific
information you want comes right to
your email account. Online alerts, such
as Google Alerts or Yahoo Alerts, can be a
powerful tool for audit planning.
Set up an alert for the specific areas you will
be auditing and let the power of the Internet
start your audit planning. Topics can be
broad or narrow. For example, an alert can
be set up for “Adult Probation,” “Maricopa
County Adult Probation,” or “Maricopa
County Adult Probation Recidivism Rates.”
The frequency of the alert can be adjusted to
meet your needs. To set up Google Alerts,
go to: http://www.google.com/alerts.
Online Alerts will search the Internet
(books, blogs, videos, newspapers,
magazines, newscasts, etc.) and email you
links to specific information about your
current or future audit topics.
Online Searches
Using Google or other search engines
has become a standard planning step for
most auditors. Search results provide a
vast amount of information. They quickly
identify emerging trends and issues about
SearchTips
Exact Phrase
Put quotation marks around the phrase. The
search “Internal Audit” will return all results with
the exact phrase “Internal Audit.”
Similar Words
To retrieve search results that contain similar
words or synonyms, put a tilde "~" symbol in
front of the word.
Exclude Words
Use a minus "-" sign in front of a word to exclude
any unwanted words.
File Type
Use the phrase “filetype:” to search for a specific
file type.
Use a string of two periods between numeric
ranges. “College Basketball Champions
1996..2005” to return all champions between
1996 and 2005.
Numeric Range
6
AudiTechie
a certain topic. However, if done carelessly, the
results can be overwhelming, invalid, and difficult
to use. Here are some tips to enhance your returns.
Google Searching Tips: If you do not find what
you are looking for, you may not be speaking
Google’s language. When speaking to Google,
eliminate punctuation; only use words, numbers,
or phrases that you want returned. Eliminate
words like “the” and “I.” Google has added
many features to help narrow search results. The
information in the preceeding table can be found
on Google's "tips and tricks" page.
Micro Blogging
Micro blogging tools, primarily Twitter, have
emerged as one of the most used communication
technologies. In some instances, citizens are
tweeting about emergencies instead of calling
911. But how can Twitter enhance our audit
planning efforts? Twitter can provide useful,
timely background information about an
organization. An auditor can search Twitter
feeds for conversations about a department or
audit topic. Department management may be
advertising performance information. Citizens
may be complaining about poor service. These
conversations may provide insight about
department personnel and reveal areas of risk.
Reporters will often tweet information that is not
published in any official source. Here in Maricopa
County, a local reporter tweeted the details of
Mobile applications, like New
Haven’s SeeClikFix App, prioritize
citizen feedback and provide
auditors with insight to emerging
issues.
7
(continued)
a trial for a County department director. The
reporter’s tweets provided insight into the inner
workings of the department’s culture and of its
external relationships.
Twitter also allows you to follow specific topics
using “Hash Tags.” (#your.topic.here). This will
filter your twitter searches and minimize the
amount of information you have to review. Hash
Tags are also a good way to identify emerging
trends.
Social Media
Performing a quick scan of social media sites like
Facebook or LinkedIn offers a unique opportunity
to gain insight into the lifestyles of the auditee.
Reviewing these sites may allow you to improve
your rapport during planning interviews.
As auditors we are also obligated to consider fraud
and identify potential red flags. By viewing the
social media sites and postings for key operational
employees, you may find photos of expensive
vehicles or exotic trips that may indicate they are
living beyond their means.
Mobile Applications (Apps)
Mobile applications are everywhere. Is your
organization using them? Data, especially citizen
data, can reveal dissatisfaction, abuse, or fraud.
Tapping into this source could assist auditors in
analyzing risk and determining audit objectives.
AudiTechie
(continued)
Interactive apps, like New Haven’s SeeClikFix
App, prioritize citizen feedback and provide
auditors with insight to emerging issues.
Surveying citizens through social media, like a
city’s Facebook page, can also provide valuable
information for audit planning.
Automated Surveys
Surveys are a great way to capture input about
an operation or department from customers,
employees, management, or other stakeholders.
With online surveys it is now feasible to reach
a target audience (large or small) and analyze
the results without exceeding your audit budget.
A couple of easy to use survey tools are offered
through Google Docs and Survey Monkey.
Maricopa County Internal Audit recently used a
survey to capture input for a procurement audit
from over 15,000 County vendors. The survey was
sent by email to all vendors, who were instructed
how to respond online. The online tool captured
the survey results and provided the details in
a format that was easily imported into Excel.
Depending on your needs, the survey formats can
be customized to include radial buttons, dropdown menus, check boxes, input boxes, and more.
Co-Sourcing
Leveraging specialists is a great way to expand your
audit knowledge and enhance planning efforts.
Using specialists for areas beyond your expertise is
also recommended by the Yellow Book. Specialists
can review planning documents, provide staff
training, and increase access to software or other
tools that are too expensive to purchase. Maricopa
County Internal Audit has used experts to write
scripts for data mining and audit programs for
specialty areas.
Predictive Analytics
More and more examples of using technology to
determine risk are becoming available. Predictive
analytics use statistical techniques and computer
programming to analyze current and historical
information to make predictions about future
events. When auditors are able to harness this
technology in their planning efforts, audit resources
can be focused more efficiently and effectively.
8
If you are a tennis fan, you may be familiar with
IBM’s SlamTracker system, which is being used
at major tournaments. SlamTracker leverages
historical and immediate data to predict player
performance and the outcomes of matches.
Los Angeles County uncovered more than 200
probable fraud cases related to child-care benefits.
Using complex algorithms, the program generates
risk scores derived from behavioral anomalies in
usage of child-care services. A social network
analysis tool within the system makes connections
between similar names, phone numbers, etc.1
New York City recently announced the
development of a “situational awareness”
platform that combines the city’s primary data
bases with predictive analytics.2 Government
auditors should be thinking about how these new
technologies can be used in audit planning.
Drones, Really?
The FAA estimates that 15,000 flying robots
will occupy the nation’s skies by 20203. Local
governments will surely be participating, given the
many useful applications of drone technology (e.g.,
infrastructure inspections, search and rescue, traffic/
crowd control, etc.). Auditors could use drones to
conduct inventories, inspect park trails, etc.
Conclusion
As new technologies emerge, auditors should always
ask themselves: (1) How can our government
benefit from this? (2) What risks are involved? and,
(3) How can our department use this?
Although audit planning can be challenging
(especially with limited resources), a little dose
of technology can go a long way. By applying
these tools and techniques, audit planning can be
enhanced by working smarter, not harder. 
_________________________________
1 http://www.governing.com/news/local/gt-child-
care-fraud-detected-in-los-angeles-county-usinganalytics.html.
2 http://www.emergencymgmt.com/safety/New-YorkCity-Law-Enforcement-Technology.html.
3 http://www.emergencymgmt.com/safety/CivilianDrones.html.
Fighting Fraud
Fraud Detection in Audit
Planning and Fieldwork
T
JASON HADAVI,
CPA, CFE
Assistant City Auditor,
City of Austin (TX)
he two sets of standards that local
government auditors typically follow,
the Red Book and the Yellow Book,
have become more explicit over the years
regarding auditors’ responsibilities related
to fraud. Both sets of standards include
fraud-related requirements for auditors
in assessing potential fraud risk as part of
planning, evaluating and testing identified
fraud risks, and reporting any fraud
identified.
Obviously auditors want to design processes
that help us meet these standards. The
process we’ve designed for our office helps
us meet both the letter of the standards,
and in our opinion the spirit and intent of
the standards. The process was designed
to help us detect fraud in our organization
(guaranteeing employment for CFEs) and
ultimately help our City better safeguard
public resources.
Over the years, leveraging the combined
expertise of our audit and investigations
staff, we’ve developed (and refined, and
refined some more) this process for
handling the fraud brainstorming and
detection requirements in the standards
within each of our audit projects. The
graphic below presents our process.
First, in the planning phase of our audit
we meet to review past allegations or
investigations. Ideally the lead on the audit
provides the investigators with background
information and preliminary objectives of
the audit and requests related allegation and
investigation information in advance of this
meeting. During the meeting, the audit
team and investigators discuss the planned
CORRIE STOKES,
CIA, CGAP, CFE
Deputy City Auditor,
City of Austin (TX)
*NOTE:WhileAustinisfortunatetohaveaninͲhouseintegrityunitthatconductsfraudinvestigations,
investigatorsthatyoucanconsultwithduringanauditcouldalsoincludeotherentitiesthat
performinvestigations(HR,lawenforcement,legaldepartment,inspectorgeneral,etc),aCFEon
yourstaff,orevenaninvestigatorinanothersimilarentity…anyonewhocantellyouaboutpast
investigations,hotlinecalls,orcontrolissuesandhelpyouthinkaboutpotentialfraud.
9
Fighting Fraud
(continued)
audit objectives and potential fraud risks given
the objectives using the information provided by
each party in advance of the meeting. We may also
use this time to develop customized fraud-related
questions to ask in interviews. Then the auditors
go out and do the planning work, including asking
fraud questions as appropriate, and come back and
summarize the results. At the end of the planning
phase, we do our risk assessment, factoring in any
information we’ve collected about fraud (and
all other pertinent information), and use that to
refine our audit objectives.
Next, in the fieldwork phase, we meet again with
the investigators to review the results of planning
and discuss the refined objective(s). If relevant
fraud risks were identified during planning, we also
discuss and develop control and/or detection tests
that would help detect fraud if it were occurring.
Then the auditors go back into the field and
execute the fraud tests and come back and discuss
the results. At that point we figure out what the
next steps (if any) should be regarding fraud.
Next steps may include referring issues that aren’t
really related to our audit to the investigators for
further work. Or, if we have sufficient, appropriate
evidence supporting fraud within the context of
our audit, we need to report it as a finding.
We don’t necessarily do every step in every audit
because for some audits we do not identify any
fraud risks up front and therefore do not come up
with tests to do in fieldwork. For example, in an
audit of an advisory board to City Council, there
may not be the potential for fraud (as they do not
have access to City resources and are not the final
authority for decisions). Or, we may come up
with some ideas in planning but then find they are
not related to the audit objectives once refined, so
we do not end up testing those controls. In cases
where we do not do detailed fieldwork testing, we
still hold the brainstorming session and document
our consideration of fraud risks.
EXAMPLE: Fraud-Related Planning for a Contract Monitoring Audit
Audit Objective: Evaluate effectiveness of key controls over contract management
Past allegations/cases:
•
Major embezzlement fraud by a contractor, involving falsified expenses and external audit
reports
•
Multiple examples of significant non-compliance with contract terms including waste of public
monies
Potential fraud risks related to the audit objective:
•
Risk of contractors falsifying payment requests and/or reporting documents to conceal
misappropriation
•
Falsified external audit reports providing false security
•
Inadequate contract monitoring activities allow fraud to go undetected (site visits not
conducted and external audit reports not verified)
•
Inappropriate employee/contractor relationships (conflicts of interest and potential for
collusion)
Fraud-related questions to incorporate into interviews:
•
Are statements made by vendors taken at face value or is additional documentation required
or additional analysis performed?
•
Do contract monitors perform site visits?
•
For contractors: How frequently does City staff communicate with you? What are you
required to report?
10
Fighting Fraud
(continued)
We’ve had decent success with this process in our
shop, leveraging auditors as a source for fraud
referrals. Several audit concerns have become
allegations for our integrity unit to investigate,
or risk response projects to proactively detect
fraud in a particular area. Both our auditors and
investigators have learned more about the other’s
function, and the improved coordination between
our auditors and investigators has helped us better
leverage our resources to make our government
more efficient, effective, accountable, and
transparent. 
_________________________________
EXAMPLE: Fraud-related Fieldwork for a Contract Monitoring Audit
Confirmed fraud risks:
•
Monitoring is not occurring consistently; paying invoices without verifying invoice validity;
not conducting site visits; not confirming background checks for sensitive positions; not
requesting/verifying external audits
•
Staff are responsible for providing technical assistance to the same vendors they are
monitoring for contract compliance
•
No guidance or training for monitoring activities
•
Specific contractors: one contractor had significant financial issues (not paying
subcontractors, potentially using City funds for unauthorized uses); another contractor had
no documentation demonstrating that services had been provided
Control/Detection Tests:
•
Confirm that external audits have been conducted and contact auditors directly
•
Conduct site visits to observe service delivery
•
Review payment requests including thorough review of supporting documentation (obtain
from contractor if contract monitor has not required adequate level of support)
•
Verify, on a sample basis, documents demonstrating service delivery and validating
expenses reported
Fraud-related Results and Referrals:
•
Overall, auditors determined that the department was not ensuring that services were
provided in accordance with contract terms and not adhering to contract management best
practices.
•
Two contractors were referred to the integrity unit for further investigation. One case was not
substantiated as no intent to defraud the City was identified and controls prevented payment
of ineligible expenses to the contractor. The other case was substantiated and ended with a
report highlighting waste of City funds by the contractor (despite receiving $58,000 from the
City for an arts-focused youth program, the contractor provided services to only one youth
compared to several hundred the year prior while under a different director).
11
Professional Issues Committee
T
he Professional Issues Committee
works on ALGA’s behalf to monitor,
evaluate, comment, and make suggestions
on proposed changes to auditing standards
by relevant professional organizations. This
work is significant, because the various
standards govern aspects of the work
auditors do. We strive to promote the
interests of local government auditors and
to provide value-added feedback to the
standards-setting bodies.
KRISTINE ADAMSWANNBERG
Chair, Professional Issues
Committee
City of Portland (OR)
The Committee meets every other month
through a conference call to discuss
upcoming exposure drafts and to review
and comment on those drafts. We compile
the committee members’ comments and
summarize them in a letter. The Committee
Chair sends the comments to the standardssetting body on behalf of ALGA. Once the
final standard is published, the Committee
analyzes it and provides the ALGA Board
with the impact our comments had on
the exposure draft. The standards-setting
organizations we monitor currently include
the following:
•
•
•
•
•
•
•
•
U.S. Government Accountability Office
(GAO)
Institute of Internal Auditors (IIA)
American Institute of Certified Public
Accountants (AICPA)
Information Systems Audit and
Control Association (ISACA)
Governmental Accounting Standards
Board (GASB)
International Organization of Supreme
Audit Institutions (INTOSAI)
Government Finance Officers
Association (GFOA)
Committee of Sponsoring
Organizations (COSO)
Currently there are 15 members on the
Professional Issues Committee, including
the Chair. The other members include:
•
•
Frank Alvarez, Broward County,
Florida
Lori Churilla, Allegheny County,
Pennsylvania
12
•
•
•
•
•
•
•
•
•
•
•
•
Andrew Clemmons, Washington
Metro Transit Authority
Ruthe Holden, LA County Metro
Transportation Authority
Allen Leatherwood, Central New
Mexico Community College
Tonia Lediju, City & County of San
Francisco, California
Helen Lew, Washington Metro. Transit
Authority
Kenneth Mory, City of Austin, Texas
Harriet Richardson, City of Berkeley,
California
Nicole Rollins, Jackson County,
Oregon
David Schroeder, City of Houston,
Texas
Lori Schubert, Waukesha County,
Wisconsin
Jeffrey Vargas, Allegheny County,
Pennsylvania
Pamela Weipert, Oakland County,
Michigan
These dedicated members spend significant
time carrying out the Committee’s work and
their diverse backgrounds and interests help
inform ALGA of a variety of professional
issues that arise. For each exposure draft,
there are often two or more committee
members that volunteer to do the review and
prepare responses on behalf of ALGA.
In the last year, the Professional Issues
Committee reviewed and provided
comments on a number of exposure drafts.
For example, the Committee commented
on COSO’s Internal Controls – Integrated
Framework draft, and provided testimony
on ALGA’s position on GASB’s Preliminary
Views on Economic Condition Reporting.
The Committee also sponsors webinars
during the year and presents at the annual
ALGA conference.
If you are interested in submitting
comments on any exposure drafts, please
contact the committee chair at Kristine.
[email protected]. 
_________________________________
Awards Committee
What Were the Judges Thinking?
A Behind the Scenes Look at the
Knighton Award Rubric
A
NANCY HOWE
Chair, Awards Committee
Lead Auditor, City and
County of Denver (CO)
KEN GAVETTE
Principal Management
Auditor
City of Portland (OR)
LGA developed the Knighton
Award in 1995 as a way to recognize
exceptional performance audit reports, and
to help audit shops continue to improve
their performance audit programs. Since
1999, the award has been given to shops
in each of three size categories – small,
medium and large. In 2005, the judges
began awarding Gold, Silver, Bronze and
Honorable Mention tiers within each
size category. Since the inception of the
program, ALGA has given out 109 awards
to 46 different audit shops.
Knighton Awards are presented for the
best performance audit reports each year,
but exactly what is meant by the “best”?
Reports submitted for award consideration
have one basic thing in common: they must
be conducted according to Yellow Book or
Red Book standards. Beyond adherence to
auditing standards, there is wide variation
among audit shops in what activities,
functions and programs are reviewed, how
audit work is conducted and how audit
reports are written.
This variation results in a wide array of
opinions on what constitutes an excellent
audit report. For example, audit shops
may report to a Council that specifically
demands short reports containing “only
the facts”. Other shops consider the public
to be a key audience and include extensive
background and other explanatory
information. Auditors who volunteer to
be Knighton Award judges come from this
wide array of shops and bring these different
perspectives to the judging process.
Because Knighton Award-winning reports
are so diverse, shops have expressed
uncertainty about how to write a
winning report. ALGA’s Awards Program
Committee (APC) has attempted to
13
provide guidance in the LGAQ (see Writing
Effective Audit Reports in the Summer 2009
Quarterly, and Knighton Awards: Then
and Now in the Fall 2010 Quarterly). Also,
writing Knighton Award-winning audit
reports was the topic of the May 2012
webinar sponsored by ALGA’s Education
Committee.
Because of the differences in audit shops’
mission, staffing, reporting structure and
culture, we have found that judges can
bring substantially different opinions as to
what constitutes a good audit report. To
step beyond guidance on writing an awardwinning report, we thought it would be
useful to focus on a tool the APC developed
to help guide the Knighton Award judging
process, and to provide some of the judges’
comments on reports submitted for the
2011 awards.
The APC Develops a Judging Rubric
The APC is charged with overseeing the
Knighton Awards process each year and
for continuing to develop and improve the
program. When we’re not working on the
judging process, we are working on ways to
improve the program, including increasing
participation. With increased participation,
there are more reports for judges to review
each year. Given that judges volunteer their
time to serve, the APC is always looking for
ways to reduce their time burden and increase
consistency across evaluations, while ensuring
the reports are adequately vetted.
For the 2011 awards year, the APC
developed a rubric in an effort to improve
the consistency and efficiency of the judging
process. A rubric is a guide that lists criteria
for scoring academic tests or projects. The
APC thought developing a rubric for the
already established five judging criteria
would be a good way to help judges focus
on what should be considered for each of
the criteria. It would also go a long way to
helping answer two enduring questions:
“What were the judges thinking?” and
“How could (fill-in-the-blank) have
Awards Committee
(continued)
possibly won?” The APC’s goal was to preserve the
inherent flexibility of the process by deliberately
leaving the criteria elements broad. We did not
want judges to consider the rubric elements to be
all-inclusive or use the rubric as a checklist.
Judges’ Application of the Rubric
Here are some examples of how last year’s judges
applied the rubric to each of the five equally
weighted criteria and how you might be able to use
the rubric in crafting stellar reports.
1. The audit scope has the potential for significant
impact, and is responsive to the needs and
concerns of decision-makers and/or the public.
Let’s face it, when we think of impact, most
of us think dollars first and foremost. And
the winning reports had huge potential dollar
savings; think tens of millions each! But money
isn’t everything. Public concerns over health
and safety also impress judges. San Diego’s
report focused on emergency medical services,
while Albany’s silver award winning report
sought to improve building code inspections.
2. Audit conclusions are persuasive, logical, and
firmly supported by the evidence, which was
gathered using appropriate research methods and
tools.
The peer judges are tough when considering
this criterion. Even some of the winning entries
received a few negative remarks from individual
judges, especially about the persuasiveness of
the reports. One thing judges do like to see
are good examples that back up conclusions.
San Diego, for example, concluded that the
City was not adequately monitoring their
contractor, and then immediately followed
that statement with a bullet list of examples.
Albany was given high points for clearly
linking objectives and findings. Their objectives
were listed and detailed, as were the findings.
As an indicator of persuasiveness, judges
also considered whether or not the auditee
concurred with the facts presented in the audit
and most or all the recommendations.
14
3. Audit recommendations that are feasible, and
will make government programs more effective
and efficient.
Again, the peer judges are tough when
considering this criterion. Judges did not
like to see recommendations that would be
expensive to implement or that could have
been improved with more communication with
the auditee. Toronto stated recommendations
clearly and directly; they were set apart using
shading and contained in each finding section.
San Diego actually ranked recommendations
by priority. An appendix explains the priority
ranking system so decision-makers have a road
map for action.
4. Audit results that are communicated in a clear,
concise way.
This is the criterion that considers what the
report looks like, and has led some critics of the
Knighton Award judging process to claim that
the award program smacks of a beauty contest.
But, like it or not, the clarity and conciseness
of an audit report directly affects the way it is
received by the public and decision-makers.
That could mean a short report that is concise
and persuasive, with just enough information
to convince a decision-maker to take action,
or a longer report with enough explanatory
background, photos, and charts and graphs
to communicate complicated subjects to
the general public. A long report can also be
concise.
Rather than cite individual examples here, we
encourage you to look at the winning reports
and pay special attention to the things the
judges say they liked, such as easy-to-read fonts
and clear sections and headers. Additionally,
photos can be revealing and are inexpensive
to include with today’s easy-to-use software.
If you can’t do anything else, try shading your
recommendations and placing them in strategic
locations, as Toronto did.
2011 Knighton Award Rubric
Criteria
1.
The audit scope has
the potential for
significant impact, and
is responsive to the
needs and concerns of
decision-makers and/or
the public.
Criteria Elements
x
x
x
2.
Audit conclusions are
persuasive, logical,
and firmly supported
by the evidence, which
was gathered using
appropriate research
methods and tools.
x
x
x
x
x
3.
4.
Audit
recommendations that
are feasible, and will
make government
programs more
effective and efficient.
x
x
Audit results that are
communicated in a
clear, concise way.
x
x
x
x
x
x
x
x
5.
An audit scope,
methodology,
recommendations, or
report format that is
innovative
x
x
x
x
x
x
x
Maximum
Score
Audit scope has the potential for significant impact. This means
the audit may result in:
¾ significant improvement in the efficiency or
effectiveness of the auditee’s operations;
¾ a significant budget savings for the agency or for the
jurisdiction (i.e., the taxpayers); or
¾ a significant positive impact on the
jurisdiction/taxpayers.
Information in the audit is used by decision-makers. For
example, lawmakers or agency personnel create or adjust laws,
rules, regulations, etc. in response to audit findings.
The audit impacts, or has the potential for significant impact,
taxpayer concerns. For example, significant health or safety
issues were identified.
20
Evidence was gathered using solid or innovative research
methods and tools.
Audit work and conclusions are clearly linked to the stated
objectives.
Conclusions are logical and persuasive because they are clearly
and consistently supported by the evidence and the audit work.
Persuasive language in the audit report helps convince the
reader of audit conclusions.
The auditee agreed with the facts underlying the
recommendations.
20
Audit recommendations are clear, specific and feasible.
Audit recommendations will make the agency or program more
effective and more efficient.
Recommendations clearly address the identified cause.
Recommendations are clearly and consistently linked to audit
conclusions.
20
Audit report is organized so the message is clear and evident
throughout the report.
Audit report contains a polished flow of ideas from introduction to
conclusion and recommendations.
Writing is smooth and easy to read, and sentence structure
promotes ease of reading.
Information presented in the report is fully relevant to the
conclusions and objectives.
The audit report is professional and clean, and strategically
incorporates breaks such as tables, charts, and white space.
Techniques, such as an executive summary and section
headings, are used to communicate results to busy readers and
decision makers.
20
Synonyms for innovative include: original, creative, inventive,
imaginative, futuristic, ground breaking, and cool.
Antonyms include: common, customary, familiar, ordinary, and
usual.
The audit had an innovative scope or topic.
The auditors used innovative audit methodologies to achieve
their objectives.
The audit resulted in innovative recommendations.
The report format or layout is original, and enhances the reader’s
ability to understand the audit results.
The innovation added value to the audit work or audit report.
20
Note–Theguidanceinthisrubricareprovidedtoensurejudgesconsidersimilarelementswhenevaluating
theKnightonAwardsubmissionsagainsttheestablishedcriteria.
15
Awards Committee
(continued)
5. An audit scope, methodology,
recommendations or report format that is
innovative.
This is the most subjective criterion, somewhat
in the eye of the beholder. Rather than limit
the judges’ perceptions, the APC decided
to use synonyms and examples to describe
“innovative”, rather than try to define it.
A unique methodology can be a winning
element. Portland Metro was singled out for its
use of a professional secret shopper to evaluate
the quality of food and beverage services
supplied by a contractor to some of their
convention and visitor venues.
In many cases, the judges found the subject of
the audit itself to be the innovative element.
San Diego audited a public-private partnership
to deliver emergency medical services, which
is one of the most unique in the country.
One judge gave Toronto “points for guts” for
targeting people in powerful positions at the
Community Housing Corporation, Canada’s
largest social housing provider.
Exhibits and photos, although not new to the
audit report structure, can still be powerful.
Instead of just criticizing management for not
monitoring the activities of a contractor, San
Diego used an exhibit to give specific examples
of activities the City could monitor to improve
performance. Albany used photos, not to
illustrate specific findings, but to provide
a backdrop of the City’s rich inventory of
historic buildings and to add visual interest.
16
The Future of the Rubric
Upon completing the judging process, ALGA
surveys the volunteer judges for feedback. In the
rubric’s first year, the judges’ overall opinion was
that the rubric was helpful. Ten of the twelve
judges who responded to the survey rated the
use of the rubric in guiding report evaluation
as “Good” or “Excellent”. Two judges rated it
“Average”, while none rated it “Fair” or “Poor”. The
rubric will not eliminate the inherent (and to some
extent, desired) subjectivity in the judging process,
but it appears to have helped guide the judges’
evaluations.
The APC will continue using the rubric for the
upcoming Knighton Award year. It will also be
on ALGA’s updated website this fall, and will be
included with the awards announcement package
so you can use it to help select the report you’d
like to submit for an award. If you’d like a copy
before then, please contact Ken or Nancy and we
will send it to you. As always, we welcome your
thoughts.
Serving as a Knighton Award judge is a fantastic
learning opportunity. It allows you to see
what auditors from other shops consider to be
exceptional audit work and audit reporting. It will
expand your horizons, deepen your understanding
of audit work, and add to your professional
credentials. So when ALGA comes calling for
volunteer judges, come on and join the fun! 
_________________________________
Did You Know?
ALGA’s Member Demographics  Fall 2012
A
LGA currently has 297
members; of these, 263 are
organizational members (audit
shops) comprising approximately
1700 auditors. As shown in the
table to the left, the majority of
ALGA shops are between 1 and 5
person shops (172 shops), with 3-5
person shops as the most common
membership category.
Membership
Category Count
1Ͳ2
3Ͳ5
6Ͳ10
11Ͳ15
16+
Associate
Individual
Lifetime
Retired
Total
69
103
48
19
24
17
1
14
2
297
As shown in the table on the right,
ALGA members reflect a variety of
local government entities. While
the majority of ALGA members
are cities and counties, ALGA
members also include other
entities such as school districts/
boards, transportation agencies,
utility districts, tribal governments,
and retirement systems.
ALGA’s membership is
geographically diverse as
well. While the majority of
organizational members (249)
are in the United States, 12 are
in Canada, and two are in U.S.
territories (American Samoa and
Guam). California and Florida
are tied for the largest number
of member organization in the
state, with 38 in each. See the
maps on following page for the
geographic distribution of ALGA
organizational members.
ShopType
AssociationofGovernments
AuthorityͲAqueduct
City
CityandCounty
County
DevelopmentCommission
DistrictͲHealthCare
DistrictͲSewer
DistrictͲUtility
DistrictͲWater
Judiciary
NativeAmericanCommunity
Port
RegionalGovernment
RetirementSystem
Schools
SchoolsͲBoard
SchoolsͲCommunityCollege
SchoolsͲDistrict
SchoolsͲUniversity
Territory
Transportation
TransportationͲAviation
TransportationͲConstruction
TransportationͲTollway
TribalGovernment
Utility
UtilityͲElectric
Total
Count
1
1
112
13
68
1
1
1
1
3
5
1
3
3
3
13
2
2
4
1
2
10
5
1
1
2
2
1
263
ALGA’s nine member board includes representatives from eight states who work for
cities (5), counties (3), and a transportation authority (1). For a list of current board
members, see the Board section of the ALGA website.
ALGA’s Membership Committee is responsible for efforts to recruit and retain
members. If you know of an audit shop that we should contact regarding potential
membership in ALGA, please contact the Membership Committee chair, Kip
Memmott at [email protected]. 
________________________________
17
18
Member News
Government Financial Manager. Dr. McCall
has served in local, national, and international
leadership positions. He is currently serving
on the Federal Accounting Standards Advisory
Board, FASAB (a nine member Board responsible
for establishing generally accepted accounting
principles for the federal government) as well as on
other national and local committees and boards.
Stockton’s Taylor Moves to Hanover
County
After 18 years at the city, Mike Taylor is leaving
his position as the City Auditor for the City of
Stockton, Virginia to become the new Director of
Internal Audit for Hanover County, Virginia.
Calgary’s Lewis and Norris Obtain
Certifications
All recipients of this award have been ALGA
members. Other ALGA members honored with
this prestigious award include:
Pam Lewis, CA and Senior Auditor for the
City Auditor’s Office in Calgary, AB, obtained her
CIA designation in May 2012. Pam has been with
the City of Calgary for over three years.
•
•
Jennifer Norris, Auditor for the City Auditor’s
Office in Calgary, obtained her CIA designation
in June 2012, after having been with the City of
Calgary for over four years. As of July 1, 2012,
she was also promoted from Auditor to Senior
Auditor.
•
Tallahassee’s McCall wins the David
M. Walker Excellence in Government
Performance and Accountability Award
for Local Governments
At the June 26-28, 2012 Intergovernmental
Audit Forum, Dr. Sam M. McCall, City Auditor
for the City of Tallahassee, FL, was awarded the
2012 David M. Walker Excellence in Government
Performance and Accountability Award for
Local Governments. The award is sponsored
by the National Intergovernmental Audit
Forum, and recognizes and honors government
audit professionals who have made sustained
contributions to improve government performance
and accountability through their leadership in
transforming government organizations. Dr.
McCall has worked for his entire career toward
improving government performance and
accountability. He worked at the Florida Auditor
General for 30 years, including 15 as the Deputy
Auditor General. During the last 13 years (1999
- 2012), he has been the City Auditor at the City
of Tallahassee. Dr. McCall is a Certified Public
Accountant, Certified Internal Auditor, Certified
Government Auditing Professional, and Certified
19
2010 – Gary Blackmer, former City Auditor
of Portland, OR (current Auditor Division
Director, State of Oregon)
2009 – Jerome J. Heer, Director of Audits,
Milwaukee County Department of Audit
2008 – Sharon Erickson, City Auditor, City
of San Jose
Professional Issues Committee
ALGA Perspectives on the
New 2012 Draft of the COSO
Internal ControlIntegrated
Framework
Introduction
H
KENNETH J. MORY CPA,
CIA, CISA, CMA
City Auditor,
City of Austin (TX)
ere we go again, after 20 years
the Committee of Sponsoring
Organizations of the Treadway Commission
(COSO) has released for comment a
2012 draft Internal Control—Integrated
Framework. The revision has generated a
significant number of comments including
some substantial ones from key professional
organizations. As a result, the release date
has been postponed from March of 2012 to
the first quarter of 2013.
Perceived Benefits of the Revised
Framework
The revised framework retains the original
five components of internal control (control
environment, risk assessment, control
activities, information and communication,
and monitoring) but incorporates
additional principles and attributes. These
additions are intended to provide clarity
in the design and development of internal
controls, support the assessment of the
effectiveness of internal controls, and help
entities operate with more agility and
confidence.
Furthermore, because the definition of
internal control and its five underlying
components are not changing, COSO
believes that the codification of the
principles will not impose additional
burdens or a higher threshold for designing
and maintaining an effective system of
internal control.
The new draft provides what COSO
believes is improved guidance and clarity
for a comprehensive risk assessment in a
number of areas. One of the most significant
enhancements is the highlighting of 17
previously embedded principles and their
related attributes. COSO believes these
principles and related attributes will assist
entities in applying judgment during the
design, implementation, maintenance, and
assessment of internal control.
COSO also believes the proposed
framework will provide organizations with
significant benefits, summarized in the
exhibit below.
In addition, COSO believes the framework
Benefits of the Updated Framework
Management
and Board of Directors
• Improve governance
• Expand use beyond
financial reporting
• Improve quality of risk
assessment
• Strengthen anti-fraud
efforts
Performance
• Adapt controls to
changing business
needs
• Greater applicability for
various business models
External
Parties
Confidence
Other
Users
12
20
Professional Issues Committee
should enable organizations to adapt to increasing
complexity and change of pace, mitigate risks,
and support sound decision-making. The updated
framework does not change core objectives or
definitions, but specifies 17 guiding principles
divided among the five components.
•
•
•
These principles remain broad as they are intended
to apply to for-profit companies (including
publicly traded and privately held companies),
not-for-profit entities, public sector organizations,
and other entities. For each principle, the
framework provides associated attributes
representing characteristics or points of focus.
The benefits of the updated framework include:
•
•
•
•
Combined principles and attributes as the
criteria to be used by management in assessing
whether there is effective internal control
Expanded reporting objective beyond
financial reporting to include non-financial
for both internal and external needs
Increased relevance of technology in today’s
environment
Clarification that the risk assessment process
includes risk identification, risk analysis (for
example, the probability of occurrence and
potential impact), and risk response (such
as how the risk should be managed, with
21
•
•
•
(continued)
acceptance, avoidance, reduction and sharing)
Clear links between identifying risks and the
achievement of an entity’s objectives
Considering risk within the overall entity
and within its sub-units (accounting, legal,
purchasing, and so on)
Incorporating risk tolerances into the
assessment of acceptable risk levels
Emphasis on the need for management to
broadly understand significant changes in
internal and external factors that may impact
the overall system of internal controls
Fraud consideration as well as risks related
to corruption and specific attributes in
identifying and evaluating such risks
Expanded discussion on governance relating
to the board of directors and subcommittees
of the board, including audit committees,
compensation committees, and governance
committees
The exhibit below provides a summary of the
codification of the 17 principles:
ALGA’s Response to the Exposure Draft
While there appear to be many benefits to the
new framework, the exposure draft generated
significant input from the user community,
including local government. The exhibit
highlights some of issues with the proposed
Professional Issues Committee
framework that were included in ALGA’s
Professional Issues Committee (PIC) March 2012
response.
These comments focus on identifying concerns
from a local government perspective, including
broadening the applicability of the framework
with regard to objectives, concepts, and
organizational dimensions. The response also
addressed clarity and preciseness in definitions
as well as applicability of concepts across the
framework. It also suggested the addition of new
categories of organizational structure to be more
inclusive of public sector structures.
Risk concerns were also a focal point of the ALGA
response. The comments again suggested tying
concepts together as well as clearer, more concise
definitions. ALGA also requested that COSO’s
framework consider the possibility of achieving or
exceeding objectives along with the risk of failing
to achieve those objectives.
22
(continued)
Conclusion
Given COSO’s extension of the release date for
the framework until the first quarter of 2013,
PIC is confident that COSO will give due
consideration to our concerns. By addressing
these, we believe the applicability and usefulness of
the new framework for local governments will be
enhanced. 
_________________________________
Audit Shop Profile
City of Portland, OR
Office of the City Auditor
http://www.portlandonline.com/auditor
(Front) City Auditor LaVonne
Griffin-Valade, Jenny Scott, Robert
Cowan, Fiona Earle, Kristine AdamsWannberg, Bob MacKay, Alexandra
Fercak. (Back) Ken Gavette, Director
Drummond Kahn, Janice Richards,
Beth Woodward, Kari Guy, Intern
Patrick Malloy, Martha Prinz. Missing
Tenzin Chophel.
City Auditor
LaVonne Griffin-Valade, City Auditor, MPA, CIA, CGAP
Drummond Kahn, Director, MS, CIA, CGAP, CGFM
Reporting Structure
The City Auditor is elected to a four-year term and is required to be a certified auditing
or accounting professional.
Office Mission
To promote honest, efficient, effective and fully accountable City Government. To fulfill
this mission, the Division audits and evaluates City operations to provide useful and
objective information to residents, City Council, and management.
Office Budget
$2.4 million
Organization Budget
$2.8 billion
Office Size
12 FTE
Staff Backgrounds
The office’s staff has a wide background, from service in local, state and federal
government to work for engineering and accounting firms.
Staff Certifications
CIA, CGAP, CPA, CA, CGFM, CISA, CCA
Standards Followed
Government Auditing Standards
Types of Projects
Performance audits, project management of the City’s financial audit contract
Projects Per Year
12 performance audits and the financial audit contract
Years as an ALGA
Member
25 years and a founding member of ALGA
Awards Received
Gold Knighton Awards in 2010, 2007, 2005
Silver Knighton Awards in 2008, 2006
Knighton Award and Special Project Awards in 1998
Gold Website Award in 2009, Bronze in 2011
Two Lifetime Achievement Members are from Portland.
Favorite Audit To
Date
We can’t pick just one. It would be like picking a favorite child! The big impact from
any report is making a difference to improve government!
Other Items of
Interest
Many members of the office serve in some capacity in ALGA leadership or on ALGA
committees.
23
Audit Planning
Smart-Scoping Audit
ProgramsA Case Study for
Small or Resource Challenged
Audit Groups
•
Synopsis
•
F
YINKA T. ALAO
Director/Compliance
Officer
Office of Contracting and
Procurement
Washington (DC)
ederally funded procurements, which
represented the bulk of the auditee’s
portfolio, were subject to multiple
layers of review within and outside our
jurisdiction – a fact, with implications,
that appeared to have been glossed over
for many years. Federal oversight of
the auditee's portfolio, which included
evaluations at the Congressional level by the
Government Accountability Office (GAO),
was exhaustive. Further, in the preceding
fiscal years, Congress had initiated GAO
audits with the stated goal of highlighting
needed improvements to the efficacy of the
Single Audit as well as in the day-to-day
administration of projects overseen by the
responsible federal agency. 1
Arguably, the scrutiny brought to bear on
this class of procurements and the standards
against which these had been evaluated
seemed to go above and beyond the
requirements of municipal regulations that
were in effect. After evaluating the business
context and our audit group’s constraints,
our sampling was scaled back to exclude
the above mentioned transactions for the
following reasons:
1. To Minimize the Duplication of Effort
and Maximize Limited Resources
It was determined that:
•
•
The federal agency provided
programmatic oversight of the
procurement process in accordance
with Federal Acquisition Regulations
(FAR);
Single Audits were performed annually
in accordance with the Single Audit
24
Act of 1984 and OMB Circular A-133
(Amend. 1996);
Local governmental audit
entities periodically sampled
these procurements as part of the
Comprehensive Annual Financial
Report (CAFR); and
The heightened level of Congressional
Scrutiny on the Single Audit and the
federal agencies' programs made further
reviews duplicative.
2. To Help Reduce the Administrative
Burden on the Procurement Organization
As noted above, the sheer volume of audit
requests can quickly overwhelm even the
best resourced procurement departments.
The extent to which an internal audit
operation can ameliorate bureaucracy by
instituting smart, concise and impactful
audit programs, the easier it will be to elicit
support for continuous improvement in the
manner in which the jurisdiction procures
its goods and services.
How it all Began
The internal audit team had been charged
with auditing the jurisdiction’s procurement
activity within the primary agency charged
with contracting, as well as satellite
operations with independent authority.
Before engaging in field work, it had been
the practice to engage auditees through an
internal control questionnaire to obtain
a “snapshot-in-time” view of the control
environment.
In response to an inquiry2, the audited
entity noted several exceptions to rules and
regulations that applied to its operations.
The most significant difference noted
was that certain projects exceeding $1
million were not reviewed and approved
individually but rather, in aggregate, as part
of an annual plan.3 It was also uncovered
that certain regulations shaped by socioeconomic policy objectives did not apply to
these projects.
Audit Planning
A Deeper Dive: Validation of Reported
Exceptions
Pursuant to a Memorandum of Agreement (MoA)
furnished by the auditee, the federal agency and
the jurisdiction agreed to pilot an approach to
expedite the process of making awards.
The audit team found the auditee’s assertions to be
consistent with the basic tenets of the agreement
which are as follows:
•
•
Annually, the auditee submits a plan
consisting of projects for review and approval
by the legislative branch; and
The legislative branch’s approval of the
plan (in aggregate) constitutes approval of
individual projects executed under the plan.
Other Considerations
The Single Audit Act of 1984 mandates an
annual review of State, Local Government and
other entities that administer or receive federal
grant assistance. As reported at that time, the
audit threshold/trigger was an aggregate of
expenditures of $500,000 or more. The auditee
estimated that eighty-five percent (85%) of its
portfolio fell into this category. Packages prepared
to fulfill requirements at the federal or local
government levels were the same excepting three
additional requirements evidencing compliance
with particular federal standards.4 The federal
entity reviewed these packages in totality
before approving the obligation of funds. We
also determined that the Federal Acquisition
Regulations (FAR) was the standard against which
these procurements were evaluated.
Further, the Compliance Supplement premised on
1996 amendments to the Single Audit Act and
revisions to the OMB Circular A-133 describes
fourteen (14) compliance requirements in
addition to program-specific standards that the
auditor must consider.5 Auditors are also required
to obtain an understanding of non-Federal
entity internal controls over Federal programs
and develop procedures to effectively test these
controls. In our view, the federal entity’s oversight
coupled with the annual Single Audit engagement
adopts and arguably goes beyond what could be
termed baseline adherence to local regulations.
25
(continued)
Non-governmental (industry) good practices were
also invoked in the evaluation of the efficacy of
these procurements. 6
Corollary, based on past experience, the duration
of the Single Audit is between 40-60 business days.
As reported earlier in that year, the responsibilities
of affected local governmental agencies
include, among other things, the gathering of
documentation, preparation and reconciliation
of data, and timely response to auditor requests
for information. Combined with similar
responsibilities in response to the Comprehensive
Annual Financial Report (CAFR), and a number
of engagements performed by local government
audit entities, the unavoidable bureaucracy can
quickly become administratively burdensome and
can present significant challenges in performing
timely, accurate and complete audits.
What We Learned
While the internal audit team had not abandoned
the possibility of periodically sampling
federally funded procurements, it was evident
that the auditee's transactions were subjected
to multi-tiered reviews within and outside
local government. Further, at that time, it was
determined that the standards employed in these
reviews, particularly as these relate to the Single
Audit, went above and beyond the standards
proffered by jurisdictional laws, regulations and
procedures.
Expediency drove my team to focus on locally
funded procurements. We believed that the
rationale for doing so was sound. Furthermore,
on a ‘go forward’ basis, we determined that the
triple constraints of scope, time and cost would
necessitate flexibility in our determination of the
approach and the extent of sampling to fulfill our
mandate. As-is, the existing depth and breadth of
oversight does not require the input of additional
resources.
Conclusively, a continuous assessment of risk
exposure, in-depth knowledge of the control
environment and verification of exceptions to
the rule, including oversight mechanisms, should
guide whether to pursue or constrain audit
activity. As I have experienced in the past, it is
Audit Planning
(continued)
difficult for audit operations of any size (more so a
small operation) to review one-hundred percent of
the programs under its purview. ‘Smart-scoping’ is
a necessary means to the ultimate end of efficient
risk assessment and mitigation. 
______________________________________
1 GAO-09-307R - Single Audit: Opportunities Exist
to Improve the Single Audit Process and Oversight
December 11, 2008 [Updated as of January 23,
2009]; GAO-09-751 - FEDERAL-AID HIGHWAYS:
FHWA Has Improved Its Risk Management
Approach, but Needs to Improve Its Oversight of
Project Costs
2 Question: Do you or any of your Agencies
procurement professionals deviate from the
documented procedures, regulations or guidelines
in performing Agency procurement activities? Are
there any notable exceptions/departures from
Municipal Regulations, Delegations of Authority,
Policies and/or Procedures?
3 Approval from the legislative branch of government
is required for every procurement $1 million dollars
and greater. This includes post award modifications
within fiscal year that would increase procurement to
or above the $1 million dollar threshold.
Modified 106; FHWA Approval - Form 1365
indicating the amount of money authorized or a nonparticipation statement) or Justification (for locally
funded projects); Proof that statement of work was
submitted to at least three (3) firms. Further at least
one of these firms must be certified Disadvantaged
Business Enterprise (DBE). Note: Any task order
package that does not have this documentation will
be returned without processing.
4
26
5 Activities Allowed or Unallowed; Allowable Costs/
Cost Principles; Cash Management; Compliance
with Davis-Bacon Act; Eligibility; Equipment and
Real Property Management; Matching, Level
of Effort, Earmarking; Period of Availability of
Federal Funds; Procurement and Suspension
and Debarment; Program Income; Real Property
Acquisition/Relocation Assistance; Reporting; Sub
recipient Monitoring; Special Tests And Provisions
6 Standards to be referenced in the Statement
of Work (SOW) as stipulated in Task Order
Directives include current design practices; FHWA
and American Association of State Highway and
Transportation Officials (AASHTO) requirements;
‘A Policy on Geometric Design of Highways and
Streets’ (Latest Edition); AASHTO Guide for the
Design of Pavement Structures’ (Latest Edition)
District of Columbia Standard Specifications for
Highways and Bridges (Latest Edition); Manual on
Uniform Traffic Control Devices (Latest Edition);
Institute of Transportation Engineers (ITE) Traffic
Engineering Handbook; Transportation Research
Board (TRB) Highway Capacity Manual; AASHTO
Standard Specifications for Structure Reports for
Highway Signs, Luminaries and Traffic Signals,
FHWA Traffic Control System Handbook; ITE
Manual of Transportation Engineering Studies; and
ITE Traffic Signing Handbook.
Audit Planning
Developing an Aviation Annual
Audit Plan
T
BARBARA COPPAGE
Deputy City Auditor,
City of Phoenix Auditor
Department-Aviation
Division, City of Phoenix
(AZ)
he City of Phoenix, Aviation
Department (Aviation) operates
Phoenix Sky Harbor International Airport
(PSHIA), one of the top ten busiest United
States airports by total passenger boardings.
Aviation also operates two general aviation
airports: Phoenix Deer Valley (busiest
general aviation airport in the county)
and Phoenix Goodyear. The fiscal year
2011 operating budget for Aviation was
$209 million and represents 9% of the
total city budget. Aviation’s 2011-2016
Capital Improvement Program (CIP)
totals $717.9 million and includes projects
for all three airports. The Aviation CIP
program is funded with Aviation operating
revenue, federal grant funds, Aviation
nonprofit corporation bonds and Passenger
Facility Charge funds. Aviation generates
approximately $309 million in revenue
annually.
The City of Phoenix City Auditor
Department (Audit) conducts audit
planning by attempting to identify the
Aviation audit universe, performing a
risk analysis, and obtaining input from
management relative to risks and controls.
27
Risk or control concerns identified by the
City’s Audit Committee, audit staff and/
or external auditors are also evaluated as
the plan is developed. Flexibility of the
plan is necessary in order to respond to the
changing needs of the City and Aviation.
Identifying the Audit Universe and
Performing a Risk Analysis
Identifying the Aviation audit universe can
be a daunting task and is by far the most
challenging aspect of audit planning. One
of the highest risks for an auditor is not
considering a material risk. For example,
not considering or including an audit
relating to one of Aviation’s largest revenue
streams. Therefore, when identifying the
audit universe, we involve management,
use several sources of information, and
consider the audit planning process “fluid”
by continuously gathering information
throughout the year, even after the audit
plan is finalized. For example, Audit
worked with Aviation staff during fiscal
year 2012 to identify risks associated
with the automated and manual parking
payment controls. Airport parking is
one of Aviation’s largest revenue streams,
generating approximately $70 million
annually.
Audit Planning
(continued)
Audit facilitates an Aviation-wide risk assessment
session every two to three years with a taskforce
comprised of Aviation staff (experts in their
field but typically not management). Through
facilitated discussions, the taskforce identifies risks
with and without associated controls and ranks
risks on a scale from one to ten. In fiscal year
2012, the taskforce identified 341 risks.
Understanding the Organization and
Business Structure
Understanding Aviation starts with reviewing
their organizational chart which shows their 14
lines of business. The budget for Aviation includes
the major construction projects in progress such
as the Phoenix Sky Train, new airport signage
and enhanced technology initiatives. Audit also
attends Aviation monthly staff meetings to keep
abreast of major issues affecting the department.
Airline meetings are also attended where Aviation
updates and provides financial and construction
information to airlines doing business at the
airport. Having an overall understanding of the
organization is another way Audit ensures our
audit planning covers the major areas of risk with
Aviation.
Using the city’s financial system (SAP), we
download financial information into data analysis
software, such as ACL, to identify high-level
trends or changes in revenue or expenditures
from year to year. ACL queries such as how many
contracts and vendors are used by Aviation and
how much is paid to each every year are also used
to identify other potential audit areas. Although a
great deal of information can be gathered through
SAP, there are times when information needs to
be clarified or put into perspective. This is where
management and staff interviews are helpful.
Interviewing Management to Gather
Information and to Discuss Fraud
Audit schedules one-on-one meetings with
Aviation management. Items discussed in these
meetings may include areas management identifies
as having potential internal control weaknesses,
unexplained decreases in revenues, large turnover
in staff, known or prior instances of fraud or even
28
to clarify their understanding of the audit process.
Although individual meetings may seem time
intensive, meetings as short as 15 minutes are still
productive.
In the past, Audit sent out surveys to management
inquiring about perceived risk or asking for
feedback on audits that would be helpful.
However, information gathered through surveys
tended to be high-level and its use was sometimes
limited in the audit planning process.
When time permits, including front line
supervisors in separate one-one-one meetings
generally results in a better discussion. These front
line supervisors are also aware of existing reports
and provide feedback that can be helpful in the
audit planning process.
Narrowing Down the Universe into
Potential Audits
Risk or control concerns identified by the City’s
Audit Committee, audit staff and/or external
auditors are also evaluated as the plan is developed.
With limited internal audit resources, leveraging
external audits ensures some audit coverage
without duplication. Other audit considerations
are the potential an audit has to reduce risk, save
money, increase efficiency, provide transparency,
and address fraud concerns.
Audit utilizes historical information to develop
the audit plan. With about 40 audits completed
each year at Aviation, tracking when audits were
last completed and which audit focal area (e.g.
expenditure, revenue, accounting and reporting,
information systems) the audit covered and the
scope of the audit is important when deciding
what audits should be included on the current
audit plan.
Creating a Preliminary Budget for Each
Audit
Audit planning budget development considers
how many hours are available and how to allocate
hours for each audit. To calculate available
hours, we consider factors such as the number
of hours that can be charged directly to a project
Audit Planning
(chargeable hours) and hours that are normally
charged to other things such as vacation, sick time,
training and administrative tasks (non-chargeable
hours). We also reserve approximately 25% of the
available chargeable hours to allow for unplanned
work and staff attrition.
In determining how to allocate hours for each
audit, we refer to historical information as a
starting point. Other factors are also considered,
such as a significant finding in a prior year audit
that warranted a larger budget, audits that are
not routinely performed, or historical audit
information that is not available. In these cases, we
budget hours for each audit based on the resources
Audit is willing to spend to address the relative
assessed risk for that area. The individual audit
budget may be adjusted during the year or upon
completion of the audit survey phase. Our audits
are relatively narrow in scope and budgets average
about 250 hours. Each year, auditors complete
approximately seven audits that take an average of
five months to complete.
Management Buy-in
Audits are categorized by high, medium or
low risk. Although Audit makes every effort
to assess risk accurately for each audit area, the
assessment is often as much art as science. To
gain additional insight, Audit seeks feedback
from the Aviation Audit Working Group (AWG).
The AWG is comprised of members from the
Aviation management team. Audit meets monthly
29
(continued)
with the AWG to discuss the progress on open
audits, relevant current issues, and the status of
outstanding audit recommendations. The AWG
also reviews the draft audit plan, which includes
all the audits with the associated financial risk
(e.g. contract values, revenue streams), if available.
The AWG identifies riskier areas, notifies Audit
of scheduling conflicts, contract concerns or
justifications why an audit should be rescheduled.
Their valuable insight in the audit planning process
ensures the rest of the year runs smoothly when
individual audits are opened. Although feedback
from management is received, it is still up to Audit
to finalize the audit plan.
Conclusion
There are many challenging aspects of developing
an annual audit plan, such as identifying the audit
universe, gathering and analyzing information
and assessing risk. Involving management in
the planning creates an atmosphere of trust.
Management involvement also creates buy-in and
leads to management more openly and willingly
sharing known risks and perceived controls.
Working together with management ensures Audit
accomplishes its mission.
The City Auditor Department provides
independent and objective feedback to assist
City management in reducing risk, meeting
organizational objectives and efficiently managing
public assets. 
______________________________________
Audit Planning
From Boundless Surveying to
Focused Planning
A
NIKI RAGGI
Niki Raggi, Office of the
City Auditor, City of Austin
(TX)
few years ago, Corrie Stokes and I
wrote an article about risk assessment,
which is an essential tool used to guide our
planning phase. Since then, our office has
further improved and refined the planning
phase. The results of this process include
increased efficiency and more timely
and relevant outputs. While it would be
presumptuous to give credit for such results
only to refocusing the planning phase, I
believe this process played a significant role.
The Old Way
Before I tell you all about our current
planning process, let me tell you how we got
there. Until a few years ago, our planning
phase was quite lengthy and broad scoped.
It was comprised of two sub-phases: the
pre-survey and the survey phases. The
former consisted in gathering background
information about the audited topic and
kicking off the audit, both internally and
with the auditee. The latter consisted of
learning about the audited topic, and
identifying related risks and vulnerabilities.
Under that umbrella there was very little to
curb auditor curiosity. Often, this process
was leading to learning a lot, often too
much, about the topics related to the audit
objective. Indeed, we were learning so much
that some of us even dared going to our
manager to announce that we did not need
to conduct fieldwork, as we had performed
so much work in survey that we could just
write the report!!!
Let me use a personal example to better
explain how we ended up in such a
situation.
The Infamous Extra Bag Test
While conducting an audit of the City
utilities’ billing system (which includes
electric, water, garbage, and other
miscellaneous charges), an audit team under
my leadership, looked at (better surveyed)
every aspect that was remotely related to the
billing process; including the billing of some
ancillary charges.
In populating our risk matrix, we had
observed that there were some inefficiencies
related to the billing of extra garbage bags
(refer to information in the box). Moved by
the desire to better understand the risks and
vulnerabilities associated with this process,
on a dark, rainy morning, at around 5 A.M.,
a small group of auditors set out to precede
a few crews on their garbage collection
routes, with the objective of identifying
extra garbage bags that did not have a
sticker, and determine if they were properly
billed.
Performing this walkthrough showed
that most of the time extra bags were
billed properly. We had identified a few
exceptions, but nothing too out of line. In
the end, we had spent a good chunk of our
planning hours looking into an issue which
was quite immaterial and marginal to the
process of billing utility charges.
In Austin, we have different sized garbage carts. The smaller the cart, the lower
the monthly cost. As such, occasionally customers have extra bags that do not fit
into their cart. If you happen to have one or more of these bags, you could either
purchase a sticker at the grocery store (for $4 per bag, at the time of our audit) and
place it on the bag to be left out next to the cart, or you could leave your extra bag
sans sticker next to your cart and be charged for it on your next utility bill (at $8 per
bag). In order for the $8 to be billed, the garbage crews needed to make note of the
extra bag(s) on a spreadsheet listing all the customers on the route, and then pass
this along information to the billing group in their department.
30
Audit Planning
A Better Way
When our office transitioned to a paperless
environment, we had to think of how to customize
our brand new automated work paper system
to our processes; so we first took a good look at
our audit process, our policies and our practices,
and concluded that there was some room for
improvement.
The Power of Words
As I mentioned at the beginning of this article, our
planning phase was comprised of the pre-survey
and the survey phase. As hinted at above, the very
name survey had the power of evoking in some of
us the desire to examine, inspect, and scrutinize
the audited topic at-large, with almost limitless
boundaries. So, the first change we made was to
combine pre-survey and survey and call it simply,
”The Planning Phase”.
Efficiency through Templates
Also, looking at our practices, we realized that
there were many inconsistencies in the way
teams were carrying out planning. Despite the
existence of checklists and templates, our planning
documents were varying significantly in content,
form, and in the level of detail. Basically, we were
re-inventing the planning steps for each audit.
We realized that creativity was really better spent
designing appropriate fieldwork steps than in
executing the planning phase of the audit, and thus
adopted a checklist approach to planning which
included the systematic use of templates. In fact,
while each audit is different, using the templates
and standard procedure called for in the Yellow
Book, allowed us to capture the information
needed to assess risk and vulnerability and narrow
down the audit focus for fieldwork.
WHAT ARE THE AUDITED
ENTITY'S KEY
OBJECTIVES, WHICH
RELATE TO THE AUDIT
(continued)
A More Focused Approach
In addition to the change in terminology and
requiring the use of templates, we tweaked our
approach to ensure that the audit focused on the
most relevant issues. The flow chart below is a
high-level overview of our revised approach to the
planning phase. This phase continues to involve
a formal assessment of risks and vulnerabilities
relevant to the preliminary audit objectives, as
well as the development and refinement of the
fieldwork objectives, scope, and methodology.
However, it is now driven by concentrating time
and efforts on key issues only: key objective, key
processes, key risks, and key vulnerabilities.
In summary, once you are given an audit objective,
you should first focus on identifying the auditee’s
key strategic objectives, as they relate to the
given audit objective. Then, once you have an
understanding of these key objectives, you work
on identifying the key processes related to these
objectives. And then, only then, you look in depth
at risks and vulnerabilities, but only at those which
are key to the processes identified as being critical
to the auditee’s key objectives.
By concentrating on the key processes and the risks
related to those processes, we ended up performing
audits that really mattered to the auditee and the
organization overall.
Providing a Context
As indicated above, it was not unusual to get lost
in the amount of information gathered in our
survey phase and ultimately lose track of what we
were doing. Knowing how easy it is to get diverted,
we introduced one more change. At this point,
we had adopted a new terminology, developed
WHAT ARE THE KEY
PROCESSES NEEDED TO
ACHIEVE THE KEY
OBJECTIVES?
WHAT ARE THE KEY
RISKS AND KEY
CONTROLS RELATED TO
THE KEY PROCESSES ?
OBJECTIVES?
31
Audit Planning
(continued)
templates, and identified a more focused approach.
But we still wanted to provide a more intuitive
context for the planning procedures, and thus
identified seven essential questions to be asked and
answered while planning an audit. Each of these
questions is tied to a planning procedure, and each
planning procedure is tied to its relevant tasks or
steps. The question should tell you the rationale
behind each procedure, and the procedure, and its
relevant steps, should provide a roadmap of what
to do.
Our questions and related procedures are
summarized in the exhibit below. As we move
through these procedures we add the information
gathered to our risk and vulnerability matrix
which we review as a team at the end of planning
to determine where to focus our fieldwork.
Going back to the extra bag issue, had I applied
this focused approach to planning, I would
have recognized early on that it was not a key
process of the billing system, and would not have
wasted precious hours going down that path.
It would have been sufficient to communicate
to management the concerns regarding the
inefficiencies noted, and focused the audit from
the beginning on the most relevant processes.
Adding Value
Since implementing the changes described in this
article, our office has significantly reduced the
number of hours spent planning our audits, thus
leaving more time for diving into the issues carried
out to fieldwork. Also, by focusing on those key
risks/controls within key processes, we now focus
the bulk of our work on what really matters, or
should matter, to management. Further, our audits
provide more timely and relevant information to
decision makers. 
______________________________________
Exhibitfor“ProvidingaContext”section
PlanningQuestion
1
Whyarewedoingthis
audit?
Whatdowealready
knowabouttheaudited
entity?
Whataretheavailable
criteriawecoulduse?
2
3
4
Whatdoweknow
abouttheoperationsor
programthatweare
auditing?
5
Whatarethekeyrisks
relatedtotheaudit
objective?
Whatarethecontrols
overthekeyrisks
identifiedabove?
Howcanweaddvalue?
6
7
PlanningProcedure
Gainageneralunderstandingoftheaudit’spurpose
Identify,gather,andreviewpriorworkrelatedtothe
auditobjectives
Researchandidentifycriteriarelatedtotheaudit
objectives
Gatherinformationabouttheoperations,program,
andprocessesbeingauditedtoidentifykey
processesthatmayaffecttheoperationsand
programsbeingauditedandtoreviewsourcesof
evidence
Gatherinformationaboutthekeyrisksassociated
withtheoperations,programs,andprocesses,and
evaluatepotentialsourcesofevidence
Usingthekeyrisksasaframework,gather
informationaboutkeycontrols
Conductanoverallassessmentofrisksandcontrols
32
ExamplesofProcedureSteps
Reviewannualauditplanandmeetwith
otherswhosurfacedtheriskand/oraudit
staffwhoperformedworkinthetopicarea
Reviewpriorauditsontopicareas;
identifypriorrecommendations
Reviewrelevantlaws,regulations,
contracts
Reviewavailabledocumentation(suchas:
budgetdocuments,businessplans,
organizationalcharts);identityandanalyze
keydatasourcesandkeyITsystems
Considerriskoffraud,waste,andabuse;
analyzedocumentationobtained;
interviewkeypersonnel.
Performwalkthroughsandobservationsof
relevantprocesses
Summarizeandrankinformationgathered
onrisksandcontrols;anddevelop
fieldworkobjectivesfromRiskand
VulnerabilityMatrix
Audit Planning
Creating Significant Impact
through the Audit Planning
Phase
W
BRIAN HARTMAN, MPA
Lead Auditor,
City and County of Denver
(CO)
DAWN HUME
Audit Supervisor
City and County of Denver
(CO)
hen starting an audit engagement,
municipal performance auditors have
numerous potential issues to explore using
limited resources. Any performance audit can
have impact, but with careful and strategic
planning, auditors can create performance
audits that result in lasting positive impact
for government entities and constituents
alike. Auditors use the planning phase to
educate themselves about the agency or
program they intend to audit, including but
not limited to reviewing applicable policies,
procedures, regulations, statutes, financial
and performance data, and conducting
interviews. However, for audits to have
significant impact, auditors should consider
other key planning elements. First, gaining
an understanding of the political climate of
an agency or program during the planning
phase is not only acceptable, but it is crucial
to ensuring that the audit addresses the wide
variety of issues that matter to the audit’s
readers and proposes feasible solutions
that are useful to policymakers. Second,
the breadth of the audit scope should be
manageable and focused on relevant issues.
Third, a preliminary audit message should be
developed to help guide the audit planning
activity. Finally, sufficient information should
be collected to create a realistic plan for
fieldwork.
Consider the Political Climate
The policy environment surrounding the
audit work you are about to start is vitally
important to ensuring an audit’s relevance
and potential impact. Accordingly, the
audit team will need to conduct interviews
and gather information not only from
key individuals within the audited entity,
but also from stakeholders external to
the audited entity. Sources of external
stakeholder information could include
interviews with members of the city council
or the county commission, minutes from or
33
broadcasts of the governing board’s recent
meetings, and discussions with management
from entity-wide agencies such as a budget
office. Additionally, local, state, and national
press coverage of the audit topic or entity
can help the audit team evaluate which
particular issues may be important to the
community and municipal leadership.
Other personnel in your office may also be
able to provide valuable context regarding
current civic events or issues related to
your engagement. For example, the Denver
Auditor’s Office employs a government
liaison whose responsibilities include
monitoring activities of and communicating
with elected officials and external political
stakeholders. By understanding the political
climate, the audit team will be better
prepared to focus on items in which both
citizens and policymakers are interested and
propose solutions that are more likely to be
implemented, which in turn can result in
significant positive change.
Keep the Audit Scope Narrow
A broad audit scope can detract focus
from relevant issues by including too many
audit steps, many of which may not lead to
significant impact. However, limiting your
scope can help your team be more thorough
in its analysis of the issues you choose to
address. Determine which section of an
agency or program your team wants to
examine based on preliminary information
you have collected. For example, rather
than leaving your scope open to all activities
and processes of the entire public works
department, limit your examination of risk
and fraud to the wastewater function or
street maintenance activities.
In addition to the team’s preliminary
knowledge of the topic, audit management
may be able to provide useful information
that will help you determine reasonable
limits for your audit scope. In some
instances, management may direct that the
audit should target something specific, such
as the financial management practices of
a particular department. A risk assessment
Audit Planning
(continued)
can help the team further narrow the scope, by
identifying potential risks to the selected section
of the agency or program in question. While
establishing the audit’s scope, keep in mind what
you know about the political climate of the topics
at hand. Appropriately limiting your scope will also
limit what the audit team will have to investigate
during the fieldwork phase and make it easier to
develop a strong, clear message during the audit
reporting phase.
Use Questions to Create a Potential Audit
Report Message
After the audit team has agreed on an appropriate
scope, test it by creating potential report messages
that could result from the established scope. Begin
by crafting some high-level researchable questions
to guide the audit planning process. Focus on
two to four major questions the audit needs to
answer. Using the preliminary information you
have gathered through research and interviews,
create potential answers to your questions. The
idea behind this exercise is not to write the report
before performing fieldwork, but rather to help the
team think about effective ways to frame the audit.
As the team begins to develop a plan for fieldwork,
keeping these questions and preliminary answers
in mind can help the team better determine
exactly what type of evidence they need to support
answers to the questions.
Naturally, as you perform fieldwork your
preliminary answers will change based on the
facts as you learn them. Furthermore, the major
questions you hope to answer can evolve as the
audit team learns more about the agency or topic.
The final scope you establish for the audit is the
basis for the report’s final message. Creating
potential messages during the planning stage can
help the audit team identify early in the audit
process which issues are most relevant or likely to
create significant impact once addressed. Thinking
about the report’s message during planning can
also help the team begin to consider the report’s
ultimate message throughout the fieldwork and
reporting phases.
Collect Sufficient Information to Create a
Meaningful Plan for Fieldwork
The team should collect as much information
34
during the planning phase as is reasonable to
adequately determine which audit steps to execute
during fieldwork. Audit steps can be as general or
specific as the team deems necessary to fulfill the
audit objectives. However, it is important that the
audit steps do not contain activities that cannot be
performed due to limitations beyond the team’s
control. For example, if an audit step specifies that the
team will perform an analysis of certain permitting
data, the team should be sure that there are adequate
data available to perform the analysis as stated.
In other words, find out from the audited entity
as much as you can about what information they
have available for you to conduct fieldwork. Don’t
be afraid to ask for things that you are not sure
you will need for the audit, and don’t be afraid to
perform analyses or interviews during the planning
phase that might also qualify as fieldwork. As long
as the audit team is mindful of the time required
of the audited entity to fulfill the requests being
made and the work assists in planning, such work
is appropriate during the planning phase. With
enough knowledge in hand, the audit team can
plan for the engagement with actionable items and
can avoid committing to impractical or irrelevant
methodologies that can take up time during
fieldwork.
Don’t Hesitate to Devote More Time to the
Planning Phase
Taking all of the above into consideration, the
planning phase of a performance audit can be
the most critical part of an audit. The scope and
methodology established for the audit can make
or break fieldwork and reporting. Planning can
also set the audit team up to succeed or fail from
a time- and resource-management perspective.
Without adequate time and resources devoted to
planning, your team can produce an unmanageably
large scope with irrelevant or impractical
methodologies, all of which can make developing
a clear message for your findings very challenging.
Making a concerted effort to understand the
political climate, limit the audit scope, and gather
sufficient information early in the engagement, your
audit team can ultimately create a clear message that
will produce an audit report with greater meaning
and impact for all stakeholders. 
______________________________________
Audit Planning
I
KYMBER WALTMUNSON
Principal Management
Auditor, King County (WA)
n the winter 2010 Quarterly, Kymber
wrote about how to use Scrum, a
project management technique widely
used for IT and engineering projects, to
plan audits. As many of you are aware, this
article caused shockwaves throughout the
local government auditing community
(shockwaves may be a little extreme, perhaps
it caused mild turbulence). In light of the
Quarterly’s focus on audit planning, we
wanted to revisit these ideas and contrast
them with traditional audit planning
(TAP), which many of us are accustomed
to using. In order to maximize bombast
and minimize insightful discussion, we
are using a “crossfire” style of back and
forth arguments and rebuttals, with Ben
supporting TAP and Kymber advocating for
Scrum.
Description of Planning Methods
BEN THOMPSON
Principal Management
Auditor, King County (WA)
Traditional Audit
Planning (TAP)
Whenever I start
an audit, I think
about a quote
from Confucius:
“A person who does not think and plan long
ahead will find trouble right at his door”. To
conform to this advice, I have used what I
am referring to as traditional audit planning
(TAP). TAP is a multi-phase process
consisting of a series of basic steps, including:
•
•
•
•
Clarifying the issue – to understand
the purpose of the audit from the
requester’s point of view;
Understanding the context – to
determine what prior work, if any, has
been done in this area and the historical
context of the program or agency you
are auditing;
Identifying the objectives of the audit
– to identify what information you will
need to gather;
Establishing a preliminary scope and
methodology – to determine the types
of questions you will be asking, from
whom, and perhaps most importantly
35
•
what you do not need to know; and
Identifying risks and potential limitations
– to understand when starting the
work issues or risks to the audit work so
that you can accurately communicate
limitations to management and clients
and avoid creating unreasonable
expectations of the work.
By going through this process, I have to be
able to create a relatively detailed project
plan which allows me to scope the audit, to
estimate resources required (staff time, travel
funds, etc.), to identify milestones, and to
provide an overall roadmap for the project.
Scrum
I usually find
trouble at my
door when Ben
comes knocking,
regardless of how
much I plan ahead. My favored approach to
project management is an iterative planning
approach called Scrum that was initially
created to manage software development.
Scrum is a project approach I have been
successfully using for three years now.
In Scrum, there is a cycle of progressive
elaboration that is represented in this
graphic. Here are the basics:
1. The team meets to plan two weeks of
work (the core “sprint” shown in the
larger, 1-4 weeks cycle in the graphic).
2. At this meeting, the team selects a large
chunk of work (the selected product
backlog that you see moving into the
Sprint Backlog pile in the graphic)
that they have determined is the most
important to achieve their project goals.
3. The team identifies all the tasks
required to finish the chunk of work
(the shorter pile called the sprint
backlog in the graphic).
4. The team then completes each task,
meeting daily to make commitments
and discuss progress (daily Scrum,
shown by the 24 hour cycle in the
graphic).
Audit Planning
(continued)
5. At the end of the two week sprint there is a
tangible output (shippable product).
Scrum is especially effective in the complex,
ambiguous, and rapidly changing environments
that auditors face.
Comparison
The core difference between the two methods for
planning includes a full, complete, committed-to
plan at the beginning of a project versus a list of
probable key issues that are continuously reviewed
and refined based on priority and perceived value.
To compare these two planning methods, we
thought of different audit scenarios, determined
the strengths and weaknesses of each method for
the scenarios and chose which method was most
effective (see next page).
Consistency with Yellow Book Standards
We are going to focus on performance audits
conducted under Yellow Book standards, as it is
that with which we are most familiar.
The Yellow Book provides auditors a fair degree
of latitude in audit planning and could even be
seen as endorsing agile techniques such as Scrum,
when it states, “Planning is a continuous process
throughout the audit”. However, it also presents a
number of requirements that auditors must meet
during the planning process including: adequately
planning and document the planning of the work
necessary to address the audit objectives, assessing
audit risk and significance, identifying potential
36
criteria, and preparing a written audit plan.
Scrum can meet all the requirements of the Yellow
Book in two sprints: a research-focused sprint and
a planning-focused sprint. The goal of the research
sprint would be to learn as much as possible about
the audited area including reviewing possible
criteria and relevant literature and gaining an
understanding of potential audit risk. In the
planning sprint the team would develop an initial
product backlog that would take the form of an
audit plan with the understanding that the backlog
will be refined and will become more specific as
the project moves forward.
TAP, when done correctly, addresses these
requirements, however (at least for audits I have
participated in) there can be a tendency to spend a
lot of time planning at the beginning of the audit
and then check the planning box and fail to revisit
the plan during the audit. I have only used Scrum
for a single audit, but in my experience it did a good
job addressing the continuous planning requirement
and the audit team took steps to address the other
requirements outlined in the Yellow Book, without
resorting to a full-blown TAP-like process.
Conclusion
In my experience with audit planning, the
project plan as it is originally written is rarely
ever representative of the actual scope or resource
requirements of the audit. I like Eisenhower’s
quote about planning: “In preparing for battle
I have always found that plans are useless, but
Audit Planning
planning is indispensable”. As I think it sums
up well that the most important outcome of
audit planning is not the project plan or any
other document associated with the planning
process; instead, it is thinking through the issues,
risks, constraints, etc. that you are going to face
completing your work. In my most recent audit
engagement, we used Scrum to plan and manage
the project. I found it refreshing and interesting
to try something new and see a lot of potential
advantages of this technique. I am not ready to
discard TAP, but I think it behooves an audit
team to explore alternate ways to think about
audit planning and try to match the planning
technique with the team and the project, instead of
simply doing what we have always done. To fit just
one more quote in: “A foolish consistency is the
hobgoblin of little minds, adored by little statesmen
and philosophers and divines” – Emerson.
Audit Scenario
New topic area in which
audit team lacks
experience
Quick turn-around audit
Project with ambiguous
boundaries and
expectations
Inexperienced audit team
Project in which staffing
resource availability is
unclear
Highly political audit
environment
Update of previous audit
There is no perfect way to conduct audit planning.
There will always be an unexpected twist or turn
whether you use TAP or Scrum. I have long been
frustrated by attempting to develop the perfect
audit plan when so much is still unknown at the
early part of an audit. It often felt to me that
we would identify areas where audit could add
significant value mid-way through the process with
few ways to refocus the work. The Yellow Book
tells us to extend audit steps when we determine
that fraud may have occurred. In many cases we
may find audit issues that are just as critical and
our audits should explore the issues further in
place of less relevant items that are already on the
audit plan. There is nothing that I hate more than
delivering audit work that doesn’t hit the mark.
Also, hobgoblins. 
______________________________________
Scrum
TAP
Iterative planning allows
team to refocus priorities
as more is learned. Preaudit work is quick but not
eliminated.
Process allows team to
move quickly toward key
issues.
Frequent reassessment
keeps up with changing
requirements.
Team-driven process
difficult without
experienced auditors to
mentor the process.
Delivering a product at the
end of each sprint will
allow scaling of scope to
match resources.
Planning process can be
used to identify and
address areas where more
background information
and expertise is needed.
May lack sufficient time to
create full project plan.
Process keeps tabs of
political environment and
keeps auditee in the loop
to manage issues as they
arise.
Continuous iteration
doesn’t add value in audits
with clear requirements.
Scope or objectives of the
audit are fixed and
inflexible
Continuous iteration
doesn’t add value if scope
is fixed.
Scope or objectives
flexible
Process increases focus
on the scope with the most
impact.
High potential for project
plan to be overtaken by
events.
Detailed project plan is
essential to make sure
team does not skip any
necessary steps.
Most project plans are not
scalable or flexible enough
to include significant
changes in staffing levels
without revision.
Robust project planning
can help ensure all sides
are considered in audit.
Project plan and other
planning outputs can be
recycled, reused, or at
least composted.
Project plan is essential to
ensure audit addresses
fixed objectives within time
or other resource
constraints.
Difficult to plan for multiple
possible objectives.
Advantage
Tie
Scrum
Scrum
TAP
Scrum
Tie
TAP
TAP
Scrum
Tie:2
TAP:3
Scrum:4
37
(continued)
Audit Planning
Planning the Evaluation of
the IT General Controls for
Government Entities
Introduction
T
SLEMO WARIGON, CIA,
CIGA, CISA, CICA, MBA
Director of Quality
Assurance and Technical
Audits, Office of the
Inspector General,
Washington (DC)
he pervasive use of the information
technology (IT) resources by
government entities increases the need
for ongoing assessment of related risks
and controls. Due to the diversity of
technologies continually being introduced
in an entity and its external environment,
IT risks pose their own special challenges.1
Such challenges require the government
auditors to keep up with emerging IT risks
without losing sight of previously identified
and unresolved threats.
This article discusses factors to be
considered in planning the evaluation of IT
General Controls (ITGCs). ITGCs provide
the foundation for a well-controlled IT
environment that supports the consistent
processing and reporting of operational and
financial data in accordance with applicable
laws, regulations, and management’s
directives. Evaluating ITGCs to help ensure
the effectiveness of a government entity’s
internal control systems is a key component
of the value-added audit process.
An Overview of IT General Controls
Section 6.23a of the United States
Government Accountability Office’s
(GAO) Generally Accepted Government
Auditing Standards (GAGAS) and GAO’s
Federal Information System Controls Audit
Manual (FISCAM)2 state that ITGCs are
the policies and procedures that apply to all
or a large segment of an entity’s information
systems, and that such controls help ensure
the proper operation of information systems
by creating the environment for proper
operation of application controls. ITGCs
apply to all systems components, processes,
and data for a given entity or systems
environment.3
38
FISCAM further notes that the
effectiveness of ITGCs at the entitywide
and system levels is a significant factor in
determining the effectiveness of business
process controls at the application
level. Thus, without effective ITGCs,
business process controls generally can
be rendered ineffective by circumvention
or unauthorized system modifications
conductive for irregularities. In essence,
ineffective ITGCs can cause an entity to
lose access to critical data, thereby adversely
affecting its operations, reputation,
goodwill, and financial health.
ITGCs include IT security policy, security
management, logical and physical access,
configuration management, separation of
key IT functions, management of systems
acquisition and implementation, and
contingency planning.
Planning the Evaluation
Standards for performance audits are
typically used to plan the evaluation of
ITGCs more effectively. GAGAS 6.06
states that auditors must adequately plan and
document the planning of the work necessary
to address the audit objectives. The planning
process in this case entails addressing the
following requirements and guidelines:
•
•
•
•
Ensure that the audit objectives, scope,
and methodology are clearly defined
(GAGAS 6.08, 6.09, and 6.10).
In situations where the audit objectives
are established by statute or legislative
oversight, use professional judgment
to adjust the audit objectives or scope
(GAGAS 6.07).
Plan the review to reduce the audit
risk to an appropriate level for the
auditors to obtain reasonable assurance
that that the evidence is sufficient and
appropriate to support audit findings
and conclusions (GAGAS 6.07).
In assessing the audit risk and
significance within the context of audit
objectives, obtain an understanding
Audit Planning
•
•
•
•
•
•
•
of: (1) the nature and profile of the systems
and needs of potential users of the audit
report; (2) relevant internal and IT controls;
(3) opportunities for fraud and abuse; (4)
ongoing investigations and legal proceedings
pertinent to the review; and (5) the results of
previous audits and attestation engagements
(GAGAS 6.11).
Identify potential criteria needed to
objectively evaluate the adequacy and
effectiveness of ITGCs (GAGAS 6.12a).
Identify sources of audit evidence and
determine the amount and type of evidence
needed given audit risk and significance
(GAGAS 6.12b).
Determine whether to use the work of other
auditors and specialists to address some of the
audit objectives (GAGAS 6.12c).
Assign sufficient staff and specialists with
adequate collective professional competence,
and identify other resources needed to
complete the audit (GAGAS 6.12d).
Ensure that the planning and performance
of the audit are appropriately communicated
to management officials, those charged
with governance, and others as appropriate
(GAGAS 6.12e).
Planning is a continuous process throughout
the audit. Thus, auditors need to adjust the
audit objectives, scope, and methodology as
work is being completed (GAGAS 6.07).
Prepare a written audit plan that includes steps
for adjusting the plan as needed and timely
communication of results to officials responsible
for prompt resolution of issues identified during
the review process (GAGAS 6.12f ).
•
•
•
•
•
One or more components of an entity’s
system of internal control designed to provide
reasonable assurance of achieving effective and
efficient operations;
39
Mechanisms for reliable financial and
performance reporting;
Compliance with the applicable laws,
regulations, contracts, and grant agreements;
The plans, policies, methods, and procedures
established to meet the entity’s mission, goals,
and objectives; and
Root causes of ineffective management
controls or unsatisfactory program
performance.
Auditors need to consider these factors in
establishing audit objectives for the ITGCs
review, and revise the objectives as appropriate to
address the current and emerging risks in the IT
environment.
Determining the Scope
Auditors should obtain a complete and accurate
inventory of mission-critical applications or
systems used by the government entity to help
define the scope of the ITGCs review. For
instance, if a business process application for tax
administration is a key area of audit interest, the
planning process should ensure that the testing
of ITGCs is designed to focus on those ITGCs
that most directly affect the application. The
evaluation of ITGCs typically includes the
following five categories:4
•
•
Establishing Audit Objectives
FISCAM states that the primary objectives
for ITGCs are to safeguard data, protect
application programs, and ensure continued
computer operations in the event of unexpected
interruptions. These objectives are consistent with
the internal control objectives (GAGAS 2.11b)
that include assessment of:
(continued)
•
•
•
Security Management: An agile framework
for managing risk, developing security policies,
assigning responsibilities, and monitoring the
adequacy of the entity’s IT controls.
Access Controls: Mechanisms designed to
limit or detect access to IT data, programs,
equipment, and facilities in order to protect
such resources against unauthorized
modification, loss, and disclosure.
Configuration Management: Controls
designed to prevent unauthorized changes
to IT resources and ensure that systems
are configured to operate securely and as
intended.
Segregation of Duties: Policies, procedures,
and an organizational structure established
to manage who can control key aspects of ITrelated operations.
Contingency Planning: Mechanisms
established to ensure that when unexpected
Audit Planning
(continued)
events occur, critical IT operations continue
without disruption or are promptly restored,
and critical and sensitive data are protected.
ITGCs Review Methodology
FISCAM identifies several critical elements that are
essential for establishing adequate controls over each
of the above five ITGC categories. Each critical
element includes a description of risks, controls,
activities, and suggested audit procedures. Auditors
can use this information to evaluate an entity’s
IT practices, and make a summary determination
on the effectiveness of the entity’s ITGCs at the
entitywide, system, and application levels.
Evaluating the effectiveness of ITGCs requires
auditors to identify control techniques
implemented by the entity to achieve each of
the control activities for ITGCs, and determine
whether such techniques are sufficient to
achieve the control objectives. If deemed
sufficient, auditors need to determine whether
they are implemented and operating effectively.
Conversely, if the control techniques are not
sufficient or not implemented as designed,
auditors should determine the effect on the entity’s
IT controls and the audit objectives.
To the extent practical, auditors should use the
recommended elements of a finding to document
all control deficiencies noted during the ITGCs
review. Such elements include criteria, condition,
cause, and effect (GAGAS 6.73-6.77). In addition,
some matters require early communication of
noted ITGC deficiencies to those charged with
governance or management due to their relative
importance or urgency for corrective follow-up
action (GAGAS 6.78).
Identifying and Using Applicable Criteria
When evaluating ITGCs, auditors should consider
using applicable criteria within the context of the
audit objectives, including:
•
The Federal Information Security
Management Act (FISMA), passed as part
of the Electronic Government Act of 2002,
mandates that Federal entities maintain IT
security programs in accordance with National
40
•
•
•
•
•
•
Institute of Standards and Technology
(NIST). The following NIST criteria can be
considered:
1. NIST SP 800-12, An Introduction to
Computer Security; The NIST Handbook,
October 1995;
2. NIST SP 800-14, Generally Accepted
Principles and Practices for Securing
Information Technology, September 1996;
3. NIST SP 800-53, Revision 4,
Recommended Security Controls for Federal
Information Systems and Organizations,
February 2012; and
4. NIST SP 800-64, Security Considerations
in the System Development Life Cycle,
October 2008.
Federal Information Processing Standard
(FIPS) Publication 200, Minimum Security
Requirements for Federal Information and
Information Systems.
FIPS Publication 199, Standards for Security
Categorization of Federal Information and
Information Systems.
ISACA auditing standards, guidelines, and
procedures.
COBIT 5: A Business Framework for the
Governance and Management of Enterprise
IT issued by ISACA, 2012.
Information security standards published
by the International Organization for
Standardization and the International
Electrotechnical Commission.
Requirements or management directives
unique to the environment and entity being
audited.
Common Audit Findings
Common audit findings from the previous
ITGCs review of the District of Columbia’s major
financial applications are detailed below four
categories.
Access to Systems and Data
• Failure to consistently restrict privileged
and general user access to key financial
applications in accordance with employee
job responsibilities or segregation of duties
considerations.
• Inconsistent performance and documentation
Audit Planning
•
•
of both physical and logical user access
administration activities, including the
approval of new user access and access changes,
periodic review of user access rights, including
whether user access is commensurate with job
responsibilities, and timely removal of user
access upon employee termination.
Use of generic accounts to perform system
administration or end user functions within
key applications without adequate monitoring
controls over such activities.
Failure to update the policy that defines
the minimum password configuration
requirements for the District’s IT systems
in approximately seven years. Further, the
policy was not effectively communicated to
responsible personnel.
System Changes
• Failure to institute well-designed systems
change policies that establish procedural
documentation requirements for authorizing,
developing, testing, and approving changes
to key financial applications and related
infrastructure software5 in the production
environment.
• Inconsistent adherence to established systems
change management procedures, including
instances in which changes made to the system
were not approved, tested or documented
appropriately per the established procedures.
• Failure to consistently restrict developer
access to the production environment of key
financial applications in accordance with
segregation of duties considerations or, if not
feasible, implement independent monitoring
controls to help ensure changes applied to the
production environment are authorized.
Systems Development and Acquisition
• Failure to consistently follow and provide
documentation for system development life
cycle policies for authorizing, developing,
testing, and approving system developments
to key financial systems.
• Usage of generic accounts during the
implementation to apply changes to the
application, operating system, and underlying
database with no evidence of monitoring these
generic accounts.
41
(continued)
IT Operations
• Failure to establish a monitoring process for
identifying and addressing production job
failures in several systems.
• Failures to retain system-generated
documentation from the scheduling and
processing utility to evidence the completion
status of system jobs scheduled through the
applications’ utilities.
Causes and Effect of Common Audit
Findings
The common conditions highlighted above
indicate deficiencies in both the design and
operating effectiveness of ITGCs considered
relevant to the access to systems and data, system
changes, systems development and acquisition,
and IT operations. We noted that although
management has made commendable progress
in remediating previous findings, additional
improvements in formalizing key ITGC processes
and creating an effective monitoring function are
needed.
The existence of these conditions increases the
risk that unauthorized changes applied to key
financial applications and the data they process
adversely affect application processing and data
integrity. As a result, the financial statements
may be materially impacted. Additionally, the
existence of these conditions impacts the reliability
of key application reports and the ability to rely
upon automated, configurable controls embedded
within key financial applications.
Conclusion
The planning process of any audit engagement is
hugely significant in terms of the audit and must,
therefore, be given sufficient time because the
planning will ultimately influence the quality and
value of the detailed audit work to be undertaken.
Auditors can use information presented in this
article to plan the evaluation of ITGCs. Focusing
on areas of audit interest and critical control
points helps auditors determine the most effective
and efficient manner to gather evidence needed to
determine the effectiveness of ITGCs over these
critical control points. 
______________________________________
Audit Planning
(continued)
1 Facing
About the Author
Slemo Warigon, CIA, CISA, CICA, CIGA,
MBA, is Director of Quality Assurance and
Technical Audits at D.C. Office of the Inspector
General. He has written extensively on audit
leadership and IT security, and his work has
appeared in numerous professional publications.
The Institute of Internal Auditors (IIA) presented
him the 1999 Outstanding Contributor Award
in Montreal, Canada, for his article, “Data
Warehouse Control and Security.” He can be
reached at [email protected].
***
42
IT Risk Head-On by Russell A. Jackson,
Internal Auditor, The Institute of Internal Auditors,
August 2012; pp. 38-39.
2 Federal Information System Controls Audit Manual
(FISCAM), GAO-09-232G, February 2009; p. 147.
3 The
Institute of Internal Auditors, Global
Technology Audit Guide (GTAG), Information
Technology Controls (2005); p. 3.
4 FISCAM,
p. 148.
5 Infrastructure
changes refer to software changes
and updates applied to underlying operating
systems and database supporting the key financial
applications.
Audit Planning
Toward a Comprehensive
Performance Scoping Model
W
HUGO TRUX, IV
Performance Project
Manager,
Ohio Auditor of State,
Ohio Performance Team
hen a school sets out to evaluate a
student’s progress, it does not just
assess it in one or two subjects; instead
it evaluates the student as a whole, along
with all subjects, behavior, attitude, etc.—
along multidimensional variables. The
same approach can also be applied to the
performance audits being conducted with
increasing regularity within government
entities.
Government performance auditing grew out
of financial auditing, with its emphasis on
risk-based scoping, i.e. auditing areas where
there is the greatest opportunity for fraud
and misrepresentation1. The other ancestor
to government performance auditing is the
area of program evaluation, which grew out
of social science academia2.
The Model
This scoping model is designed to assure
that a comprehensive overview is considered
when scoping begins and objectives are
laid out. Based on the premise that you do
not get answers to questions you don’t ask,
the comprehensive overview is designed to
address and query all significant aspects of an
organization. The model has five (5) main
43
foci which clearly overlap and influence oneanother. The performance audit product is
the “sweet spot” in the center.
The five foci are: Resources, Information,
People, Process and Outcomes. The
questions described below are only meant
to be illustrative of the areas to explore, and
certainly are not meant to be inclusive.
The first focus is on Resources. Looking
at resources allows us to follow the money,
and is one of the traditional aspects of
government performance audits. This
category looks at how the money is used as
it pertains to the mission, where it comes
from, as well as, how is it spent. What
types of needs assessments or business
cases are prepared to support major capital
purchases? What types of competitive
purchasing policies are utilized to procure
goods and services? Are there sufficient or
excess assets, and how are they disposed of ?
What types of controls exist to manage and
protect resources?
Scoping Information is where we follow the
data. All enterprises require information
and the communication of that information
to survive. With this focus, we reach
beyond the existing computer systems to
also ask whether information is optimized.
Is the right information collected? Is it in
Audit Planning
(continued)
the right format? Is it accessible to those who need
it, and when they need it? Is it adequate, relevant,
and accurate? What types of safeguards are in
place to assure data integrity and data protection?
People are the enterprise. To follow the people,
not only do we look at salaries and overtime, but
the organizational structure, span of control, job
descriptions, bargaining agreements, and employee
policies. Are employees properly utilized, what
are the staffing patterns; what kinds of control
and communication systems exist; what kind of
assignments are they given (and not given); what
kind of training and evaluation do they receive?
On a macro level, we can identify waste due to
flawed policies, inefficient work environment, and
staffing assignment vis-à-vis goal alignment.
Originally labeled “systems”, Process has a
new significance thanks to the Lean Six Sigma
disciplines3. Process answers: “What are you
doing?” It emphasizes the effort entailed in
producing outputs and products. Process
auditing asks what the entity does, and how they
do it. How much time, motion, and material are
consumed? What are cycle times? How much
error is inherent in the process, including both
the inputs and the process itself ? Process waste
is identified for elimination. Opportunities to
reduce risk are identified for mitigation within
program areas. Processes are also reviewed for
streamlining and/or automation.
Outcomes are probably the hardest scoping
area to deal with, and are most often omitted
because of their complexity. By following the
outcomes, we ask, “Where’s the added value?”
or “Where’s the bang for the buck?” How do we
optimize the value received or improve the efficacy
of program delivery? Is the outcome what the
customer desired, and more importantly, is the
outcome what the customer needs?4 Are customer
needs optimally met; are customers satisfied?
Are performance targets appropriate? How do
these compare against comparable benchmarks?5
Dollar-cost accounting, cost benefit analyses,
and cost-outcome accounting are financial tools
employed in following the processes.
44
Understanding outcomes also requires one to
zoom out for a wider view, beyond the individual
level, to the societal level. Some policies are
effective at the micro level, but cause more harm or
unintended consequences at the macro level.6
Using the Model
Audit managers who are planning audits can
use this model as a checklist to validate the
comprehensiveness of their planned audit. They
can ask themselves in every section: “Have we
looked at Resources, Information, People, Process
and Outcomes? “ Or, “Have we followed the
money; have we followed the data?”
Hopefully, a more comprehensive performance
audit will provide even greater return for your
customer’s investment.
Next Steps
To better understand the current state of the art
of scoping performance audits, it would be useful
to empirically research public and private sectors
audits. The survey would analyze the procedures or
questions asked, and categorize them according the
model presented in this paper. This research could
also determine the reason for exclusion of some foci:
e.g. due to budgets, client specification, etc.
A useful development step would be to develop
a comprehensive list of objectives to be used in
scoping, writing letters of agreement, and even
developing questions or procedures. If such a list
was set up in SharePoint or in an online community
(possibly LinkedIn or a trade association),
performance auditors could contribute to a shared
body of knowledge. To optimize its usefulness,
such information could be categorized by type
of institution (school, police department, mental
health service) and by function (finance, humanresources, call center, etc.). 
______________________________________
About the Author
Hugo Trux is a performance analyst manager for
the Ohio Auditor of State. Mr. Trux has more
than three decades of experience in research,
planning, and consulting in both private and
public sectors. He holds a M.A. degree in Political
Science from Ohio State University.
Audit Planning
Opinions presented in this article represent the
author’s thoughts, and not the position of the
Ohio Auditor of State.
Thanks to the few people who offered invaluable
input, especially: Ron Foster, Auditor General at
City of Oshawa, Ontario; and Virginia Bateman,
Virginia Commonwealth University.
***
1
GAO, Government Auditing Standards, 2011.
E.g. Dated but still relevant is my well-thumbed
handbook: Anderson, Wayne. F.; Frieden, Brenden
J.; Murphy, Michael. A. eds., Managing Human
Services, Washington D.C.: International City
Management Association, 1977.
2
Lean Six Sigma is a process improvement concept
comprised of “Lean” and “Six Sigma” methodologies,
resulting in the elimination of the seven kinds of
wastes (often classified as Defects, Overproduction,
Transportation, Waiting, Inventory, Motion and OverProcessing) and provision of goods and service
at a rate of 6 sigma, or 3.4 defects per million
opportunities (DPMO).
3
45
(continued)
For example, food stamp recipients don’t need
food stamps, they need sustenance. The two are
not the same. Is there a better way of delivering
sustenance with lower overhead, lower fraud, and
less loss of esteem?
4
We want to caution against overreliance on
benchmarking, especially for entities willing to be
different and innovative. Benchmarking can lead
one to chase the median, and avoid seeking the
leading edge. Benchmarks might end up becoming
a strategic vision blinder.
5
E.g. Providing support for single mothers can
have the unintended consequence of keeping them
from getting married. Or: Loose fishing quotas can
maximize profits for a fishing community, but can
eventually deplete fish and cause ruin to the same
population.
6
Audit Planning
Performance Audits: Success Is
in the Design
W
ERIN NOEL
Fiscal and Policy Analyst,
Office of the Independent
Budget Analyst
San Diego (CA)
e know that to comply with
Generally Accepted Government
Auditing Standards (GAGAS), auditors
must plan their fieldwork and planning
must be documented. Although the form
and the content of the written plan and
the level of detail will vary among audits
depending on the audit’s complexity,
controversy, and risk, among other things.
All successful audits are based on sound
planning. Well planned audits will have
shorter fieldwork and report writing phases
because time has been spent up front
reviewing and assessing information to
identify key issues and determining how
to best approach these issues to identify
the four elements of a finding—criteria,
condition, cause, and effect (C3E). Well
planned audits also consider potential
limitations and ways to mitigate them,
reducing roadblocks and rework in later
phases. Particularly considering the fiscal
challenges that many U.S. cities currently
face, sound planning is critical to effectively
focus limited audit resources on key issues
and risks.
The Design Matrix
My background and training at th e
Government Accountability Office (GAO)
shaped my approach to strong planning for
performance audits, and I have adjusted this
approach as needed to fit different issues
and audits in local government. If you are
familiar with GAO, you know that every
tool and approach is tested and retested,
piloted and re-piloted, and this was the
case with the Design Matrix—a valuable
planning tool for both audit staff and
managers. The Matrix includes the audit
objectives, scope, and methodology, and
GAO requires that it be completed at the
end of the planning phase for all audits.1
The primary focus of this article is to discuss
the value and development of the Design
Matrix, but I want to note that GAO uses
it in conjunction with the Performance
Audit Tool and the Project Plan.2 The
Performance Audit Tool is essentially a
GAGAS-based checklist used by audit
teams during planning to determine
whether internal controls, compliance with
laws and regulations, and/or fraud, risk or
other potential abuses are significant to the
performance audit objectives. Potential
issues identified by the audit team are
incorporated into the Design Matrix. The
Project Plan is completed based on the
methodology in the Design Matrix and
specifies the tasks, staff responsibilities, and
estimated milestone dates of the audit.
Value of the Matrix for Local Government
Auditors
The Design Matrix has been a valuable
Figure 1: Flow and Linkages of Audit Phases and Related Outputs
46
Audit Planning
(continued)
planning tool for GAO audits at the federal
and practices of other cities and organizations.
level, but is it useful or appropriate for audits
Identifying criteria up front will provide a
of local governments? Without question, the
comprehensive starting point to drive your
Design Matrix can be a great planning tool for
questions and prevent holes that you miss or
performance audits at the local level whether
will have to fill in later, rather than just fishing
you have a narrow focus, such as a specific
for information. For example, let’s say we are
program to audit, or a broad focus, such as an
auditing a department’s asset management
entire department. This is because the process of
program. Auditor A interviews department staff to
completing the Matrix helps auditors review and
determine what their program includes and finds
assess a large amount of information and identify
that they have established goals and objectives,
the key issues that need further investigation. It
are conducting an asset inventory, and have a
also helps staff to
short-range plan in
think forward to
the works. But, is the
Figure 2: Components of Asset Management
set up findings
department doing
development (C3E)
well or not?
and the report
message. Figure
On the other hand,
1 illustrates the
Auditor B conducts
flow and linkages
research and identifies
between the four
the components of
general phases of a
asset management in
performance audit
federal guidance. See
and the related
Figure 2. Auditor B
outputs at each
can use these criteria
phase. Even though
to drive questions
we are discussing
for department staff.
the planning phase,
If the department’s
it is important
program does not
to consider the
include the other
Source:
Federal
Highway
Administration
subsequent stages
components, such as
of the audit,
conducting condition
because the audit
assessments, then that
plan will be the guide for fieldwork and basis for
is a finding. Without identifying criteria up front,
development of findings and report writing.
the findings can be thin and you will not have a
complete picture of how the department is doing.
Importance of Starting with Criteria
Further, the risk is increased that you may miss
The Design Matrix helps you to think about
something that is important or have to go back to
C3E early in the audit so that your findings and
the department later to fill in the holes.
recommendations will be well supported, and
identifying criteria while conducting preliminary
Developing the Design Matrix
research is an important starting point. Criteria
GAO has at least two versions of the Design
are the standards and measures against which
Matrix, and I adapted the six-column version
auditors compare and evaluate performance and
in Figure 3 based on my experience in local
are required to comply with GAGAS. Criteria
government. The key is that the Matrix can be
provide context for understanding audit results
adjusted based on the needs of your organization,
and strengthen the report message. Common
auditors, and audits, including level of detail,
criteria sources include legislation, regulations,
etc. Before discussing how to develop the Design
codes, and mandates; department policies,
Matrix, there are two important points to
administrative regulations, guidance, and manuals; remember. First, while I will discuss each column
industry standards, guidance, and best practices;
in turn, you will be moving back and forth across
47
Audit Planning
(continued)
Figure 3
Column 1
Column 2
Column 3
Column 4
Column 5
Column 6
Researchable
Question(s)
Risks or
Negative Effect
Scope and Methodology
Information
Required and
Sources
Limitations and
Mitigation Plan
What We Will
Likely Be Able
to Say
Without
formalized
requirements and
processes to plan,
communicate, and
coordinate
activity, there is
the risk that
departments and
private entities will
excavate into a
street multiple
times, which
increases the cost
for the City and
causes the street
condition to
degrade at a
faster rate.
Scope: Inter-departmental and interorganizational coordination issues related
to the City’s street maintenance over the
past 5 years. This includes resurfacing
(asphalt overlay and slurry seal) for
maintenance or emergency repairs, but
excludes pothole repairs because they are
part of ongoing street maintenance rather
than the resurfacing program. This also
excludes cement streets because the
treatment is different and also is not part of
the resurfacing program.
________________________________
Documents and
Data:
Question 1: To
what extent is
the
Transportation
Department
maintaining the
integrity of
newly
resurfaced
streets?
1(a) To what
extent does the
Transportation
Department plan,
communicate,
and coordinate
resurfacing
projects with
other
departments and
private entities
that conduct work
on or under
streets?
1(b) To what
extent are City
departments and
private entities
required to obtain
permits for street
excavations?
1(c) To what
extent do City
departments
monitor and
enforce permit
and/or trench cut
moratorium
requirements?
1(d) What is the
status of the
City’s planned
GIS system for
identifying project
conflicts, and how
will this help
prevent trench
cuts into newly
resurfaced
streets?
If City
Departments are
not enforcing
permit
requirements for
private entities
excavation of
streets, taxpayers
will have to
subsidize street
maintenance and
repairs.
•
Municipal
Code
§62.1203 §62.1204
•
Department
al policies
and
procedures
•
Street
resurfacing
contracts
Methodology:
Criteria
1. San Diego Municipal Code §62.1203
and §62.1204 – Prohibits excavations
into streets which have received
asphalt overlay and slurry seal for
three years and one year,
respectively, with a few exceptions
such as emergencies, installation of
new services, and nonlinear
excavations.
1.
2.
Transportation and Engineering
Departments’ policies and procedures
for coordinating project conflicts.
Review literature and interview
officials from other cities to identify
best/recommended practices for interdepartmental coordination of work
conducted on or under City streets.
Condition
1.
Conduct a sample of streets,
including site assessments and
photographs of street conditions, to
determine the extent to which trench
cuts are being made into newly
resurfaced streets.
2.
Interview Transportation staff to
determine status and functions of the
planned GIS system for identifying
project conflicts.
Cause
1.
Interview City department staff
responsible for coordination of work
and identification of project conflicts.
2.
Trace newly resurfaced streets in
sample to permits to determine
reasons for trench cut, such as
emergency repairs.
Effect
1.
Review industry information on
impacts of trench cuts and public
complaints received by Council
Members and/or City departments
relating to trench cuts in newly
resurfaced streets.
48
•
Records of
excavation
permits
Sources:
•
Transportat
ion and
Engineerin
g
Department
staff
•
Pavement
Manageme
nt System
•
Developme
nt Services
Department
Permit
Tracking
System
Limitation:
Based on our first
street maintenance
audit, we know that
the Pavement
Management
System contained
street condition
information that was
out of date.
Mitigation Plan:
We will pull our
street sample from
recent resurfacing
contracts and make
site visits and take
photographs to
ensure the accuracy
of information for
our sample.
We expect to find
that newly
resurfaced
streets are being
excavated during
the moratorium
period and
coordination of
work performed
on or under City
streets is limited.
We may be able
to recommend
that the Public
Works
Department
develop a
Citywide
excavation plan
for all planned
maintenance
work and share
this with all
departments and
private entities
that conduct work
on or under City
streets.
Audit Planning
columns as you develop and refine each question
and related methodology, limitations, etc. Secondly,
the Design Matrix [and related Project Plan] is a
living document. Even with good planning, as you
get into fieldwork you may find an issue that needs
more or less attention and resources. As findings
evolve and with the agreement of management, you
can adjust the Matrix to make sure you cover the
most important issues.
Column 1 - Researchable Questions
This column essentially includes the audit
objectives. The use of the question format rather
than statements helps you to think through how
you are going to answer something and really
requires doing the heavy thinking up front.
Researchable questions may be either descriptive or
evaluative, depending on what we hope to find out.
Descriptive questions are most appropriate when
we anticipate that our work on a particular objective
will be to describe the status of a program or process.
Note that a question that is phrased descriptively
and is not expected to lead to recommendations
should not contain causes or effects.
(continued)
Column 2 - Risks or Negative Effect
The risk column essentially represents the negative
effect or consequence, including the probability
and severity of real or potential impacts. Risk is
largely the basis for determining which audits to
conduct, particularly given limited audit resources.
Column 3 - Scope and Methodology
The scope establishes the parameters or boundaries
of the audit given budget, staff, risks, and extent of
issues. For example, can you limit an audit of staff
overtime to one or two departments instead of
citywide based on risks identified? It may also be
beneficial to explain why you are limiting the scope.
In contrast, evaluative questions, such as those that
ask how or to what extent something is occurring,
are appropriate when you have an indication of
a finding and think it is likely that it will lead to
recommendations. Broad questions followed by
more pointed sub-questions can sometimes add
clarity and help develop more substantive findings.
Importantly, when framing questions, consider
how you might identify C3E. It is good practice
to think about these elements early in the audit so
that the evidence is sufficient and appropriate to
support the findings and recommendations.
The rest of this column should describe the
methodology—how you plan to gather and
evaluate the information in Column 4 and
otherwise conduct the audit to answer the
researchable questions. As discussed earlier, if
you expect researchable questions to lead to
a recommendation, it is important to include
a methodology that will help lead to the four
elements of a finding. For criteria, you may have
already identified specifics and these can be spelled
out. As I mentioned before, criteria will drive some
of your researchable questions. If you haven’t yet
identified specific criteria, you can either describe
general criteria or explain how you plan to find
criteria. Ultimately the goal is to identify steps
that will enable you to answer the researchable
questions given the available information and
data (in Column 4) and to focus on steps that can
be accomplished within realistic timeframes and
resources. This also includes thinking through
potential limitations and ways to mitigate them (in
Column 5).
Ultimately, the goal is to construct questions that
you can realistically answer given the information
and methodology in the successive columns. Tying
the questions with the information in the other
columns will also help you to avoid questions
that contain open-ended timeframes, require an
unrealistic amount of data collection, involve an
inordinate amount of resources to obtain, or would
take you down a “rabbit hole” without sufficient
risk or impact. During findings development, these
researchable questions may be refined and will
ultimately become reporting objectives.
Column 4 - Information Required and Sources
This column should include the kinds of
information and data you will need to conduct the
steps in the methodology as well as the sources,
such as data systems, documents, or department
staff. It is important to consider what it will take to
answer the overall question. If Column 1 includes
sub-questions, you do not need a one-to-one
ratio across the Matrix. If you already know your
criteria, you can specifically include it here or
provide general information, such as departmental
policies and procedures.
49
Audit Planning
(continued)
Column 5 - Limitations and Mitigation Plan
This column will require you to anticipate
conditions that might limit your ability to obtain
needed information or data, conduct the steps
in your methodology, or answer the researchable
questions. If the limitations could threaten your
ability to answer the researchable question,
consider rewording the question and/or altering
the scope to decrease that risk. For any limitations
included in this column, you should include
steps you can take to mitigate that limitation.
Considering potential limitations and ways to
mitigate them at this point will help to reduce
roadblocks and rework in later stages of the audit.
Column 6 - What we will likely be able to say
At this stage of the audit, a discussion of the
return on investment and what you might expect
to find can be a challenging but also valuable
exercise. This column should not include potential
recommendations, because the team needs to
maintain as much objectivity as possible. For
example, it is preferable to say we expect to find
or we may be able to recommend. This column
should provide an assessment of likely findings
and other potential outcomes of our audit work
and may also include an evaluation of how
much additional resources and time are needed.
Obtaining an understanding of the likely impact of
our work makes it much easier for audit managers
to assess requests for additional resources,
particularly relative to other audits.
50
Setting Up Findings Development and
Report Writing
One of the greatest values of the Design Matrix
is that it helps set up findings development and
report writing as shown in Figure 4. The answers
to your researchable questions could ultimately
become findings or they may drop off. The scope
and methodology will help you to develop C3E
which will become well-supported findings and
the basis for recommendations in your report.
Key Takeaways
Effective planning will help to ensure a quality
audit, effectively focus limited resources on key
issues and risks, and ensure that your methodology
will get to the heart of the issues. Identifying
criteria up front will provide a comprehensive
starting point to drive your questions and prevent
holes that you miss or have to fill in later. While
there are many approaches to audit scoping and
planning that can be effective, and different types
of audits and situations, the Design Matrix is an
effective tool for planning performance audits
because it will help you to:
• review and assess a large amount of
information and identify the key issues that
need further investigation; and
• think forward to set up findings development
(C3E) and the report message and
recommendations. 
______________________________________
Quarterly Quizzer
Quarterly Quizzer… Has Moved to Moodle (Online Only)
All future Quarterly Quizzers will be
posted to the ALGA website using the
new Moodle training module. They’ll
be available under the “Training >
Quizzers” menu option. As a result of
the move, Quizzer completion and
CPE distribution procedures have
changed, as briefly described below.
New Quizzer Process…
1. Log on to the ALGA Website at
algaonline.org.
2. Navigate to the Quarterly Quizzers
page.
3. Select the Quarterly Issue for
which you’d like to receive CPE
(Summer 2011 and thereafter).
4. Select the Quizzer from the
available options (as shown to the
left). You may have to select the
Quizzer in other views as well
(e.g., as shown here).
5. Answer each question and click
“Submit all and finish” (or “Save
without submitting”).
6. Print and retain a copy of your
CPE certificate as evidence of
your CPE.
Old Quizzer Process…
1. Log on to the ALGA website.
2. Navigate to the Quarterly Quizzers
page.
3. Select the Quarterly Issue for
which you’d like to receive CPE
(Summer 2011 and earlier).
4. Answer each question and click
“Submit”.
5. Your electronically graded
Quizzer will be e-mailed to you
(75% or above = 1 CPE).
6. Print and retain a copy of your
graded results as evidence of your
CPE.
Important information:
You are responsible to retain documentation to support
your CPE. The Quizzer will qualify for CPE under the
Government Audit Standards, but ALGA does not
guarantee that the Quizzer will meet individual state
CPA license requirements or the requirements of other
professional associations. You will need to make that
determination for yourself based on the rules of your
state board of accountancy.
Please check out the website for more details.
Your Quizzer score will appear
(75% or above = 1CPE).
51
Abstracts of Recently Completed Audits
Following are summary abstracts for audits recently completed by local government audit organizations. More detailed
abstracts are available on the ALGA website (www.governmentauditors.org) where you can search for recent abstracts and
search the ALGA abstract archives. These abstracts are a highly valued resource to other auditors, so please take the time to
submit an abstract when you complete an audit. Abstracts for completed audits can be submitted via the ALGA website. For
more information on submitting abstracts, please see the back of this journal.
Please NOTE: Beginning in the Winter 2012 issue, the abstract summaries will no longer
be printed in the Quarterly. Instead, the full abstracts will continue to be available on the
ALGA website and the audit titles will be included in each Quarterly.
FINANCE
Business License Taxes: Providing Better Guidance and Customer Service Will Increase Revenues (May 2012).....................
Hotel Occupancy Tax Revenue Audit (May 2012)..................................................................................................................................
Financial Condition of Metro, FY2001-02 to FY2010-12 ( June 2012).............................................................................................
Sheriff 's Office Payroll ( July 2012)..............................................................................................................................................................
Recreation and Community Services Bank Account Audit (May 2012)............................................................................................
54
54
54
54
55
HOUSING AND SOCIAL SERVICES
Atlanta Tax Allocation Districts ( July 2012)............................................................................................................................................. 55
Portland's Housing Bureau: Bureau acting on risks, although more remains to be done (May 2012).......................................... 55
MISCELLANEOUS
Review of Controls Over Procurement and Payment Functions at Toronto Community Housing Corporation,
(TCHC), subsidiary: Housing Services Inc. (February 2012).........................................................................................................
311 Toronto - Full Potential For Improving Customer Service Has Yet To Be Realized (October 2011)..................................
Animal Care and Control Performance Audit ( July 2012)....................................................................................................................
Assessor’s Office Performance Audit ( June 2012)....................................................................................................................................
Atlanta Fleet Services Inventory Controls ( July 2012)...........................................................................................................................
Audit of City Parking Contracts ( July 2012)............................................................................................................................................
Review of the Energy Retrofit Program at Community Centres & Arenas ( June 2012)...............................................................
FY 2012 Boards and Commissions Risk Assessment...............................................................................................................................
Gracedale Agency Fund ( June 2012)..........................................................................................................................................................
Review of Inventory Controls at Transportation Services Storage Warehouses (May 2012)..........................................................
Downtown Office Space: City uses most of its owned space, but lease practices need attention (April 2012)..........................
Front yard and Boulevard parking – Improvements needed to enhance Program effectiveness (February 2012)......................
Performance Measure Certification ( July 2012).......................................................................................................................................
Residential Solid Waste: Customer rates accurate, but monitoring should continue ( June 2012)...............................................
Special Request Report on Short-term Rentals (A and B) (May 2012)..............................................................................................
Sustainability Management Follow-up Strong foundation created ( June 2012)...............................................................................
The City and Toronto Community Housing Corporation Needs to Strengthen its Oversight of Subsidiaries and
Other Business Interests (February 2012).............................................................................................................................................
52
56
56
56
56
57
57
57
58
58
58
58
59
59
59
59
60
60
Abstracts of Recently Completed Audits
PARKS AND RECREATION
Forestry Management Audit (August 2012).............................................................................................................................................. 60
PROCUREMENT & CONTRACT COMPLIANCE
American Recovery and Reinvestment Act (ARRA) Funding Performance Audit (May 2012)...................................................
Cemetery Contract Follow-Up Audit (August 2012).............................................................................................................................
Court Tower-Audit of Contract Terms and Conditions (April 2012)................................................................................................
Purchasing Card Program ( July 2012)........................................................................................................................................................
61
61
61
61
PUBLIC SAFETY
Seattle Police Department's In-Car Video Program ( June 2012)........................................................................................................ 62
Performance Audit of King County Sheriff 's Office and Law Enforcement Oversight ( July 2012)........................................... 62
Sheriff ’s Office Patrol Service Agreements ( July 2012)......................................................................................................................... 62
PUBLIC UTILITIES
Atlanta Department of Watershed Management Back Billing of July 2008 Rate Increase ( July 2012)...................................... 63
Follow-Up Audit of Austin Water Utility Water Loss ( June 2012)..................................................................................................... 63
Portland Water Bureau: Further advances in asset management would benefit ratepayers ( June 2012)..................................... 63
PUBLIC WORKS
Public Works - Capital Street Project Construction Administration ( June 2012).......................................................................... 64
53
Abstracts of Recently Completed Audits
was to identify hotel occupancy tax deficiencies and to
educate the hotel owners and operators on applicable laws
and requirements for documentation and remittance of hotel
occupancy taxes. As part of this audit, we reviewed the tax
exemptions and exclusions claimed by four hotels within
the City’s full purpose jurisdiction. We identified three
hotels that did not comply with the City’s hotel occupancy
tax ordinance, with a resulting deficiency of approximately
$84,806. We made one recommendation to ensure the
collection of tax deficiencies identified in this audit.
FINANCE
FINANCE: Business License Taxes: Providing Better
Guidance and Customer Service Will Increase Revenues
(May 2012)
CONTACT INFORMATION
Claudette Biemeret, Auditor II
2180 Milvia Street, 3rd Floor
Berkeley, CA 94704
510.981.6750
[email protected]
www.cityofberkeley.info/auditor/
FINANCE: Financial Condition of Metro, FY2001-02
to FY2010-12 ( June 2012)
CONTACT INFORMATION
Suzanne Flynn
Metro Auditor
600 NE Grand Avenue
Portland, OR 97232
503-797-1891
[email protected]
www.oregonmetro.gov/auditor
SUMMARY: The Berkeley City Council enacted the
business license tax ordinance in 1977 to raise revenue for
municipal purposes. The license fee for most businesses is
based on their annual gross receipts. In fiscal year 2011, the
city received $14.5 million from more than 13,000 business
licenses. We conducted this audit to determine if Finance
assigned correct tax codes, accurately calculated taxes,
and when appropriate, accurately assessed penalties and
interest. Finance staff did not always assess business license
taxes, penalties, and interest accurately and consistently,
due to complex and unclear requirements in the city’s
municipal code, lack of guidance, and responsibilities being
dispersed among multiple units within Finance. Errors and
inconsistencies cause inequitable taxation and can lead to
businesses appealing their cases. Finance also did not follow
recognized best practices or take full advantage of available
options for collecting delinquent business license accounts.
Consequently, the city will likely collect only $50,000
of the $1.1 million in delinquent accounts. More timely
and persistent collection efforts could generate additional
revenue of $90,000 annually.
SUMMARY: This report reviews Metro's financial
condition by assessing trends over the last ten years.
The Metro Auditor's Office completes this audit every
two years and this is the third report in the series. Based
upon financial indicators that are recommended by the
International City/County Management Association, it
provides a check-up on how well Metro is doing financially.
This year, in an attempt to improve the report's quality,
expenditure information by individual departments was
added. Financial standing indicators are positive. Metro has
consistently been well above the recommended liquidity
ratio. The majority of Metro's debt is covered by voterapproved tax increases to pay off general obligation bonds.
FINANCE: Hotel Occupancy Tax Revenue Audit
(May 2012)
FINANCE: Sheriff 's Office Payroll ( July 2012)
CONTACT INFORMATION
Rachel Snell, Assistant City Auditor
301 W. 2nd Street
Austin, Texas 78701
(512) 974-2552
[email protected]
http://www.austintexas.gov/page/archive-auditor-reports
CONTACT INFORMATION
Toni Sage, Audit Supervisor
Maricopa County Internal Audit Department
301 W. Jefferson St., Ste 660
Phoenix, AZ 85003
(602) 372-1004
[email protected]
www.maricopa.gov/internal_audit
SUMMARY: The City of Austin levies a hotel occupancy
tax (HOT) of nine percent (9%) on qualified room stays
costing more than two dollars per night in hotels, motels,
tourist homes, tourist courts, lodging houses, inns, rooming
houses, and bed and breakfasts. The objective of the audit
SUMMARY: The largest annual expenditure of the
Maricopa County Sheriff ’s Office is payroll ($220 million),
for 3,550 employees. The purpose of this review was to
determine if payroll transactions and personnel transfers are
54
Abstracts of Recently Completed Audits
processed accurately and funded appropriately. We found
that payroll processing is mostly accurate. However, payroll
controls can be strengthened. We recommended that
management (1) strengthen internal controls for personnel
transfers, including the use of required forms to document
and approve employee transfers, (2) develop and implement
a requirement for direct supervisor approval of reported
time, (3) align MCSO and County policies and develop
procedures to ensure consistency.
SUMMARY: We undertook this audit because the city’s
use of tax allocation districts to finance redevelopment
has grown to encompass 20% of the city’s land area and
15% of total assessed property value. Neither the city nor
its redevelopment agent, Invest Atlanta, systematically
tracks progress toward meeting redevelopment plan goals.
The redevelopment plan for each tax allocation district
is adopted by ordinance following public hearing. The
redevelopment plan establishes the district’s geographic
boundaries; explains why the area requires public subsidy;
outlines the scope of the economic development projects
and project costs; estimates the frozen tax base and tax
increment amounts; and identifies plans to issue bonds.
Without systematic tracking of progress compared to the
redevelopment plan, the city lacks a mechanism to tell
when a redevelopment plan is substantially complete and
no more public subsidy is needed.
FINANCE: Recreation and Community Services Bank
Account Audit (May 2012)
CONTACT INFORMATION
Kim Taylor
Council Auditor’s Office
117 West Duval Street, Suite 200
Jacksonville, FL 32202
(904) 630-1625
[email protected]
[email protected]
http://www.coj.net/City-Council/Council-Auditor/
Reports.aspx
HOUSING AND SOCIAL SERVICES: Portland's
Housing Bureau: Bureau acting on risks, although more
remains to be done (May 2012)
CONTACT INFORMATION
Jennifer Scott
Portland Audit Services Division
1221 SW 4th Ave, Room 310
Portland, Oregon 97204
503-823-3538
[email protected]
SUMMARY: The audit objectives were to determine
whether: 1) internal controls were in place and functioning
as intended for the imprest checking accounts maintained
by Recreation and Community Services (RCS) and 2)
disbursements from the RCS imprest checking accounts
were properly supported and authorized. It appears
that RCS did not comply with the City’s policy and
procedure manual that establishes internal controls for
imprest checking accounts. Overall, it appears that the
disbursements from the RCS checking accounts were
properly supported and authorized; however, we did find
multiple issues with the Special Events imprest account.
SUMMARY: Since the Portland Housing Bureau was
formed in 2010, it has seen two rounds of staffing cuts and
two new Bureau Directors. Because the Bureau is new,
complex, and has experienced such significant change,
auditors performed a broad-based audit to assess risk areas
at the Bureau. The objective of this audit was to assess risk
at the Bureau and identify areas for future audit work. In
order to meet the objective, our audit included a broad
review of the Bureau, including its mission, objectives,
tasks and organizational structure. Auditors found six
risk areas the Bureau should focus on. These are areas of
potential risk, and the report identifies areas the Bureau
can work on to help make the most of available resources.
The audit report also checked up on recommendations
made in 2007 in an audit on the Bureau's 10-year Plan
to End Homelessness, and followed-up on results of the
City's financial audit. The Bureau has addressed most of
the recommendations in the 2007 audit of the plan to
end homelessness, and the Bureau has resolved significant
deficiencies from the 2009 and 2010 financial audits.
HOUSING AND SOCIAL SERVICES
HOUSING AND SOCIAL SERVICES: Atlanta Tax
Allocation Districts ( July 2012)
CONTACT INFORMATION
Stephanie Jackson
City Auditor's Office
68 Mitchell Street
Suite 12100
Atlanta, Georgia 30303
404.330.6678
[email protected]
www.atlaudit.org
55
Abstracts of Recently Completed Audits
and information 24 hours a day, seven days a week. Key
audit findings are: based on March 2011 call statistics, 1
in 5 calls to 311 Toronto was not answered, and 1 in 10
callers waited longer than 3 minutes before the call was
answered. The varying performance level among individual
staff, the high staff absenteeism rate, and the existing
monitoring system, impact 311 Toronto’s ability to answer
calls in a timely manner and should be addressed. The audit
also indentified a number of opportunities to improve
operations while reducing costs: ensuring the number of
staff on overnight shift matches call volume; reviewing
the level and placement of Information Technology staff
currently designated for supporting 311 Toronto; and
developing a business case on incorporating telephone selfserve technologies into 311 operation to help improve call
response and efficiency.
MISCELLANEOUS
MISCELLANEOUS: Review of Controls Over
Procurement and Payment Functions at Toronto
Community Housing Corporation, (TCHC), subsidiary:
Housing Services Inc. (February 2012)
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
City of Toronto
55 John Street, Metro Hall, 9th Floor
Toronto, Ontario M5V 3C6
416-392-8030
[email protected]
SUMMARY: The objective of this review was to assess
the extent to which Housing Services Inc.’s (HSI)
administrative structure and control framework supports
sound financial management and compliance with
purchasing policies and procedures.
Description: The current independent operating
subsidiary structure and governance model need to be
balanced against barriers which impact the efficiency
and effectiveness of both TCHC and HSI. Key issues
identified are: lack of a TCHC-wide procurement strategy
for construction and maintenance services impedes
operational efficiency; organization structure can pose
challenges for developing an effective procurement
strategy; and lack of clear accountability for TCHC
procurement decisions executed by the subsidiary.
MISCELLANEOUS: Animal Care and Control
Performance Audit ( July 2012)
CONTACT INFORMATION
Chris Horton, Audit Supervisor
City and County of Denver Auditor’s Office
201 W. Colfax Ave., Dept. 705
Denver, CO 80202
(720) 913-5024
[email protected]
www.denvergov.org/auditor
SUMMARY: This audit evaluated the city’s Animal Care
and Control Division to assess the efficacy of pet licensure.
We determined that dog and cat licensure, though it
raises revenue for the city, is redundant to public health
regulations over dogs and cats, is widely ignored, and is
less effective than at reuniting lost pets with their owners
than methodologies such as micro-chipping. Accordingly,
we recommended that the city eliminate the pet licensure
requirement and replace it with an optional pet registration
program. The audit also identified possible ways to close
the gaps between expenditures and revenues, including
increasing some fees.
MISCELLANEOUS: 311 Toronto - Full Potential For
Improving Customer Service Has Yet To Be Realized
(October 2011)
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
City of Toronto
55 John Street, Metro Hall, 9th Floor
Toronto, Ontario M5V 3C6
416-392-8030
[email protected]
MISCELLANEOUS: Assessor’s Office Performance
Audit ( June 2012)
SUMMARY: The City of Toronto Auditor General
conducted a review of the City’s 311 operations to assess its
operating effectiveness and efficiency. The City achieved
a significant milestone in improving customer service
when it launched 311 Toronto in September 2009. 311
Toronto provides the public with one easy-to-remember
phone number to obtain non-emergency City services
CONTACT INFORMATION
Chris Wedor, Audit Supervisor
City and County of Denver Auditor’s Office
201 W. Colfax Ave., Dept. 705
Denver, CO 80202
56
Abstracts of Recently Completed Audits
(720) 913-5021
[email protected]
www.denvergov.org/auditor
unlocked rooms, rooftop access, a damaged perimeter fence,
and distribution of keys to multiple people.
MISCELLANEOUS: Audit of City Parking Contracts
( July 2012)
SUMMARY: The purpose of the audit was to evaluate
the Denver Assessor’s Office’s information management
practices and its customer relations approach, specifically
with regard to whether tax-exempt properties within the
City and County of Denver are properly categorized.
Our review affirmed that exempt properties are being
properly categorized by the Assessor’s Office. However,
we did offer two recommendations: first, to upgrade the
software program used by the Assessor’s Office to monitor
exempt properties and, second, to update the policies and
procedures to accurately reflect the manual processes being
used to handle exempt properties.
CONTACT INFORMATION
Reuben Iyamu, Senior Auditor
City of Tallahassee
300 S. Adams Street
Tallahassee, Florida 32311
850-891-8309
[email protected]
http://www.talgov.com/auditing/
SUMMARY: The audit was conducted to evaluate
whether (1) City parking contracts were competitively
awarded and properly executed, (2) parking operations
were in accordance with contractual terms, (3) parking
revenues due the City were properly and timely received,
and (4) responsible City departments performed adequate
monitoring and oversight of contracted parking operations.
The audit addressed the operation of parking facilities by
the City’s contractor, Republic Parking Systems, during the
period January 2008 through December 2010, and certain
related events through the end of our fieldwork in February
2012. Overall, City parking contracts were properly
and adequately managed and administered by the City’s
contractor; revenues were properly collected, accounted
for, and paid by the contractor to the City; and the City’s
monitoring and oversight efforts over the contractual
operations and activities were appropriate and adequate.
MISCELLANEOUS: Atlanta Fleet Services Inventory
Controls ( July 2012)
CONTACT INFORMATION
Stephanie Jackson
City Auditor's Office
68 Mitchell Street
Suite 12100
Atlanta, Georgia 30303
404.330.6678
[email protected]
www.atlaudit.org
SUMMARY: We undertook this audit because of risks
we identified in our 2008 audit of fleet services and due
to the inherent risks in managing inventory. The recorded
value of fleet services’ inventory of vehicle parts was $1.9
million as of March 1, 2011. The Office of Fleet Services’
inventory records overstate the total value and number
of items on hand, which indicates potential for theft or
fraud and reduces operational efficiency. We also identified
discrepancies between inventory records and the number
of items on the shelf in 9 of a random sample of 30 parts.
These inaccuracies indicate risk of undetected theft and lost
or missing assets. Further, employees were not conducting
monthly counts of parts inventory, as required by fleet
services’ written policies.
MISCELLANEOUS: Review of the Energy Retrofit
Program at Community Centres & Arenas ( June 2012)
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
Auditor General’s Office, City of Toronto
55 John Street, Metro Hall 9th floor
Toronto, M5V 3C6
416-392-8030
[email protected]
SUMMARY: This review evaluated the management of
the energy retrofit program and processes with a view to
confirming that anticipated savings were achieved. The
audit identified improvements on a go forward basis to
ensure that anticipated energy savings are
maximized and actual results quantified and reported
to City Council. Recommendations included changes
Because the inventory and Oracle systems are not
linked, staff enters information in both. This dual entry
weakens the controls in each system intended to separate
incompatible duties and ensure items are accounted for
when received. We observed security risks at all facilities
except the airport locations, including inadequate lighting,
57
Abstracts of Recently Completed Audits
to ensure that full benefits are realized from newly
implemented building automation systems.
ensure compliance with laws and regulations regarding the
Supplemental Security Income (SSI) resource limit.
MISCELLANEOUS: FY 2012 Boards and
Commissions Risk Assessment (May 2012)
MISCELLANEOUS: Review of Inventory Controls at
Transportation Services Storage Warehouses (May 2012)
CONTACT INFORMATION
Kathie Harrison, Auditor
City of Austin, Office of the City Auditor
301 West 2nd Street, Suite 2130
Austin, Texas 78701
(512) 974-2805
[email protected]
www.austintexas.gov/page/archive-auditor-reports
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
City of Toronto
55 John Street, Metro Hall, 9th Floor
Toronto, Ontario M5V 3C6
416-392-8030
[email protected]
SUMMARY: Our objective in conducting this review
was to assess the effectiveness of Transportation Services
Division’s controls over traffic control devices inventory.
The Traffic Plant Installation and Maintenance (TPIM)
unit is part of the Transportation Services Division. The
unit is responsible for managing the design, installation
and maintenance of all electrical traffic control and related
devices in the City of Toronto.
In order to repair traffic signal devices in a timely manner
or install new ones as required, TPIM rents warehouse
space and stores approximately $6.7 million of inventory.
At the time of our review, Transportation Services was not
operating according to the corporate materials management
strategy and this contributed to certain internal control
weaknesses we identified.
SUMMARY: Based upon the risk assessment, the highestranked boards were the Planning Commission, Board
of Adjustment, Public Safety Commission, and Animal
Advisory Commission. Issues contributing to higher
rankings include, but not limited to, disagreement on
board mission; limited or no monitoring of potential
conflicts of interest; and inadequate board documentation
such as meeting agendas, meeting minutes, and annual
reports. Additionally, some boards are sovereign boards or
subject to state or federal law.
MISCELLANEOUS: Gracedale Agency Fund ( June
2012)
CONTACT INFORMATION
Stephanie Rath-Tickle
Controller's Office, County of Northampton
669 Washington St.
Easton, PA 18042
610-559-3257
[email protected]
www.northamptoncounty.org
MISCELLANEOUS: Downtown Office Space: City
uses most of its owned space, but lease practices need
attention (April 2012)
CONTACT INFORMATION
Bob MacKay
Office of City Auditor, Audit Services Division
1221 SW 4th Ave., Rm. 310
Portland, OR 97204
503-823-3562
[email protected]
www.portlandonline.com/auditor/auditservices
SUMMARY: This audit is one in a series of periodic
audits of all agency funds performed for the purpose
of supplementing the year-end work prescribed by the
County’s external auditor. The focus of the audit was on
the receipt and disbursement functions and on compliance
with laws and regulations regarding the handling of
resident funds. The purpose of the audit was: to determine
the adequacy of internal controls and to test compliance
with laws, regulations, policies and procedures. Our tests
were conducted on transactions occurring in December,
2011. We found that running a high balance notification
report for the Resident Account on a monthly basis will
SUMMARY: The City of Portland, Oregon, owns and
leases office space to house employees in the downtown
core. The objectives of our audit were to determine
whether the City was using most or all of its owned
downtown office space and also to determine how the City
makes decisions to enter into external downtown office
space leases. We examined which standards or criteria are
58
Abstracts of Recently Completed Audits
used to decide when and where to lease, and the effect the
lease process has on optimizing City-owned office space.
Key findings were as follows: the City is using almost all
of the office space it owns downtown. Even if all of the
vacant City owned space were fully utilized, the City
would still need to lease office space. Although the City
has appropriate standards and criteria to administer its
leases, the leasing function is neither formalized, nor does
it follow a planned schedule. The City also does not always
follow policy in regards to leasing external office space. This
inconsistency can cost more money and not fully optimize
the City’s use of the office buildings it owns.
SUMMARY: Internal Audit reviewed 43 Managing
for Results performance measures from five County
agencies for accuracy and reliability. Twenty-two (51%)
measures were certified and 21 (49%) were not certified.
The primary reasons measures were not certified were a
lack of supporting documentation and inadequate formal
procedures for collecting, measuring, and reporting
performance.
MISCELLANEOUS: Front yard and Boulevard
parking – Improvements needed to enhance Program
effectiveness (February 2012)
CONTACT INFORMATION
Kari Guy
1221 SW 4th Avenue, Room 310
Portland, OR 97204
503.823.3544
[email protected]
www.portlandonline.com/auditor/index.cfm?c=26649
MISCELLANEOUS: Residential Solid Waste:
Customer rates accurate, but monitoring should continue
( June 2012)
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
City of Toronto
55 John Street, Metro Hall, 9th Floor
Toronto, Ontario M5V 3C6
416-392-8030
[email protected]
SUMMARY: The City of Portland regulates residential
garbage collection through a franchise system. We
conducted this audit to determine whether the franchise
and rate setting process provides fair rates for customers
and haulers, and whether the City is accurately collecting
and using solid waste franchise fees. We found that
the City's rate model is based on verified hauler costs,
and is consistent with the methodology used by other
jurisdictions in the region. We also found that the City
is accurately collecting and using solid waste franchise
fees. However, we noted that incentives and disincentives
applied to rates depart from the cost of service, and
recommended that the City either eliminate these
incentives or clearly document the rationale and expected
outcome.
SUMMARY: The objectives of this audit were to
determine compliance with relevant policies and
procedures, evaluate the program for effectiveness and
efficiency, determine whether parking revenues were
accounted for properly and evaluate internal controls
related to this program. This audit reviewed the City's
internal controls over the front yard and boulevard parking
permits. Our recommendations included establishing
a centralized monitoring and management model for
all districts involved in this program, building a set of
consistent practices and strengthening controls over the
collection and write-off of outstanding permit fees.
MISCELLANEOUS: Special Request Report on Shortterm Rentals (A and B) (May 2012)
MISCELLANEOUS: Performance Measure
Certification ( July 2012)
CONTACT INFORMATION
Naomi Marmell
Office of the City Auditor
301 W 2nd Street, Suite 2130
Austin, TX 78701
512-974-1372
[email protected]
CONTACT INFORMATION
Eve Murillo, Deputy County Auditor
Maricopa County Internal Audit Department
301 W. Jefferson St., Ste 660
Phoenix, Arizona 85003
(602) 506-7245
[email protected]
59
Abstracts of Recently Completed Audits
SUMMARY: This project compared the use of 311 and
911 between identified short-term rentals and a sample
of residential properties. No significant differences in
common call types, average number of calls, or percent
of properties with calls were noted. The project also
tracked the change in number of short-term rentals listed
on websites, and found that total listings increased from
approximately 2000 in February 2012 to approximately
2000 in April 2012, largely due to the increase in listings on
one website.
level objectives and expected accountability to the City.
In fulfilling its mandate, TCHC has established eight
wholly owned subsidiary companies. In addition, TCHC
participates in five joint ventures. TCHC also has an
equity position in a number of other entities. The major
objective of the review was assess if there was compliance
with provisions in the Shareholder Direction between the
City and the TCHC. While we recognize the need to
strike a balance between parental oversight and subsidiary
independence, the extent of current TCHC and City
oversight is limited.
MISCELLANEOUS: Sustainability Management
Follow-up Strong foundation created ( June 2012)
PARKS AND RECREATION
CONTACT INFORMATION
Brian Evans
Metro Auditor
600 NE Grand Ave
Portland, Oregon 97232
503-797-1904
[email protected]
www.oregonmetro.gov/auditor
PARKS AND RECREATION: Forestry Management
Audit (August 2012)
CONTACT INFORMATION
Rachel Snell, Assistant City Auditor
Office of the City Auditor
301 West 2nd Street
Austin, Texas 78701
512-974-2805
[email protected]
www.austintexas.gov/auditor
SUMMARY: Metro made significant progress on the
recommendations from the 2009 audit, Sustainability
Management: Focus efforts and evaluate progress.
We found nine of the eleven recommendations were
implemented and two were in process. Metro created
a strong foundation for its sustainable business model.
Institutionalizing these efforts into everyday management
decisions will help Metro make progress towards its longterm goals.
SUMMARY: The City’s Urban Forester and Forestry staff
seeks to provide public tree care services in order to provide
the Austin community with a safe and healthy urban
forest. In FY 2012, the Forestry group’s approved budget
was $1,660,575 with 24 FTEs. The duties of the Urban
Forester are defined in City Code chapter 6-3, which also
requires the Urban Forestry Board to develop and revise a
Comprehensive Urban Forest Plan for the Urban Forester
to administer. Our audit objective was to evaluate whether
the Parks and Recreation Department (PARD) Forestry
group is achieving City Code-defined objectives. Several
key elements for a successful forestry program are not in
place, limiting the ability of the Forestry group to comply
with City Code. The City’s Urban Forestry Board has
not established a Comprehensive Urban Forest Plan, and
the Urban Forester has not presented a standard of care
for trees and plants to the Urban Forestry Board. Because
these are not in place, the Urban Forester is unable to
comply with duties as mandated in City Code. In addition,
we identified operational inefficiencies and control
weaknesses, such as inefficient forestry planning and
staffing issues, no supervisory review of work performed,
and information system challenges that hinder the effective
management of the City’s urban forest.
MISCELLANEOUS: The City and Toronto Community
Housing Corporation Needs to Strengthen its Oversight
of Subsidiaries and Other Business Interests (February
2012)
CONTACT INFORMATION
Jeffrey Griffiths, Auditor General
City of Toronto
55 John Street, Metro Hall, 9th Floor
Toronto, Ontario M5V 3C6
416-392-8030
[email protected]
SUMMARY: The Toronto Community Housing
Corporation (TCHC) is wholly owned by the City of
Toronto. In establishing TCHC, the City approved a
Shareholder Direction that set guiding principles, high
60
Abstracts of Recently Completed Audits
Cemetery Contract Review audit. The audit scope includes
the steps taken by PARD since 2010 to address the two
recommendations from the original audit. We determined
that the two recommendations from the June 2010 audit
are underway. In addition, based on our observations, the
overall condition of the cemeteries has not significantly
improved since our original audit.
PROCUREMENT & CONTRACT
COMPLIANCE
PROCUREMENT & CONTRACT COMPLIANCE:
American Recovery and Reinvestment Act (ARRA)
Funding Performance Audit (May 2012)
CONTACT INFORMATION
Sonia Montano, Audit Supervisor
City and County of Denver Auditor's Office
201 W. Colfax Ave., Dept. 705
Denver, CO 80202
720-913-5157
[email protected]
www.denvergov.org/auditor
PROCUREMENT & CONTRACT COMPLIANCE:
Court Tower-Audit of Contract Terms and Conditions
(April 2012)
CONTACT INFORMATION
Christina Black, Audit Supervisor
Maricopa County Internal Audit Department
301 W. Jefferson St., Ste 660
Phoenix, AZ 85003-2148
(602) 506-7430
[email protected]
www.maricopa.gov/internal_audit
SUMMARY: The purpose of the audit was to follow
up on previous audit recommendations regarding the
management of the city’s American Recovery and
Reinvestment Act (ARRA) funding and to assess program
effectiveness. Overall, we found that ARRA funds have
had a tangible, long-term impact on the city. However,
we identified opportunities to improve citywide grant
administration going forward. Specifically, enhanced
oversight of grant amendments is necessary, all recipients of
grant sub-awards should be chosen through a competitive
bid process, and Fiscal Accountability Rules related to the
grant closeout process should be expanded.
SUMMARY: The purpose of the review was to determine
if the contractor billed the County in accordance with
various Guaranteed Maximum Price contract terms and
conditions. Moss Adams reviewed billings submitted
by the contractor and subcontractors from April 1, 2010
through September 30, 2011. Under the direction of
Internal Audit, the consulting firm Moss Adams LLP
reviewed work-in-progress expenditures and project
controls for the Downtown Court Tower construction
project. Expenditures subject to review totaled $240
million. The audit identified three instances where
construction controls could be improved.
PROCUREMENT & CONTRACT COMPLIANCE:
Cemetery Contract Follow-Up Audit (August 2012)
CONTACT INFORMATION
Niki Raggi
Office of the City Auditor
301 West 2nd Street, Suite 2130
Austin, Texas 78701
(512) 974-2073
[email protected]
http://www.austintexas.gov/page/archive-auditor-reports
PROCUREMENT & CONTRACT COMPLIANCE:
Purchasing Card Program ( July 2012)
CONTACT INFORMATION
Janet McWilliams
Office of the City Auditor
200 North Walker Suite 212
Oklahoma City , OK 73102
405-297-2186
[email protected]
www.okc.gov/auitor/index.html
SUMMARY: In June 2010, the Office of the City
Auditor (OCA) issued an audit report titled Cemetery
Contract Review. The original report contained two
recommendations aimed at clarifying contractual
responsibilities and establishing and implementing an
effective contract monitoring system. Our objective for
this audit was to determine whether, and to what degree,
Parks and Recreation Department (PARD) management
has implemented the recommendations from the original
SUMMARY: The objectives of the audit were to
evaluate the adequacy and determine the effectiveness of
controls over ProCard purchases during the six months
ended March 31, 2012 and to evaluate the status of
61
Abstracts of Recently Completed Audits
recommendations and related management responses
included in our previous audit of the ProCard program
dated November 23, 2010. Established controls over
ProCard purchases during the six months ended March
31, 2012 were adequate and operating effectively.
Recommendations included in our previous ProCard audit
report dated November 23, 2010 have been substantially
addressed, however procedures for verifying vendor
registration and on-site review of ProCard purchases could
be further enhanced. We recommended management
compare vendor names associated with vendor numbers
in the banking software with those in the City's financial
system to identify transactions where cardholders entered
incorrect vendor numbers and allow assessment of
those transactions to determine if purchases were from
unregistered vendors. We also recommended management
use a less predictable method for selecting cardholders to
review and increase the timeliness of scheduled reviews to
enhance the effectiveness of the on-site review procedures.
PUBLIC SAFETY: Performance Audit of King County
Sheriff 's Office and Law Enforcement Oversight ( July
2012)
CONTACT INFORMATION
Susan Baugh
King County Auditor's Office
516 - 3rd Ave. Rm. W1033
Seattle, WA 98104
206-296-0376
[email protected]
[email protected]
http://www.kingcounty.gov/operations/auditor/
Reports/Year/~/media/operations/auditor/
documents/2012Documents/KCSO_OLEO_Report_
FINAL/KCSO_OLEO_Report_FINAL.ashx
SUMMARY: This audit of the Sheriff ’s Office Internal
Investigations and Office of Law Enforcement Oversight
(OLEO) functions is the first in a series of annual audit
reports mandated by the County Council in Ordinance
16511. Conducted in conjunction with a national law
enforcement consulting firm, Hillard Heintze LLC, the
purpose of this audit was to evaluate the current state
of the Sheriff ’s Office internal investigation operations
and practices, and assess the effectiveness of OLEO in
providing council-directed oversight of the IIU. The
audit also included a review of national best practices for
managing citizen-initiated and internally generated police
misconduct and use of force complaints. Significant issues
with KCSO’s policies for investigating and documenting
complaints, and inconsistent adherence to those policies
among KCSO units undermine organizational and
individual accountability. KCSO’s inability to enforce its
procedures for complaints and policy violations was also
inconsistent with the Commission on Accreditation of
Law Enforcement Agencies (CALEA) standards or best
practices.
PUBLIC SAFETY
PUBLIC SAFETY: Seattle Police Department's In-Car
Video Program ( June 2012)
CONTACT INFORMATION
Jane Dunkel
City of Seattle Office of City Auditor
700 Fifth Ave, Suite 2410
PO Box 94729
Seattle, Washington 98124
206-684-7892
[email protected]
www.seattle.gov/audit
SUMMARY: Seattle City Councilmember Nick Licata
asked us to review the Seattle Police Department’s (SPD)
In-Car Video (ICV) Program to determine how many ICV
recordings SPD regularly makes, how many public requests
for copies of these recordings SPD receives, and how
many of these requests SPD is able to fulfill. To answer
these questions, we analyzed data from SPD’s IT Unit
and reviewed SPD’s Public Disclosure and Video Units’
paper records. We found that of the 166 public disclosure
requests for in-car videos SPD received during our 3 month
sample period, SPD provided copies to 40 percent of
requestors, was unable to locate videos for 25 percent of
requests, and was exempt under Washington State law from
disclosing 32 percent (3 percent were duplicate requests).
We make six recommendations for how SPD can improve
its In-Car Video Program.
PUBLIC SAFETY: Sheriff ’s Office Patrol Service
Agreements ( July 2012)
CONTACT INFORMATION
Eve Murillo, Deputy County Auditor
Maricopa County Internal Audit Department
301 W. Jefferson, Suite 660
Phoenix, Arizona 85003
(602) 506-7245
[email protected]
www.maricopa.gov/internal_audit
62
Abstracts of Recently Completed Audits
shortly after the back billed adjustments were posted to
accounts.
SUMMARY: Numerous governmental entities (cities/
towns, special districts, and agencies) enter into agreements
with the Maricopa County Sheriff 's Office (MCSO)
to provide law enforcement services. Our primary
objective was to review controls over the administration
and monitoring of the agreements. The audit focused
on MCSO's pricing calculations, cost recovery, revenue
collection and compliance with statutes and policies.
Agreed-upon costs were reasonably captured but there was
no documentation to support the justification for which
costs should be charged or excluded from chargeback to the
entities. In addition, MCSO is not tracking expenditures
associated with each agreement and cannot determine the
total cost to provide services. One agreement required
full cost recovery; however, we found that MCSO had
subsidized the cost of service by approximately $500,000
over a four year period as a result of inadequate rates and
unbilled administration fees. Substantially all of the
$48 million in patrol agreement revenue reviewed was
collected. However, we found that a revenue reconciliation
process was not in place and that payments by entities were
not made timely.
PUBLIC UTILITIES: Follow-Up Audit of Austin Water
Utility Water Loss ( June 2012)
CONTACT INFORMATION
Kathie Harrison, Auditor
Office of the City Auditor
301 West 2nd Street, Suite 2130
Austin, Texas 78701
(512) 974-2805
[email protected]
http://www.austintexas.gov/page/archive-auditor-reports
SUMMARY: In 2009, the Office of the City Auditor
issued an audit report titled Austin Water Utility Water
Loss. The report contained 23 recommendations aimed
at improving the accuracy, quality, and availability for the
Utility to use in its planning process.
Our objective was to confirm whether, and to what
degree Austin Water Utility (AWU) management has
implemented selected recommendations from the original
audit. After assessing the risk associated with each
recommendation, we chose four for testing. The audit
scope included steps taken by AWU management since
2009 to implement these four recommendations.
PUBLIC UTILITIES
PUBLIC UTILITIES: Atlanta Department of
Watershed Management Back Billing of July 2008 Rate
Increase ( July 2012)
PUBLIC UTILITIES: Portland Water Bureau: Further
advances in asset management would benefit ratepayers
( June 2012)
CONTACT INFORMATION
Stephanie Jackson
City of Atlanta City Auditor's Office
68 Mitchell Street
Suite 12100
Atlanta, Georgia 30303
404.330.6678
[email protected]
www.atlaudit.org
CONTACT INFORMATION
Beth Woodward, Senior Management Auditor, CIA,
CGAP, CCA
Contact Information: Office of City Auditor, Audit
Services Division
1221 SW 4th Avenue, Room 310
Portland, OR 97204
(503) 823-3543
[email protected]
www.portlandonline.com/auditor/index.cfm?c=26649&
SUMMARY: We undertook this audit at the request of
the Atlanta City Council. The City Utilities Committee,
in Resolution No. 09-R-0104, asked us to conduct “an
analysis of the department’s customer billing and service
termination activities, including but not limited to
commercial and residential bills and service terminations
issued in December of 2008 and January of 2009.” The
department, Atlanta City Council, and local media
received numerous customer complaints regarding water
billing, meter functionality and shutoffs in early January,
SUMMARY: Our primary audit objective was to
determine the status and results of the Water Bureau’s
efforts to manage its physical assets. Although a
recognized leader among U.S. utilities in implementing
asset management (AM) principles, we found that the
Bureau’s slow data management progress limited its efforts
to manage assets cost-effectively. We also found that
63
Abstracts of Recently Completed Audits
the Bureau was not using service levels systematically in
budgeting and rate setting, and had completed neither an
overall AM plan nor most asset-specific plans. The report
summarizes key AM principles and benefits. It includes
nine recommendations for improving capability to benefit
ratepayers.
PUBLIC WORKS
PUBLIC WORKS: Public Works - Capital Street
Project Construction Administration ( June 2012)
CONTACT INFORMATION
Lori Rice, Audit Manager
Office of the City Auditor
200 North Walker, Ste. 212
Oklahoma City, Oklahoma 73102
405-297-3858
[email protected]
www.okc.gov/auditor/index.html
SUMMARY: The objective of this audit was to evaluate
the adequacy and determine the effectiveness of capital
project construction administration procedures established
by the Public Works Department to ensure the timely
completion of street projects final accepted during fiscal
year 2011. Based on the results of our audit, we believe that
established procedures are not adequate to ensure timely
completion of capital street projects; construction timelines
and processes for managing those timelines are not
adequately developed; and project timeline information is
not adequately accumulated and assessed.
64
How To Submit Abstracts & Quarterly Content
FOR ABSTRACTS
1. Log on to algaonline.org. You must log on to view & access the form.
2. Navigate to the online submission form: Resources > Abstracts > Submit an Abstract.
3. Submit! Enter (type or copy/paste) and submit your abstract details as prompted in the form.
Abstract Summaries and Descriptions will not be published in the Quarterly, but will be displayed and searchable online.
For the Summary, please provide a single paragraph describing what you did and what you found. The description
may include scope, background, objectives, significant findings, unique methodologies, recommendations, or other
information you wish to share with ALGA members.
Suggested Abstract Categories… include the following:
Contract Compliance
Finance
Information Technology
Parks and Recreation
Procurement
Public Safety
Public Works
Public Utilities
Social Services
Miscellaneous
(sample on-line Abstract submission form)
65
How To Submit Abstracts & Quarterly Content
FOR ARTICLES AND MEMBER NEWS ITEMS
Articles may be submitted for any topic, but those relating to advertised themes will receive first priority in any given
Quarterly. Upcoming themes and submission titles are as follows:
•
•
•
•
Winter 2012 due November 15: Impactful Audit Practices – Fieldwork
Spring 2013 due February 15: Impactful Audit Practices – Analytical techniques
Summer 2013, due May 15: Impactful Audit Practices – Reporting
Fall 2013 due August 15: How-To References – Compilation of step-by-step tools, techniques, and activities to enhance
performance auditing
Format guidelines:
•
•
•
•
•
•
•
•
One page is roughly equivalent to 450-500 words
Target length is a maximum of 1,500 words, which translates roughly to four pages with the inclusion of visual
elements such as photos or charts
For each article, please include a suggested headline, author’s name, title and organization represented
Article text should be submitted separately, in an unformatted Word file
To maximize the quality of graphics for both print and web presentations, author headshots and other graphics must
be submitted separately from the article text in jpg form
Indicate the desired location of the any graphics within the article by enclosing the instruction in brackets. For
example, [INSERT GRAPH 1 here]
Please include the proper attribution of any copied elements within your article to their respective sources
Remember to thoroughly proofread your article
Member News may be submitted for member promotions, certifications, awards, retirements, or other relevant ALGA
member news.
Questions? If you have questions or encounter problems submitting an Article, Abstract or Member News item, please
contact the ALGA editor at [email protected].
66
Association of Local Government Auditors
Membership Application
Federal Identification Number: 23-2539807
The Association of Local Government Auditors was formed to bring together professional local government auditors, to provide
opportunities for the free exchange of information, to offer pertinent education and training, to improve the quality of auditing in
local government, to provide a forum for local government auditing issues, and to encourage and uphold the highest quality
standards of professional ethics. Federal and state, or other interested parties may join as associate members.
Organization Name______________________________
Organizational Membership
Address_______________________________________
(based on the total number of auditors in your local government
audit organization)
______________________________________________
City____________________State______ Zip_________
Telephone_____________________________________
Fax___________________________________________
Website Address _______________________________
Key Contact____________________________________
Title___________________________________________
Email_________________________________________
Please remit the completed membership form with payment
(or payment information) to:
ALGA Member Services
449 Lewis Hargett Circle, Suite 290
Lexington, KY 40503
ˆ 1-2
ˆ 6-10
ˆ 16 +
ˆ 3-5
ˆ 11-15
$180 US
$475 US
$910 US
$260 US
$675 US
Individual Membership 1 ˆ$325 US
(any local government employee, other than the head of the audit
organization, whose primary function is auditing local government
entities)
Associate Membership 1 ˆ$325 US
(CPA firms, individual CPAs, state or federal auditors and other
interested parties)
Plus New Members joining between April 1 & June 30
(see page 4 of Membership Application)
ˆ 1-10
ˆ 11+
$100 US
$200 US
To pay by Visa or MasterCard, please provide:
ˆ Check here to receive a paper copy of the member
directory. (You will NOT receive a paper copy unless this
box is checked!)
Cardholder Name: _______________________________
# of Local Government Auditing Quarterly Copies
ˆ0
ˆ1
ˆ2
ˆ 3
(Maximum allowed based on info in graph on page 3)
ˆ Joining as a referral? Who referred you to ALGA?
Telephone:_____________________________________
Billing Address:__________________________________
______________________________________________
Cardholder Email:________________________________
Card Number:___________________________________
Exp. Date:________Card Type:_____________________
_____________________________________________
Audit Organization Reports to:
Type of Jurisdiction:
a. (
b. (
c. (
d. (
e. (
f. (
g. (
a. (
b. (
c. (
d. (
e. (
f. (
g. (
h. (
) Elected Auditor/Public
) Council/Board
) Audit Committee
) Executive Officer
) CFO/Finance
) Other
) Not Applicable
) City
) County
) Schools/Universities
) Transportation Authority
) City and County
) State/Province
) Utility
) Other (describe)
Type of Audits Performed:
________________________
Financial
Performance
Other
% of Total
Audits
Performed
_________
_________
_________
Total number of
auditors on staff ______
Total audit
organization staff ______
Annual Jurisdiction
Operating Budget:
a. (
b. (
c. (
d. (
e. (
) up to $25 million
) $25 to $100 million
) $100 to $500 million
) $500 million to $1 billion
) Over $1 billion
Total Jurisdiction
Employees:
a. (
b. (
c. (
d. (
e. (
) up to 1,000
) 1,000 to 5,000
) 5,000 to 10,000
) 10,000 to 25,000
) over 25,000
LGAQ
ALGA Member Services
449 Lewis Hargett Circle, Suite 290
Lexington, KY 40503-3590
Presort Standard
US Postage
Paid
Permit 850
Lexington, KY