PANCEA Manual
QUALITY ASSURANCE
Quality Assurance Checks
This section is organized into a discussion of general issues and a list of specific data quality
checks.
General Issues
The following represent several important general checks on the data.
Complete the instruments
We need a response to each for every question asked, even if this response is “Not Applicable”
(N/A) or "Don't Know" (DK). Blank cells should imply that there is a plan to ask these questions
later, not that they have simply been forgotten. For a fuller discussion of data codes, see the
manual chapter “Data Collection”
Completeness with actual data is most essential for “forest” issues such as data on key outputs.
DK for “tree” issues (e.g., cost details) are undesirable, but having some is acceptable.
Provide adequate detail
Enough information should be provided to accurately determine costs and outputs for
subsequent data cleaning and analysis. For example:

Capital goods and recurrent goods representing $1000 or more in spending should be
sufficiently detailed to later determine market prices from alternative suppliers. See
“Costing Techniques” for more discussion of market pricing.

Recurrent services that are contracted out should be explained well enough to indicate
the nature of the services provided. E.g., ”Provided a workshop for VCT counselors
including refresher training for all counselors and an advanced training for supervisors”
would be sufficiently detailed, whereas “Training workshop” would not. If the resource
content of a contract is available, please include this as well (e.g., 70% personnel
wages; 20% per diems and 10% supplies).

Personnel job categories and titles should have explanatory notes as needed to ensure
that we have a good idea of the responsibilities that each person carries. “Medical
Director” at an STI clinic would be sufficiently informative whereas “MD” would not as
good.
Ensure agreement across sections and instruments
Data should be logically consistent within the same instrument and across instruments. For
example:



The work start date given on the “Personnel Roster” of the ARQni cannot be later than
the work end date.
ARQni and ADCni cost data should be identical if they were tabulated for the same
period of time and for the same item.
ADC intervention outputs should be consistent with the output data provided in HIPPI II
and in ARQ intervention sheets.
 G-1 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual

QUALITY ASSURANCE
ARQni should have costs for supplies, such as condoms or test kits, the use of which is
documented in ARQ intervention sheets.
Base data are on written records as much as possible
Written records should be the primary source of information when available. Data source
information should be complete so that there is information on how data were collected and an
indication of its quality (e.g., in general, written records are more accurate than recall). See the
“Data Collection” chapter for detail on data sources.
Provide units
If units of an item are ambiguous, types and quantities should be specified. For example, when
prices are in any currency other than the local currency, the currency type should be noted in
the comments fields. Another example: if 500 “boxes” of condoms were donated, we need to
know how many individual condoms are in each box.
Specific quality checks
In this section, we list specific data quality assurance checks that the data teams should do
within and between instruments. We have done this for ARQni, ARQsti, and ARQvct. The
quality checks for other ARQ intervention sheets and for ADC will be similar in style, and largely
overlapping due to shared questions.
Items in bold are of particular importance. These are “forest” issues and are usually critical to
the cost-effectiveness ratio or to a basic understanding of the intervention’s services and their
effectiveness. See especially Q10 in ARQsti and Q13 and 1Q4 in ARQvct. These questions
pertain to the are the key outputs for these two interventions.
See
note1
Q#
Quality Assurance
Check?
ARQ ni
0.Cover
y
y
?
y
All
4.
8.
9.
A response is given for each question, even if the response is "don't
know" or "not applicable".
1.Facility Characteristics
The date is more recent than the date given in Q3.
The selected interventions include those indicated in HIPPI, part 1, Q1
& Q2.
The date is the same or more recent than the date given in Q4.
1
"DB Validation" means that the MS Access database will be able to do the quality assurance
automatically. "y" = yes. "n" = no. "b" = both data collectors and the MS Access database should
examine.
 G-2 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
?
y
y
y
10.
11.
15.
16.
20.
y
y
y
21.
22.
All
QUALITY ASSURANCE
The selected interventions include those checked in HIPPI, part 1, Q1
& Q2.
The date given is in the past (or earlier this year).
The answer is < = 7.
The answer is < = 24. Make sure the answer applies to the in
The area should be the catchment area from which most patients
come; not the entire country.
The number is < = to the number given in Q20, for each column.
The number is < = to the number given in Q19, for each column.
A response is given for each question, even if the response is "don't
know" or "not applicable".
2.Governance
b
y
5.
If “6. never”, then Q6 should be “0”.
9.
If the answer to Q8 is "yes", an appropriate description of performance
measurement is given here.
All
A response is given for each question, even if the response is "don't
know" or "not applicable".
3a Buildings
5.
y
8.
10, 15
y
y
b
y
y
y
n
n
n
13.
18.
30.
30..
31.
32.
All
1.
1.
2.
2,
n
2 and
3.
This area must be smaller than the figure given in Q6.
The date given is equal to or more recent than the date given in
"Facility Characteristics" Q3.
Rent or mortgage should match the values in question 5 of the ADCni,
“Building” sheet.
The date given is equal to or prior to the date given in Q8.
The date given is equal to or prior to the date given in Q8.
The answer given to Q29 is "yes", if the answer given to Q25 & Q26 is
"yes" for any building.
A source is provided if the answer to Q29 is "yes".
The answer is < = 24.
The answer is < = 7.
A response is given for each question, even if the response is "don't
know" or "not applicable".
3b.Personnel Roster
Different staff with the same initials are marked with a "(1)", "(2)", etc.
after the relevant initials.
The same staff member with multiple rows of data is given the same
initials in all rows of data.
A clear description is conveyed of the overall responsibilities of each
staff member.
Amy acronyms that we are likely to be unfamiliar with are
explained (e.g., TLS=TB Liaison Supervisor and DDHS=Director
of District Health Services )
All volunteers and trainees have a job category describing their
responsibilities accompanying their title.
 G-3 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
2. and
3.
n
7.
y
y
y
9.
10.
10.
14.
y
y
17.
y
17.
y
21.
22.
18.,
19.
n
All
y
2.
y
2., 6.
n
5.
7.
n
21.
All
y
1.
n
n
n
y
y
1.
1.
2.
5, 8,
11, 12
QUALITY ASSURANCE
All staff members whose job titles have changed over time have
multiple rows of data.
The date given is equal to or more recent than the date given in
"Facility Characteristics" Q4.
If an amount is provided in Q10, the answer here is "yes".
The amount given is in local currency.
If not, a note should document the type of currency used.
People with higher job classifications should almost always be
receiving higher pay than those with a lower classification. Where this
is not true, a brief note is needed.
If the answer to Q15 is "yes", then the answer given here is "yes" as
well.
The answer is > 0 and < = 60.
If an answer outside this range is given, an explanation is provided.
The answer is > 0 and < = 60.
If an answer outside this range is given, an explanation is provided.
% allocations sum to 100%, for each row of data.
If “yes” for any row, Q23 and Q24 should be “NA”
Efforts were made to complete these questions as much as possible.
Not all answers are "don't know".
If most answers are "don't know", an explanation is provided
regarding why these data were not collected.
A response is given for each question, even if the response is "don't
know" or "not applicable".
3c. Personnel turnover
The date given is today's date or in the past.
The date given in Q2 + the time given in Q6 approximates "3c
Personnel Roster", Q7 (the start date) for the replacement staff
member.
If “no” for any row, then Q6 is “NA” for that row.
The answer given is roughly on the same scale as "3b(1) Staff Roster"
Q10 for the replacement staff member.
Allocations are a very important item as these figures have a
large effect on how intervention costs are ultimately tabulated.
Please spend sufficient time to get very good estimates of these
data.
A response is given for each question, even if the response is "don't
know" or "not applicable".
3d.Capital Goods
Descriptions are detailed enough to cost the item at a store later.
Detailed descriptions are only needed on items on which $1,000 or
more were spent in 12 months.
Data were collected from facility records as much as possible.
No items have been double-counted.
The date given is equal to or more recent than the date given in
"Facility Characteristics" Q3.
The amount given is in local currency.
 G-4 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
QUALITY ASSURANCE
If not, a note should document the type of currency used.
5.
8.
10.
y
y
13.
y
y
n
All
1
2.
3., 4.
4.
6 - 8.
y
Items
y
Items
n
Items
n
3., 4.,
n
7.
4.
y
y
8.
11.
y
12.
15.
All
2.
Value of each piece of equipment is the value of the item when
donated/purchased as opposed to a current valuation
The answer is "n/a" if the answer to Q7 is 0%.
If the answer to Q9 is "yes", than the proportion is > 0%.
Allocations are a very important item as these figures have a
large effect on how intervention costs are ultimately tabulated.
Please spend sufficient time to get very good estimates of these
data.
A response is given for each question, even if the response is "don't
know" or "not applicable".
3e.Recurrent Expenses, Goods
Adequate description is given for each individual item > $1000, to
allow market pricing.
If the answer to Q1 is "yes", a number is mandatory here.
If both are “0”, Q5 – Q9 should be “NA”.
If “0”, then Q3 must also be “0” for that item.
Amounts given in local currency.
If not, a note should document the type of currency used.
All line items from records are inserted, beginning at row 14, for
all line items that don't match exactly one of the items listed in
rows 1-10.
An explanation is provided for any item that is double-counted.
(This may occur any time line items in records subsume
individual items listed in rows 1-10.)
Adequate description is given for each line item, so that we can
determine what does and does not belong in each category of
recurrent expenses.
Units are noted as necessary (e.g., each unit is a "case of 100", rather
than as a single count).
Allocations are a very important item as these figures have a
large effect on how intervention costs are ultimately tabulated.
Please spend sufficient time to get very good estimates of these
data.
The answer is approximately 12 * Q3, if purchases are done evenly
over time
If not, a note is provided explaining the unexpected answer.
The answer is approximately 12 * Q7, if purchases are done evenly
over time.
If not, a note is provided explaining the unexpected answer.
If this is more than zero, the “Recurrent goods” row of Q3 of “3a.
Total Facility Expenditures” sheet must also be more than zero.
If “0”, Q13 must be “NA”
% allocations sum to 100%, for each row of data.
A response is given for each question, even if the response is "don't
know" or "not applicable".
3f.Recurrent Expenses, Services
An explanation is provided for any item that is double-counted.
(This may occur any time line items in records subsume
 G-5 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
3, 4, 6
n
2.
y
y
5.
6.
7.
Items
n
Items
n
Items
y
All
y
y
y
4.
8.
9.
All
QUALITY ASSURANCE
individual items listed in the “Service description” column.)
The amount given is in local currency.
If not, a note should document the type of currency used.
The answer is approximately 12 * Q1, if purchases are constant over
the year.
If not, a note is provided explaining the unexpected answer.
% allocations sum to 100%, for each row of data.
If this is more than zero, the “Recurrent services” row of Q3 of “3a.
Total Facility Expenditures” sheet must also be more than zero.
Allocations are a very important item as these figures have a
large effect on how intervention costs are ultimately tabulated.
Please spend sufficient time to get very good estimates of these
data.
Adequate description is given for each line item, so that we can
determine the nature of the service and, if possible, the resources
used to deliver it (see manual).
Spending for labor is reported either here or in Personnel, but not
in both places.
All line items from records are inserted, beginning at row 20, for
all line items that don't match exactly one of the items listed in
rows 1-10.
A response is given for each question, even if the response is "don't
know" or "not applicable".
4.Finance
Percentages sum to 100% or less.
The answer is > = 0% and < =100%.
Answers sum to 100%.
A response is given for each question, even if the response is "don't
know" or "not applicable".
5a. Inputs -- facility expenditures
1.
2.
4.
5.
7.
8.
9.
“Last month” should generally not exceed “Most recent FY” and
usually will be much smaller
Data should cover the entire 12 months of the most recent FY. This
can be done either by using the “month” column or the “quarter”
column.
“Last month” should generally not exceed “Most recent FY” and
usually will be much smaller
Data should cover the entire 12 months of the most recent FY. This
can be done either by using the “month” column or the “quarter”
column.
“Last month” should generally not exceed “Most recent FY” and
usually will be much smaller
Data should cover the entire 12 months of the most recent FY. This
can be done either by using the “month” column or the “quarter”
column.
For the same person the “Job title” should be the same one used in
question 3 of “Personnel roster”
 G-6 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
All
QUALITY ASSURANCE
A response is given for each question, even if the response is
"don't know" or "not applicable".
5b. Outputs -- facility expenditures
1.
n
2.
4.
The list of services should include those services that constitute
the bulk of the facility’s function.
“Last month” should generally not exceed “Most recent FY,” usually
will be much smaller, and may equal about 1/12 as much.
Data should cover the entire 12 months of the most recent FY, if
available. This can be done either by using the “month” column or the
“quarter” column.
ARQvct
1.
The date given is 1-2 months prior to the current date.
2.
This must be a 12-month period.
2.
This period should rely on records, if any records exist.
3.
This date is in agreement with the date given in the ARQni, facility
characteristics, Q9.
If “yes” for “Was this the same for ALL of the most recent fiscal
year?” then the answer must be “NA” for “(If "no") What did you
provide previously? “and for “(If "no") When did you make this
change?” This rule holds for all questions with this structure.
4.
6,
6.
7.
8.
9,
10.
11
11.
11.
12.
13.
13.
No one listing should be > than the global answer given in the
first row of Q6.
“Men” > = “MSM; “Women > =”Pregnant women”
Q7 < = Q8, because pre-test counseling is usually done in groups.
Men” > = “MSM; “Women > =”Pregnant women”
Q9 < = Q8. Number of tests administered will almost always be
less than the number of individuals receiving pre-test counseling,
and will never be more.
Q10 < Q9
Q11 < Q10
No one listing should be > than the global answer given for Q11.
Men” > = “MSM; “Women > =”Pregnant women”
Q12 < = Q13
Q13 > = Q12
This is one of two of the most essential output information
items – both for the last month and for the last fiscal year by
time period. Whatever information is available should be
entered. Maximum data collection effort indicated.
 G-7 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
QUALITY ASSURANCE
This is one of two of the most essential output information
items – both for the last month and for the last fiscal year by
time period. Whatever information is available should be
entered. Maximum data collection effort indicated
This is an important question. Take enough time to get a
detailed answer.
If one on one, then Q7 should = or be very close to Q7, (i.e.,
sessions = individuals)
If Q31. > 40 hours, services must be offered on some evenings or
weekends (Q32).
Q35 < = Q36. Could be on the same day if using rapid tests.
If Q42 = “0”, Q41 = “0” or “NA”
If a price is listed in Q40., then Q42. can't be 100%. (In the rare
case that this happens, a note should be provided.)
If Q42 = 100%, then Q39. must be "no" or “NA”.
14.
15.
20.
31.
35.
41.
42
39.
The answer should be reasonable, considering the number of
counselors listed on the staffing sheet in the ARQni, and given the
number of clients served. (Q6.)
> = the largest of the following: Q50, Q51, or Q52. "days"
questions.
Q57. should list different facilities than Q60.
43.
54
60.
ARQ sti
y
y
n
y
1.
2.
2.
3.
4.
6.
7.
8.
8.
No one listing should be > than the global answer given in the
first row of Q6.
“Men” > = “MSM; “Women > =”Pregnant women”
Q7 < = Q6.
Q8 > = Q6.
“Men” > = “MSM; “Women > =”Pregnant women”
8,
No one listing should be > than the global answer given for Q8.
6,
y
y
y
y
The date given is 1-2 months prior to the current date.
This must be a 12-month period.
This period should rely on records, if any records exist.
This date is in agreement with the date given in the ARQni, facility
characteristics, Q9.
If “yes” for “Was this the same for ALL of the most recent
fiscal year?” then the answer must be “NA” for “(If "no") What
did you provide previously? “and for “(If "no") When did you
make this change?” This rule holds for all questions with this
structure.
 G-8 
Version 7/31/2017
PAN_man_QA_v4
PANCEA Manual
y
QUALITY ASSURANCE
9.
No one listing should be > than the global answer given for Q9
10.
This is the most essential output information – both for the
last month and for the last fiscal year by time period. Whatever
information is available should be entered. Maximum data
collection effort indicated.
y
10.
y
y
y
10.
14.
15.
16.
b
y
y
y
y
y
y
y
23.
24.
31.
35.
36.
36.
39.
40.
41.
42.
43.
n
y
y
54
60.
Make sure that data is provided for a full 12 months in the
“Services: FY detail” section.
Respective answers are < = 9 for etiologic diagnoses
> = HIV listing in Q9.
< = HIV listing in Q10.
This is an important question. Take enough time to get a
detailed answer.
The answer for STI services should approximate facility hours
given in ARQni "facility characteristics" Q16.
If Q23. > 40 hours, services must be offered on some evenings or
weekends.
31< 30
The prices listed must be less than the prices given in Q32.
If a price is listed in Q35., then Q36. can't be 100%. (In the rare
case that this happens, a note should be provided.)
If prices were inclusive in Q33., then should be inclusive here, and
visa versa.
If Q36 = 100%, then Q39. must be "0".or “NA”
If a price is given in Q39., then the answer here must be < 100%.
If Q40. is 100%, the answer here should be "no".
The price given here should be less than the price given in Q39.
The answer should be reasonable, considering the number of
counselors listed on the staffing sheet in the ARQni, and given the
number of clients served. (Q6.)
> = the largest of the following: Q51, Q52, or Q53. "days"
questions.
Q57. should list different facilities than Q60.
 G-9 
Version 7/31/2017
PAN_man_QA_v4