UNDER EMBARGO UNTIL APRIL 29, 2008, 12:01 AM LOCAL TIME Compliance with Vaccination Recommendations for U.S. Children Elizabeth T. Luman, PhD, Kate M. Shaw, MS, Shannon K. Stokley, MPH Background: Official recommendations for the routine vaccination of U.S. children, made by the Advisory Committee on Immunization Practices (ACIP), specify the vaccines for administration, the number of doses that should be given, the age ranges for administration, the minimum ages at which doses are considered valid, the minimum intervals between doses within a series, and several additional vaccine-specific adjustments and exceptions. Federally reported estimates of vaccination coverage measure only compliance with the required number of doses; other recommendations are not routinely evaluated. Methods: Analysis of vaccination histories for 17,563 U.S. children aged 19 –35 months from the 2005 National Immunization Survey. Main Outcome Measures: Compliance with, and incremental impact of, each vaccination recommendation. Results: Estimated coverage was 72% for the standard vaccination series accounting for all recommendations, 9 percentage points lower than coverage based solely on counting doses. Overall, 19% of children were missing one or more doses, while 8% had received an invalid dose, and 9% were affected by other recommendations. The proportion of noncompliance due to missed doses versus other recommendations varied by state and by antigen. Conclusions: Approximately 28% of children were not in compliance with the official vaccination recommendations. Missed doses accounted for approximately two thirds of noncompliance, with the remainder due to mis-timed doses and other requirements. Measuring compliance with all ACIP recommendations provides a valuable tool to assess and improve the quality of healthcare delivery and ensure that children and communities are optimally protected from vaccine-preventable diseases. (Am J Prev Med 2008;xx(xx):xxx) © 2008 American Journal of Preventive Medicine Introduction O fficial recommendations for the routine vaccination of young children in the U.S. are made annually by the Advisory Committee on Immunization Practices (ACIP), in collaboration with the American Academy of Pediatrics and the American Academy of Family Physicians. These recommendations specify the vaccines and the number of doses that should be given, the ages or age ranges for routine administration, the minimum ages at which doses are considered valid, the minimum intervals between doses within a series, and several additional vaccine-specific adjustments and exceptions.1 The recommendations are designed to ensure that vaccinations provide maxFrom the National Center for Immunization and Respiratory Diseases (Luman, Stokley), and the National Center for Chronic Disease Prevention and Health Promotion (Shaw), CDC, Atlanta, Georgia Address correspondence and reprint requests to: Elizabeth T. Luman, PhD, CDC, 1600 Clifton Road NE, MS E05, Atlanta GA 30333. E-mail: [email protected]. imum effectiveness and protection against vaccinepreventable diseases.2 Over the past few years, the vaccination schedule has become increasingly complex with the addition of new vaccines and vaccine-combination options; this ever-growing complexity poses logistical challenges for vaccination providers in ensuring that their patients are appropriately vaccinated. Vaccination coverage estimates are used widely by national, state, and local public health departments, as well as vaccination providers, as a basis for programmatic and policy decisions: to evaluate the quality of vaccination services, to target additional services, and, when linked to surveillance data, to assess the success of vaccination strategies in preventing disease. Federally reported estimates of early-childhood vaccination coverage in the U.S., published annually by the National Center for Immunization and Respiratory Diseases of the CDC, measure the percentage of children who have received the appropriate number of doses of the recommended vaccines.3 This dose-counting measure of Am J Prev Med 2008;xx(xx) © 2008 American Journal of Preventive Medicine • Published by Elsevier Inc. 0749-3797/08/$–see front matter doi:10.1016/j.amepre.2008.01.033 1 vaccination coverage has been used for nearly 50 years to evaluate the success of vaccination programs in reaching children, to determine demographic risk factors for undervaccination, to identify geographic areas of low coverage, and to examine trends over time.4 In contrast, evaluation of the other components of the ACIP recommendations has been less thorough, and may give additional insight into the quality of healthcare services and any potential gaps in population protection. By examining compliance with each recommendation, public health staff can pinpoint both strengths and areas of improvement for individual immunization providers and develop provider-specific strategies to improve the delivery of vaccination services. This study evaluates compliance with ACIP recommendations, collectively and individually, to better understand the extent to which the recommendations are being followed. These ACIP-compliant estimates are compared with standard dose-counting estimates to determine the incremental impact of each recommendation on overall vaccination coverage. Methods National Immunization Survey Estimates of vaccination coverage among U.S. children aged 19 –35 months are derived annually from the National Immunization Survey (NIS). The NIS uses random-digit dialing to survey households with age-eligible children followed by a questionnaire mailed to the children’s vaccination providers to validate vaccination information.5 The NIS was approved by the CDC’s IRB, and caregivers provided verbal consent for participation and provider contact. This analysis was conducted in 2007 using data from the 2005 NIS, which included information for 17,563 children with a completed interview (84% of households identified with an age-eligible child, Council of American Survey Research Organizations [CASRO] response rate 65%) and adequate vaccination history from the vaccination providers (64% of children with a completed interview). Data were weighted to account for nonresponding households and vaccination providers, as well as lower vaccination coverage among children in households without land-line telephones, to ensure that estimates were representative of all children in the U.S. Details of the NIS methods have been reported elsewhere.5,6 Outcome Measures The ACIP-specified number of vaccine doses, the minimum ages at which doses are considered valid, the minimum intervals between doses within a series, and additional requirements are presented in Table 1.2,7–11 Pneumococcal conjugate vaccine was also recommended during the period when children in the 2005 NIS were receiving vaccinations (2002–2005); however, it was not included in this report because widespread vaccine shortages resulted in several temporary changes to the recommendations.12–19 2 The authors defined ACIP-compliant coverage as the percentage of children who did not need any additional vaccinations at the time of the interview. Vaccination doses administered more than 4 days prior to the approved minimumacceptable age were considered age-invalid. Similarly, vaccinations administered more than 4 days prior to the approved minimum interval between doses of a vaccine were considered interval-invalid, as were doses of live vaccine administered 1–27 days after another live vaccine. If a child received age- or interval-invalid doses, these doses were excluded as per ACIP standards, and status was determined based on the remaining doses. PEDIARIXTM vaccine (SmithKline Beecham Biologicals, Rixensart, Belgium)9 is a combination of diphtheria and tetanus toxoids and acellular pertussis vaccine (DTaP), poliovirus, and hepatitis B vaccines given at 2, 4, and 6 months. Children receiving PEDIARIXTM who were given a birth dose of hepatitis B vaccine thus receive an extra dose of hepatitis B vaccine at 4 months. Administration of this extra dose is acceptable,9,10 and the dose was not considered invalid. However, for ACIP-compliant coverage, it is not counted as one of the three required doses in the hepatitis B series, as per the recommendations. Additional adjustments were made for Haemophilus influenzae type b vaccine (Hib) and varicella vaccine. Fewer doses of Hib were required for children who received their first dose after age 6 months, as per ACIP recommendations. Also, because the NIS does not collect vaccine-manufacturer information, lenient assumptions were made regarding the need for the 6-month dose of Hib vaccine. If children who received three Hib doses were given the third dose prior to age 12 months (the required minimum age for the final dose using either the three- or four-dose schedule), they were considered not compliant, whereas if the dose was administered at ⱖ12 months, they were considered compliant. The varicella vaccination requirement was satisfied if the child received the vaccination, or if he or she had a history of varicella disease, as reported by the caregiver. Varicella doses administered during the 4-day grace period prior to age 12 months were considered valid. Dose-counting coverage for each vaccine was calculated as consistent with methods for data annually reported by CDC.3 The number of vaccine doses that each child had received by the time of the interview was counted. With the exception of varicella, all doses were included regardless of age at receipt; for varicella vaccine, doses given before age 12 months were excluded. Three doses of Hib were required. In addition to coverage for individual vaccines, dose-counting and ACIP-compliant coverage for the 4:3:1:3:3 vaccination series are reported. For dose-counting coverage, this series consists of at least four doses DTaP; three doses of poliovirus; one dose of measles, mumps, and rubella vaccine (MMR); three doses of Hib; and three doses of hepatitis B vaccine. For ACIP-compliant coverage, one to four doses of Hib vaccine may be required, depending on age at first dose and age at last dose, as previously discussed. Coverage with the 4:3:1:3: 3:1 series (4:3:1:3:3 plus one dose of varicella vaccine), which became the primary federally reported series outcome in 2006,3 is also presented. American Journal of Preventive Medicine, Volume xx, Number xx www.ajpm-online.net Table 1. Recommended and minimum ages and intervals for early-childhood vaccinations, 2005a Vaccination dose Recommended age for routine administration (months) Minimim acceptable ageb Diphtheria and tetanus toxoids and acellular pertussis 1 2 6 weeks 2 4 10 weeks 3 6 14 weeks 4 15–18 12 months Poliovirus 1 2 6 weeks 2 4 10 weeks 3 6–18 14 weeks Measles, mumps, and rubella 1 12–15 12 months Haemophilus influenzae type be 1 2 6 weeks 2 4 10 weeks (3)f 6 14 weeks 4 12–15 12 months Hepatitis B 1 0–2 Birth 2 1–4 4 weeks 3g 6–18 6 months Varicellah 1 12–18 12 months Recommended interval between doses (months) Minimum acceptable interval between dosesc 2 2 6–12 4 weeks 4 weeks 4 months 2 2–14 4 weeks 4 weeks d 2 2 6–9 4 weeks 4 weeks 8 weeks 1–4 2–17 4 weeks 8 weeks d a Approved by the Advisory Committee on Immunization Practices (ACIP), the American Academy of Pediatricians, and the American Academy of Family Physicians. See also www.cdc.gov/mmwr/preview/mmwrhtml/mm5301-Immunizationa1.htm. Pneumococcal conjugate vaccine is also recommended, but was not included in this analysis.2,7 b Doses given within 4 days before the minimum age for all vaccines are considered acceptable; doses given prior to this must be repeated.2 c Doses given within 4 days before the minimum interval are considered acceptable; doses given prior to this must be repeated.2 d Live vaccines (e.g., MMR and varicella) not administered on the same day should be administered ⱖ4 weeks apart. If live vaccines are separated by ⬍4 weeks, the vaccine administered second should not be counted as a valid dose and should be repeated.2 e Fewer doses of Hib are required for children who received their first dose after age 6 months. If the first dose is administered at age 7–11 months, two doses with a 2-month interval and a third dose at age 12–15 months are recommended; if the first dose is administered at age 12–14 months, two doses with a 2-month interval are recommended; if the first dose is administered after age 14 months, one dose is recommended.8 f In most cases, four doses with a third dose at age 6 months are recommended. However, the age 6-months dose is not required if polyribosylribitol phosphate-meningococcal outer membrane protein (PRP-OMP, PedvaxHib, manufactured by Merck) is used for the first two doses.8 g An additional dose of hepatitis B may be administered to children aged 4 months via the combination vaccine PEDIARIX™; while administration of this extra dose is acceptable, the minimum age for the final dose of hepatitis B vaccine is age 6 months.9,10 h For susceptible children (i.e., children who lack a reliable history of chickenpox as judged by a clinician)11 Analysis Estimates of percentages and SEs were calculated using SUDAAN version 9.0.1. For all statistical tests, the level of significance was set a priori at pⱕ0.05. All analyses were weighted and account for the complex sampling design of the NIS.5 Results Vaccination Coverage Estimated coverage in 2005 with the 4:3:1:3:3 vaccination series incorporating all ACIP recommendations was 72% (Table 2). This is 9 percentage points lower than coverage based solely on counting doses. ACIPcompliant coverage was lowest for DTaP and highest for poliovirus vaccine. Differences between the ACIPcompliant and dose-counting coverage measures were largest for Hib and hepatitis B, moderate for DTaP, and negligible for poliovirus and MMR vaccines. ACIPMonth 2008 compliant coverage was slightly higher than dosecounting coverage for varicella vaccine. Effect of Each Recommendation Overall, 19% of children were missing one or more doses in the 4:3:1:3:3 series at the time of the interview (Table 3). Most commonly, children were missing the fourth dose of DTaP (14%); the single dose of MMR (9%); or the third dose of poliovirus vaccine (8%). Approximately 6% of children received at least one age-invalid vaccination in the 4:3:1:3:3 series, and 3% received at least one interval-invalid vaccination (including 0.3% who received MMR too soon after varicella); 8% received one or the other. Approximately 14% of children had a third dose of hepatitis B vaccine prior to the minimum valid age of 6 months; 11% went on to receive a fourth dose, while the remaining 3% did not. The incremental effect of excluding invalid doses, Am J Prev Med 2008;xx(xx) 3 Table 2. Vaccination coverage among children aged 19 –35 months using ACIP-compliant and dose-counting coverage measures, 2005 ACIP-compliant coverage 95%CI Dose-counting coverage Vaccine % % 95%CI Diphtheria and tetanus toxoids and acellular pertussis Poliovirus Measles, mumps, and rubella Haemophilus influenzae type b Hepatitis B Varicella 4:3:1:3:3a 4:3:1:3:3:1b 84.1 (83.1, 85.1) 85.7 (84.8, 86.6) 91.6 (90.9, 92.4) 91.7 (90.9, 92.4) 90.6 (89.8, 91.3) 91.5 (90.8, 92.2) 88.6 (87.8, 89.3) 93.9 (93.2, 94.4) 89.4 89.1 72.3 69.4 (88.6, (88.3, (71.1, (68.2, 90.1) 89.9) 73.5) 70.6) 92.9 87.9 80.8 76.1 (92.3, (87.0, (79.7, (75.0, 93.5) 88.7) 81.8) 77.2) a For dose-counting coverage, at least four doses of diphtheria and tetanus toxoids and acellular pertussis vaccine; three doses of poliovirus vaccine; one dose of measles, mumps, and rubella vaccine; three doses of Haemophilus influenzae type b (Hib) vaccine; and three doses of hepatitis B vaccine. For ACIP-compliant coverage, one to four doses of Hib may be needed, depending on age at first dose and age at last dose. b 4:3:1:3:3 plus one dose of varicella vaccine ACIP, Advisory Committee on Immunization Practices after accounting for missing doses, was a reduction in 4:3:1:3:3 coverage of 6 percentage points; estimates for hepatitis B and DTaP vaccines were most affected (4% and 2%, respectively). The adjustment for children who received three doses of Hib but should have received four doses reduced the coverage estimate by 6 percentage points, while allowing children who started the vaccine series late to receive fewer doses increased the estimate by 2 percentage points (Table 4). For varicella vaccine, allowing a 4-day grace period before age 12 months increased the coverage estimate by 1 percentage point, as did adjusting for a history of varicella disease. Differential Effects Among States The percentage of children who were missing one or more vaccine doses in the 4:3:1:3:3 series ranged from ⬎30% in two states to ⱕ20% in 30 states, while the percentage of children who were not ACIP compliant ranged from ⬎30% in 13 states to ⱕ20% in two states (Figure 1; Appendix, located online at www.ajpm-online. net). After accounting for missed doses, the incremental decrease in vaccination coverage because of other requirements ranged from 2% in Arkansas to 15% in Colorado (median 9%; Appendix). Discussion More than one in four U.S. children aged 19 –35 months were not in compliance with official vaccination recommendations, half again as many as reflected 4 by the dose-counting method. Missed doses accounted for approximately two thirds of non-adherence to recommendations; however, focusing only on missed doses disregards a substantial number of children who were not in compliance because of mis-timed doses and other vaccine-specific requirements. The ACIP recommendations for early-childhood vaccination are based on the number of doses and the ages that have been shown to be safe and effective in controlled clinical trials. The public health implications of departure from the recommendations are not known in all cases, and may vary by antigen and by dose. For live vaccines such as MMR, a single dose is usually sufficient to produce an immune response. By contrast, inactivated vaccines require multiple doses; the first dose primes the immune system, while the protectiveimmunity response develops after the second or third dose. For example, as many as 20% of children who receive two doses of poliovirus vaccine do not seroconvert, while virtually all recipients become seropositive after three doses.20 Thus, children who receive fewer than the recommended number of doses may not be protected against the disease. Similarly, mis-timed vaccinations can lead to the reduced immunity of individuals and increased levels of population susceptibility. For example, maternally derived measles antibodies persist in many infants until age 11 months and may prevent the development of immune response to vaccination, increasing the risk of primary vaccine failure and subsequent infection in children who receive the measles vaccine before age 12 months.21–23 Similarly, young infants respond infrequently and with low antibody levels to Hib24 and the pertussis portion of DTaP,25 while this immune response improves with age. Allowing sufficient time intervals between doses in multidose vaccines is also necessary to produce adequate immune response.2 For example, studies have shown that an interval of 2 months between doses of poliovirus vaccine is preferred to a 1-month interval.20 Other vaccine-specific exceptions and adjustments are important as well. The two schedules for Hib vaccine reflect the variability in formulation between manufacturers.8 Adjustments for age at first dose of Hib reflect the improved immune response and decreased risk of disease among older children.8 Adjusting for a history of varicella disease accounts for immunity conferred by the disease; however, as varicella disease becomes less common,26 the importance of this adjustment is diminishing.27 Failure to account for invalid vaccinations and other components of the ACIP recommendations has long been acknowledged as a limitation of the dose-counting method of assessing vaccination coverage.4 However, dose-counting is a practical option as a primary surveillance measure because calculating it is substantially simpler than calculating compliance with all recom- American Journal of Preventive Medicine, Volume xx, Number xx www.ajpm-online.net Table 3. Percentage of children with missed and invalid doses, 2005 Did not receive dose Dose % (CI) Diphtheria and tetanus toxoids and acellular pertussis 1 1.2 (1.0,1.5) 2 2.2 (1.9,2.6) 3 3.9 (3.5,4.5) 4 14.3 (13.4,15.2) Series Poliovirus 1 1.8 (1.6,2.2) 2 3.1 (2.7,3.5) 3 8.3 (7.6,9.1) Series Measles, mumps, and rubella 1 8.5 (7.8,9.2) Haemophilus influenzae type b 1 1.4 (1.2,1.7) 2 2.7 (2.4,3.1) (3) NA (NA) 4 6.1 (5.6,6.8) Series Hepatitis B 1 1.4 (1.1,1.7) 2 2.8 (2.5,3.2) 3 7.1 (6.5,7.7) Series Varicella 1 10.6 (9.9,11.4) 4:3:1:3:3b Series 19.2 (18.2,20.3) Received invalid dose % (CI) Incremental effect on coveragea % (CI) 0.3 (0.2, 0.2 (0.1, 0.1 (0.0, 2.2 (0.2, 2.5 (2.0, 0.5) 0.4) 0.2) 2.6) 3.0) ⫺1.6 (1.3, 2.1) 0.4 (0.3, 0.3 (0.2, 0.1 (0.1, 0.6 (0.4, 0.7) 0.5) 0.2) 0.8) ⫺0.1 (0.0, 0.2) 1.0 (0.8, 1.3) ⫺0.9 (0.7, 1.2) 0.5 (0.4, 0.3 (0.2, 0.1 (0.0, 1.1 (0.9, 1.5 (1.2, 0.7) 0.4) 0.2) 1.4) 1.7) ⫺0.1 (0.0, 0.2) NA (NA) 0.2 (0.1, 0.4) 3.4 (3.0, 3.9) 3.6 (3.1, 4.1) ⫺3.6 (3.1, 4.1) 1.4 (1.1, 1.7) ⫺1.2 (1.0, 1.5) 7.9c (7.2, 8.7) ⫺6.2 (5.6, 6.9) a Incremental effect on coverage after accounting for doses not received For standard coverage, at least four doses of diphtheria and tetanus toxoids and acellular pertussis vaccine; three doses of poliovirus vaccine; one dose of measles, mumps, and rubella vaccine; three doses of Haemophilus influenzae type b (Hib) vaccine; and three doses of hepatitis B vaccine. For ACIP-compliant coverage, one to four doses of Hib may be needed, depending on age at first dose and age at last dose. c 6.2% of children received an age-invalid dose; 2.5% received an interval-invalid dose (including 0.3% who received MMR too soon after varicella). ACIP, Advisory Committee on Immunization Practices; NA, not applicable. b mendations, historical continuity allows evaluation of trends over time, and missed doses account for the majority of noncompliance. Previous studies have examined specific components of the ACIP recommendations, and additional meaTable 4. Percentage of children affected by other ACIP recommendations, 2005 Vaccine % (CI) Incremental effect on coverage % (CI) Haemophilus influenzae type b Received 3 but needed 6.3 (5.7, 6.9) ⫺6.3 (5.7, 6.9) 4 doses Needed fewer doses 15.2 (14.3, 16.2) ⫹1.8 (1.5, 2.2) due to late first dose Varicella Vaccination not 2.6 (2.2, 3.1) ⫹0.9 (0.7, 1.1) needed due to history of varicella disease ACIP, Advisory Committee on Immunization Practices Month 2008 sures of vaccination coverage have been proposed.28 –32 The WHO recommends routinely reporting coverage estimates that exclude invalid doses, in addition to those that include all doses33; in the U.S., several local studies34 –36 and one national study37 have evaluated coverage that excludes invalid doses. Recently, work has been done to examine the timeliness of vaccinations and to determine whether children in the U.S. are adequately protected from vaccine-preventable diseases during their first 2 years of life.38 – 44 Another study compared varicella coverage among all children to that among susceptible children, excluding those with a history of varicella disease.27 However, to the authors’ knowledge, the current study is the first nationally representative assessment of compliance with the complete set of ACIP recommendations. The coverage estimates for Hib and hepatitis B vaccines were most affected by the study’s adjustments. The variability in recommendations for different formulations of Hib vaccine poses an analytic challenge, because manufacturer information is not collected in Am J Prev Med 2008;xx(xx) 5 Missing One or More Doses >30% (2 states) 21%–30% (19 states) 11%–20% (29 states) 0%–10% (1 state) 11%–20% (2 states) 0%–10% (0 states) Not ACIP-Compliant >30% (13 states) 21%–30% (36 states) Figure 1. Estimated percentage of children not in compliance with ACIP recommendations* by state, 2005 *For dose-counting, at least four doses of diphtheria and tetanus toxoids and acellular pertussis vaccine; three doses of poliovirus vaccine; one dose of measles, mumps, and rubella vaccine; three doses of Haemophilus influenzae type b (Hib) vaccine; and three doses of hepatitis B vaccine. For ACIP compliance, one to four doses of Hib may be needed, depending on age at first dose and age at last dose. the NIS. The standard dose-counting measure of coverage requires receipt of only three doses, differentially inflating coverage among children who are following 6 the four-dose schedule; this analysis has shown that the resulting inflation in coverage is substantial, even compared to the lenient definition used here. Second, the American Journal of Preventive Medicine, Volume xx, Number xx www.ajpm-online.net exclusion of invalid doses had a relatively large effect on hepatitis B vaccine coverage. Some early third doses of hepatitis B vaccine were likely due to the use of PEDIARIXTM vaccine. While the administration of an extra dose of hepatitis B vaccine via PEDIARIXTM at 4 months is acceptable, counting it as one of the three required doses in the hepatitis B series is not consistent with ACIP recommendations.9 This study is subject to at least two limitations: First, the NIS relies on identification of vaccination providers by the household respondents and on the complete and accurate reporting of vaccination histories by these providers; inaccurately or incompletely reported dates may have caused differential underestimation of ACIPcompliant coverage compared to dose-counting coverage. Second, the NIS does not collect information on medical contraindications to vaccination, such as severe allergic reaction after a prior dose of vaccine. However, true contraindications to vaccination are few and most are temporary,2 and would be equally reflected in ACIP-compliant and dose-counting coverage measures. The findings suggest that the number of children who needed additional doses to bring them into compliance with ACIP recommendations at the time of the interview was nearly 50% greater than dose-counting alone would imply, and up to double that number in some states. While priority should be given to ensuring that children receive all of the recommended vaccine doses, compliance with the remaining recommendations should not be overlooked. In practice, vaccination providers must sometimes balance the competing goals of administering valid vaccinations and reducing missed opportunities. Administering a vaccination a few days early is preferred to missing the opportunity to vaccinate a child who is unlikely to return at the appropriate time; however, the observance of both age-appropriate vaccination and sufficient time between doses in a series maximizes the immune response and the vaccine’s efficacy.45 In general, providers and parents are advised to adhere to the specifics of published guidelines whenever possible. Compliance with all ACIP recommendations is an essential measure of the quality of healthcare service delivery. Many state immunization programs use the assessment, feedback, incentives, and exchange of information (AFIX) strategy to assist vaccination providers with improving the quality of service delivery.46,47 State health officials review a representative sample of office-based charts to measure vaccination coverage and to identify missed opportunities for vaccination. The Comprehensive Clinic Assessment Software Application (CoCASA), which was designed to assist with implementation of the AFIX strategy (www.cdc.gov/ nip/cocasa/default.htm), now evaluates compliance with each component of the ACIP recommendations in addition to dose-counting coverage. State immunization information systems may also be useful for evaluMonth 2008 ating compliance with ACIP recommendations. A comprehensive approach to vaccination-coverage assessment that encompasses several measures, including both a dose-counting measure and a measure of compliance with all ACIP recommendations, may give immunization providers a more accurate picture of the quality of care within their practices and provide a more complete evaluation of vaccination programs, revealing their specific strengths and weaknesses and eliciting data-driven solutions to ensure that children and communities are optimally protected from vaccine-preventable diseases. Dr Luman had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. This research and the National Immunization Survey were conducted through funding by the CDC, USDHHS. This study, including design and conduct, data collection, analysis and interpretation of the data, and manuscript preparation, review, and approval, was conducted under the auspices of the USDHHS and the CDC. CDC gave final approval of the manuscript. The authors gratefully acknowledge the contribution of Lawrence Barker, Susan Chu, Diane Simpson, Jeanne Santoli, Abigail Shefer, and Jim Singleton for their review and suggestions. The findings and conclusions in this report are those of the author(s) and do not necessarily represent the views of the CDC. No financial disclosures were reported by the authors of this paper. References 1. CDC. Recommended childhood and adolescent immunization schedule— U.S., 2006. MMWR Morb Mortal Wkly Rep 2006;54:Q1–Q4. 2. CDC. General recommendations on immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP) and the American Academy of Family Physicians (AAFP). MMWR Morb Mortal Wkly Rep 2006;55:1– 48. 3. CDC. National, state, and urban area vaccination coverage among children aged 19 –35 months: U.S., 2005. MMWR Morb Mortal Wkly Rep 2006;55:988 –93. 4. Simpson DM, Essati-Rice TM, Zell ER. Forty years and four surveys: How does our measuring measure up? Am J Prev Med 2001;20(4S):6 –14. 5. Smith PJ, Battaglia MP, Huggins VJ, et al. Overview of the sampling design and statistical methods used in the National Immunization Survey. Am J Prev Med 2001;20(4S):17–24. 6. Zell E, Ezzati-Rice TM, Battaglia M, Wright R. National Immunization Survey: the methodology of a vaccination surveillance system. Public Health Rep 2000;115:65–77. 7. CDC. Recommended childhood and adolescent immunization schedule— U.S., 2005. MMWR Morb Mortal Wkly Rep 2005;53:Q1–Q3. 8. CDC. Haemophilus b conjugate vaccines for prevention of Haemophilus influenzae type b disease among infants and children two months of age and older: recommendations of the ACIP. MMWR Morb Mortal Wkly Rep 1991;40:1–7. 9. CDC. Notice to readers: FDA licensure of diphtheria and tetanus toxoids and acellular pertussis absorbed, hepatitis B (recombinant), and poliovirus vaccine combined (PEDIARIXTM) for use in infants. MMWR Morb Mortal Wkly Rep 2003;52:203– 4. 10. CDC. Combination vaccines for childhood immunization: recommendations of the Advisory Committee on Immunization Practices (ACIP), the Am J Prev Med 2008;xx(xx) 7 11. 12. 13. 14. 15. 16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28. 8 American Academy of Pediatrics (AAP), and the American Academy of Family Physicians (AAFP). MMWR Morb Mortal Wkly Rep 1999;48:1–15. CDC. Prevention of varicella: recommendations of the Advisory Committee on Immunization Practices (ACIP). MMWR Morb Mortal Wkly Rep 1996;45:1–27. CDC. Prevention of pneumococcal disease among infants and young children: recommendations of the ACIP. MMWR Morb Mortal Wkly Rep 2000;49:1–35. CDC. Decreased availability of pneumococcal conjugate vaccine. MMWR Morb Mortal Wkly Rep 2001;50:783– 4. CDC. Updated recommendations on the use of pneumococcal conjugate vaccine in a setting of vaccine shortage—Advisory Committee on Immunization Practices. MMWR Morb Mortal Wkly Rep 2001;50:1140 –2. CDC. Notice to readers: pneumococcal conjugate vaccine shortage resolved. MMWR Morb Mortal Wkly Rep 2003;52:446 –7. CDC. Notice to readers: limited supply of pneumococcal conjugate vaccine. MMWR Morb Mortal Wkly Rep 2003;52:1234. CDC. Updated recommendations on the use of pneumococcal conjugate vaccine: suspension of recommendation for third and fourth dose. MMWR Morb Mortal Wkly Rep 2004;53:177– 8. CDC. Notice to readers: pneumococcal conjugate vaccine shortage resolved. MMWR Morb Mortal Wkly Rep 2004;53:851–2. CDC. Updated recommendations for use of pneumococcal conjugate vaccine: reinstatement of the third dose. MMWR Morb Mortal Wkly Rep 2004;53:589 –90. Vidor E, Meschievitz C, Plotkin S. Fifteen years of experience with Vero-produced enhanced potency inactivated poliovirus vaccine. Pediatr Infect Dis J 1997;16:312–22. Albrecht P, Ennis FA, Saltzman EJ, Krugman S. Persistence of maternal antibody in infants beyond 12 months: mechanism of measles vaccine failure. J Pediatr 1977;91:715– 8. Wilkins J, Wehrle PF. Additional evidence against measles vaccine administration to infants less than 12 months of age: altered immune response following active/passive immunization. J Pediatr 1979;94:865–9. Orenstein WA, Markowitz L, Preblud SR, Hinman AR, Tomasi A, Bart KJ. Appropriate age for measles vaccination in the U.S. Dev Biol Stand 1986;65:13–21. Smith DH, Peter G, Ingram DL. Responses of children immunized with the capsular polysaccharide of H. influenzae type B. Pediatrics 1973;52:637– 44. Di Sant’Agnese PA. Combined immunization against diphtheria, tetanus, and pertussis in newborn infants I. Production of antibodies in early infancy. Pediatrics 1949;3:20 –33. Seward JF, Watson BM, Peterson CL, et al. Varicella disease after introduction of varicella vaccine in the U.S., 1995–2000. JAMA 2002;287:606 –11. Luman ET, Ching PLYH, Jumaan AO, Seward JF. Uptake of varicella vaccination among young children in the U.S.—a success story in eliminating racial and ethnic disparities. Pediatrics 2006;117:999 –1008. Rodewald L, Maes E, Stevenson J, et al. Immunization performance measurement in a changing immunization environment. Pediatrics 1999;103:889 –97. 29. Bolton P, Hussain A, Hadpawat A, Holt E, Hughart N, Guyer B. Deficiencies in current childhood immunization indicators. Public Health Rep 1998;113:527–32. 30. Glauber JH. The immunization delivery effectiveness assessment score: a better immunization measure? Pediatrics 2003;112:e39 – 45. 31. Fairbrother G, Freed GL, Thompson JW. Measuring immunization coverage. Am J Prev Med 2000;19(3S):78 – 88. 32. Lieu TA, Massoudi MR, Miroshnik IL, O’Brien MA, Coltin KL, Rodewald LE. Immunization status among children newly enrolled in a health plan: a new frontier for quality measurement? Am J Manag Care 2003;9:121–7. 33. WHO. Immunization coverage cluster survey—reference manual. Geneva, Switzerland: WHO, 2005. http://www.who.int/vaccines-documents/DocsPDF05/ www767.pdf. 34. Vivier PM, Alario AJ, Simon PR, et al. Immunization status of children enrolled in a hospital-based Medicaid managed care practice: the importance of the timing of vaccine administration. Pediatr Infect Dis J 1999;18:783– 8. 35. Butte AJ, Shaw JS, Bernstein D. Strict interpretation of vaccination guidelines with computerized algorithms and improper timing of administered doses. Pediatr Infect Dis J 2001;20:561–5. 36. Hamlin JS, Wood D, Pereyra M, Grabowsky M. Inappropriately timed immunizations: types, causes, and their relationship to record keeping. Am J Public Health 1996;86:1812– 4. 37. Stokley S, Maurice E, Smith PJ, Klevens RM. Evaluation of invalid vaccine doses. Am J Prev Med 2004:26;34 – 40. 38. Luman ET, McCauley MM, Stokley S, Chu SY, Pickering LK. Timeliness of childhood immunizations. Pediatrics 2002;110:935–9. 39. Luman ET, Barker LE, McCauley MM, Drews-Botsch C. Timeliness of childhood immunizations—a state-specific analysis. Am J Public Health 2005;95:1367–74. 40. Luman ET, Barker LE, Shaw KM, McCauley MM, Buehler JW, Pickering LK. Timeliness of childhood vaccinations in the U.S.— days undervaccinated and number of vaccines delayed. JAMA 2005;293:1204 –11. 41. Dominguez SR, Parrott S, Lauderdale DS, Daum RS. On-time immunization rates among children who enter Chicago public schools. Pediatrics 2004;114:e741–7. 42. Cotter JJ, Bramble JD, Bovbjerg VE, et al. Timeliness of immunizations of children in a Medicaid primary care case management managed care program. J Natl Med Assoc 2002;94:833– 40. 43. Dombkowski KJ, Lantz PM, Freed GL. The need for surveillance of delay in age-appropriate immunizations. Am J Prev Med 2002;23:36 – 42. 44. Dombkowski KJ, Lantz PM, Freed GL. Risk factors for delay in ageappropriate vaccination. Public Health Rep 2004;119:144 –55. 45. Watson JC, Peter G. General immunization practices. In: Plotkin SA, Orenstein WA, Eds. Vaccines. 3rd ed. Philadelphia: W.B. Saunders Company, 1999. 46. Briss PA, Rodewals LE, Hinman AR, et al. Reviews of evidence regarding interventions to improve vaccination coverage in children, adolescents, and adults. Am J Prev Med 2000;18(1S):97–140. 47. Shefer A, Santoli J, Wortley P, et al. Status of quality improvement activities to improve immunization practices and delivery. findings from the Immunization Quality Improvement Symposium, October 2003. J Public Health Manag Pract 2006;12:77– 89. American Journal of Preventive Medicine, Volume xx, Number xx www.ajpm-online.net Appendix Percentage of children not in compliance with ACIP vaccination recommendations by state, 2005 Diphtheria and tetanus toxoids and acellular pertussis 4:3:1:3:3a Am J Prev Med 2008;xx(xx) Not ACIPcompliantb Missed doses National 27.7 19.2 Alabama Alaska Arizona Arkansas California Colorado Connecticut Delaware District of Columbia Florida Georgia Hawaii Idaho Illinois Indiana Iowa Kansas Kentucky Louisiana Maine Maryland Massachusetts Michigan Minnesota Mississippi Missouri Montana Nebraska Nevada New Hampshire New Jersey New Mexico New York North Carolina North Dakota Ohio Oklahoma Oregon 26.8 28.2 30.4 34.6 31.6 31.2 23.1 25.1 36.5 Measles, mumps, and rubella Haemophilus influenzae type b Noncompliance due to Noncompliance due to Noncompliance due to Noncompliance due to Noncompliance due to State Poliovirus Not ACIPcompliant Not ACIPcompliant Missed doses Otherc 7.1 3.8 10.9 12.1 ⫺1.2 9.0 14.6 13.4 13.4 11.6 12.7 12.7 15.6 19.5 5.6 11.9 9.0 12.1 7.4 4.6 8.8 8.8 14.0 3.4 2.7 4.4 1.3 4.2 8.1 3.9 6.8 5.5 5.5 15.6 15.9 13.1 10.3 11.0 7.6 9.8 9.5 6.5 18.8 16.4 14.1 10.5 12.8 9.0 10.5 9.4 ⫺1.0 ⫺3.2 ⫺0.5 ⫺1.0 ⫺0.2 ⫺1.8 ⫺1.4 ⫺0.7 0.1 5.8 2.5 4.4 9.3 7.7 8.1 3.1 0.8 7.6 6.1 8.2 4.7 1.6 6.7 4.6 0.9 6.6 6.0 8.8 10.0 5.4 12.3 8.8 13.7 16.0 9.3 8.3 8.1 16.6 7.9 12.5 13.3 15.0 3.6 7.3 7.3 13.4 10.3 10.8 9.4 19.1 12.2 8.7 6.6 9.6 8.1 5.0 4.8 5.5 9.5 7.0 7.0 10.0 7.4 2.1 4.5 5.3 7.1 5.9 9.0 5.0 16.8 6.7 3.6 2.2 4.1 7.9 4.3 3.5 2.6 7.1 0.9 5.5 3.3 7.6 1.5 2.8 2.0 6.3 4.4 1.8 4.4 2.3 5.5 9.4 5.5 10.4 20.4 9.8 14.7 12.9 15.9 14.0 9.9 15.0 9.2 4.6 5.5 11.8 9.9 10.2 19.5 9.7 14.8 15.1 8.9 8.1 10.6 22.6 13.7 17.2 16.6 18.5 16.7 11.0 15.8 9.3 4.6 6.6 13.3 11.6 12.1 24.5 10.1 15.6 17.1 0.5 ⫺2.6 ⫺0.2 ⫺2.2 ⫺3.9 ⫺2.5 ⫺3.7 ⫺2.6 ⫺2.7 ⫺1.1 ⫺0.8 ⫺0.1 0.0 ⫺1.1 ⫺1.5 ⫺1.7 ⫺1.9 ⫺5.0 ⫺0.4 ⫺0.8 ⫺2.0 7.9 10.2 6.3 2.4 7.3 5.6 3.3 3.8 11.7 18.9 10.1 10.4 7.7 9.6 6.3 8.1 4.0 9.3 3.8 2.3 16.5 13.3 11.8 7.7 16.2 13.7 12.4 8.7 0.3 ⫺0.4 ⫺0.6 ⫺1.0 3.8 5.7 10.7 12.7 6.0 3.8 2.2 6.8 9.9 8.2 14.8 19.8 3.7 4.5 11.3 15.7 6.2 3.7 3.5 4.1 11.3 12.7 14.5 20.2 12.8 13.7 14.2 23.8 ⫺1.5 ⫺1.0 0.3 ⫺3.6 8.5 0.9 11.4 6.1 5.3 10.6 6.5 10.5 11.6 11.5 9.1 7.9 6.0 5.8 12.9 5.9 9.2 11.2 11.0 8.4 6.8 4.8 4.8 8.4 0.6 1.3 0.4 0.5 0.7 1.1 1.2 1.0 4.5 9.9 9.2 13.7 13.2 10.1 10.9 8.8 8.0 10.6 3.1 11.1 7.6 12.1 6.3 3.8 2.9 6.8 8.8 6.8 ⫺1.9 6.1 1.1 3.8 7.1 5.9 1.2 1.8 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.3 0.0 0.0 0.0 0.2 0.0 0.0 0.0 0.4 0.0 0.0 0.0 9.8 7.2 10.5 13.4 7.0 9.0 8.3 8.4 10.7 11.3 7.8 9.3 3.0 7.0 10.0 10.8 10.0 7.9 6.2 15.4 12.4 8.0 7.1 10.3 13.5 7.0 8.8 8.3 7.3 9.8 10.8 7.8 6.4 3.0 6.3 8.6 9.0 9.7 6.4 5.2 14.3 11.7 1.8 0.1 0.2 0.0 0.0 0.2 0.0 1.1 0.9 0.5 0.0 2.9 0.0 0.7 1.4 1.8 0.3 1.5 1.0 1.1 0.7 12.1 8.0 13.0 16.6 12.3 14.1 9.7 7.8 12.8 11.8 11.8 8.9 2.2 11.4 11.4 11.5 12.1 14.5 12.1 20.6 9.1 6.3 5.5 8.6 7.3 4.6 6.0 6.6 7.0 5.2 5.7 3.6 4.2 0.6 4.7 6.8 10.6 5.5 8.5 3.3 10.6 3.7 13.6 8.4 8.0 5.5 0.8 0.0 0.0 0.0 17.4 10.4 9.4 6.4 14.7 8.8 7.1 5.6 2.7 1.6 2.3 0.8 15.2 15.8 9.6 6.2 3.4 6.8 9.1 12.7 0.0 0.0 0.0 0.0 8.4 7.5 11.2 17.6 7.7 6.9 10.2 17.3 0.7 0.6 1.0 0.3 9.8 9.5 12.9 19.5 Otherc 8.5 15.9 14.3 1.6 8.4 8.3 0.1 9.4 16.7 24.6 20.8 32.2 22.1 16.6 13.9 15.8 26.5 10.1 3.6 9.6 2.4 9.5 14.6 9.2 9.3 10.0 13.1 16.2 18.0 26.7 18.2 15.9 10.4 13.0 17.1 10.4 15.4 16.9 26.2 16.5 12.3 10.4 11.7 14.7 2.7 0.8 1.1 0.5 1.7 3.6 0.0 1.3 2.4 8.4 10.9 11.1 15.4 9.5 5.8 5.4 7.9 9.0 8.4 10.9 10.8 15.4 9.5 5.8 5.4 7.9 9.0 0.0 0.0 0.3 0.0 0.0 0.0 0.0 0.0 0.0 29.6 22.2 28.9 35.5 25.8 28.1 21.9 25.9 26.8 32.4 27.6 28.6 11.0 24.2 23.5 24.9 27.7 29.3 22.5 40.3 25.4 20.7 15.3 19.9 21.9 16.5 21.9 15.1 16.2 20.3 24.0 16.7 17.7 6.5 17.3 14.8 16.4 20.7 20.4 10.9 33.3 17.2 8.9 6.9 9.0 13.6 9.3 6.2 6.8 9.7 6.5 8.4 10.9 10.9 4.5 6.9 8.7 8.5 7.0 8.9 11.6 7.0 8.2 19.4 12.7 16.5 17.7 15.5 17.5 13.3 12.1 15.5 20.8 10.8 11.6 3.8 14.2 11.9 15.2 16.2 16.4 8.9 27.7 12.8 17.4 11.9 15.5 16.9 11.3 17.5 12.8 11.5 14.3 19.5 8.6 10.2 3.0 13.0 10.7 13.8 14.7 14.4 8.3 26.5 12.2 2.0 0.8 1.0 0.8 4.2 0.0 0.5 0.6 1.2 1.3 2.2 1.4 0.8 1.2 1.2 1.4 1.5 2.0 0.6 1.2 0.6 12.1 7.0 10.8 9.5 8.4 10.2 7.7 8.6 5.1 8.6 2.9 7.9 1.6 6.9 3.7 5.2 5.6 5.1 4.3 13.7 4.9 12.1 7.0 10.8 9.5 8.4 10.2 7.7 8.6 5.1 8.3 2.9 7.9 1.6 6.7 3.7 5.2 5.6 4.7 4.3 13.7 4.9 35.0 28.0 25.6 18.8 21.8 21.6 18.4 14.8 13.2 6.4 7.2 4.0 17.8 18.3 12.4 9.3 15.6 17.8 11.0 8.7 2.2 0.5 1.4 0.6 14.4 8.4 8.0 5.5 24.6 25.0 35.0 36.4 15.0 15.9 24.3 27.1 9.6 9.1 10.7 9.3 13.5 13.6 23.4 22.9 12.2 12.1 20.3 21.3 1.3 1.5 3.1 1.6 3.4 6.8 9.1 12.7 Missed doses Otherc Missed doses Noncompliance due to Otherc Not ACIPcompliant Not ACIPcompliant Missed doses Varicella Noncompliance due to Otherc Not ACIPcompliant Not ACIPcompliant Otherc Hepatitis B Missed doses Otherc Missed doses 8.e1 8.e2 American Journal of Preventive Medicine, Volume xx, Number xx Appendix (continued) Diphtheria and tetanus toxoids and acellular pertussis 4:3:1:3:3a Measles, mumps, and rubella Haemophilus influenzae type b Noncompliance due to Noncompliance due to Noncompliance due to Noncompliance due to Noncompliance due to State Poliovirus Not ACIPcompliantb Missed doses Otherc Not ACIPcompliant Missed doses Otherc Not ACIPcompliant Missed doses Otherc Not ACIPcompliant Missed doses Otherc Not ACIPcompliant Missed doses Otherc Hepatitis B Varicella Noncompliance due to Noncompliance due to Not ACIPcompliant Missed doses Otherc Not ACIPcompliant Missed doses Otherc Pennsylvania Rhode Island South Carolina South Dakota Tennessee Texas Utah Vermont Virginia Washington West Virginia Wisconsin Wyoming 22.4 28.1 28.1 16.8 16.9 21.5 5.6 11.2 6.6 12.2 13.8 18.5 12.2 12.1 17.4 0.0 1.7 1.1 9.2 4.8 10.7 9.2 4.4 10.7 0.0 0.4 0.0 6.7 6.2 13.2 6.4 4.8 12.7 0.3 1.4 0.5 8.6 7.5 16.0 6.3 5.8 9.8 2.3 1.7 6.2 7.6 10.1 8.3 4.3 3.7 5.9 3.3 6.4 2.4 13.4 4.0 13.0 10.8 3.8 12.6 2.6 0.2 0.4 23.8 28.2 29.3 31.1 30.4 24.7 30.9 34.6 25.1 30.4 13.1 17.1 21.6 25.9 18.5 14.2 22.2 25.1 17.8 21.4 10.7 11.1 7.7 5.2 11.9 10.5 8.7 9.5 7.3 9.0 10.9 15.6 18.7 19.7 15.3 15.8 18.3 19.3 13.7 19.1 10.3 14.3 16.3 19.1 13.8 13.3 14.9 17.4 13.4 16.9 0.6 1.3 2.4 0.6 1.5 2.5 3.4 1.9 0.3 2.2 5.9 5.5 7.3 13.6 3.9 7.0 12.1 7.6 5.1 7.9 5.9 5.4 7.3 13.6 3.9 7.0 11.6 7.6 5.1 7.9 0.0 0.1 0.0 0.0 0.0 0.0 0.5 0.0 0.0 0.0 6.2 8.0 12.1 11.9 7.8 6.7 10.0 14.3 9.3 12.5 5.6 8.0 10.9 11.0 7.4 6.7 9.4 14.3 8.4 10.6 0.6 0.0 1.2 0.9 0.4 0.0 0.6 0.0 0.9 1.9 9.6 13.5 15.0 12.9 10.1 14.7 11.4 13.3 9.7 10.7 5.9 6.5 7.2 9.0 2.5 2.8 6.4 4.7 7.7 5.3 3.7 7.0 7.8 3.9 7.6 11.9 5.0 8.6 2.0 5.4 14.9 12.1 11.7 11.0 13.4 6.2 14.7 15.7 9.9 16.6 4.7 5.3 8.5 8.7 6.0 2.7 12.6 8.2 7.4 9.6 10.2 6.8 3.2 2.3 7.4 3.5 2.1 7.5 2.5 7.0 12.8 9.2 10.5 15.2 27.9 7.5 19.9 18.5 9.3 19.6 14.3 10.2 11.1 18.8 31.5 10.1 23.4 18.9 13.0 22.8 ⫺1.5 ⫺1.0 ⫺0.6 ⫺3.6 ⫺3.6 ⫺2.6 ⫺3.5 ⫺0.4 ⫺3.7 ⫺3.2 Minimum Maximum Median Range 11.0 40.3 28.1 29.3 6.5 33.3 18.4 26.8 2.4 14.6 9.0 12.2 3.8 27.7 15.6 23.9 3.0 26.5 13.8 23.5 0.0 4.2 1.3 4.2 1.6 15.4 7.9 13.8 1.6 15.4 7.9 13.8 0.0 0.8 0.0 0.8 3.0 17.6 9.3 14.6 3.0 17.3 8.4 14.3 0.0 4.5 0.7 4.5 2.2 20.6 11.4 18.4 0.6 12.7 6.3 12.1 ⫺1.9 11.9 5.6 13.8 3.6 19.8 12.1 16.2 2.1 16.8 7.4 14.7 0.9 10.2 3.9 9.3 4.0 27.9 11.8 23.9 3.8 31.5 13.0 27.7 ⫺5.0 2.6 ⫺1.1 7.6 a For dose-counting coverage, at least four doses of diphtheria and tetanus toxoids and acellular pertussis vaccine; three doses of poliovirus vaccine; one dose of measles, mumps, and rubella vaccine; three doses of Haemophilus influenzae type b vaccine; and three doses of hepatitis B vaccine. For ACIP-compliant coverage, one to four doses of Haemophilus influenzae type b vaccine may be needed depending on age at first dose and age at last dose. b Half-width of the 95% CI for 4:3:1:3:3 non-compliance: 4%–9% c Incremental decrease in compliance due to other ACIP recommendations after accounting for doses not received. Negative values indicate compliance increased due to other recommendations. ACIP, Advisory Committee on Immunization Practices; DTaP, diphtheria and tetanus toxoids and acellular pertussis vaccine; Hib, Haemophilus influenzae type b vaccine; MMR, measles, mumps, and rubella vaccine.
© Copyright 2026 Paperzz