COAGULATION AND TRANSFUSION MEDICINE Original Article The Influence of the Reference Mean Prothrombin Time on the International Normalized Ratio GREGORY C. CRITCHFIELD, MD, M S , 1 2 3 AND STERLING T. BENNETT, MD, M S 1 ' 4 5 The International Normalized Ratio (INR) is a mathematical transformation of the prothrombin time (PT). The transformation requires a laboratory to compute the geometric mean of its own reference population. In this paper, the authors examine how the reference mean PT influences the INR accuracy and precision using a validated probabilistic model. The variance of the geometric mean of reference populations in three laboratory settings was determined. Because the variance of an individual laboratory geometric mean is not directly determinable by simple parametric equations, its variance is estimated using bootstrap analysis. The geometric mean is compared to the computationally simpler arithmetic mean for effects on accuracy and precision of the resulting INR. The study shows mathematically and empirically that using the arithmetic mean biases INR determinations so that patients tend to be over-anticoagulated. However, in the laboratory settings examined, the amount of bias was both statistically and clinically insignif- cant. An analysis of the effect on the INR of errors in estimating the geometric mean reference PT also is performed. For large biases in estimating the reference mean, the INR can be significantly affected and can trigger inappropriate clinical actions in patients. The authors demonstrate empirically and mathematically that biases in the geometric mean reference PT do not affect the INR coefficient of variation. However, they produce significant differences in confidence intervals for INR determinations. Laboratories must exercise care in determining specific reference means to ensure that biases do not occur in geometric mean reference PT determinations. This can be achieved by circumspection in the selection of normal subjects for the reference population, carefully reviewing the data, and performing the proper calculations on the data. (Key words: Prothrombin time; International Normalized Ratio; Mathematical model; Analytical precision; Accuracy; Statistical bias; Monte Carlo simulation; Bootstrap analysis) Am J Clin Pathol 1994;102:806-811. More than 500,000 patients in the United States are under continuous oral anticoagulant treatment. Proper treatment requires that a balance be reached between too much anticoagulation,which leads to hemorrhagic complications,and too little, which increases the risk of thrombosis. By reporting prothrombin times (PTs) as International Normalized Ratios (INRs), clinical laboratories can help to decrease the frequency and severity of hemorrhagic complications that arise from anticoagulant treatment, and at the same time, reduce the risk of thrombosis in patients taking warfarin.1"4 To compute the INR requires three elements: the patient's measured PT; the geometric mean PT of the laboratory's reference population; and the sensitivity of the thromboplastin used in the assay.5"6 The INR equation is a simple exponential transformation involving a patient PT result (the numerator) scaled by the mean of the reference population, all raised to a power (the International Sensitivity Index of the thromboplastin) 3 : Supported in part by the National Science Foundation Grant DMS9105134. Manuscript received October 15, 1993; revision accepted February 21, 1994. Address reprint requests to Dr. Critchfield: Department of Pathology, Utah Valley Regional Medical Center, 1034 North 500 West, Box 390. Provo, UT 84603. Laboratory where P is the patient's measured PT; S is the International Sensitivity Index (ISI); M is the (geometric) mean PT of the reference population; and / is the INR. Examples of potential errors in the determination of the geometric mean PT of the reference population include substitution of the arithmetic mean for the geometric mean, computational errors, and errors in sampling the reference population. In this paper, the effects on the accuracy and precision of the INR of errors in the specification of the reference population mean PT are examined, both theoretically and in actual laboraFrom the lntermountain Laboratory Data Project;' IHC Laboratory Services, Salt Lake City. Utah; ^Department of Pathology. Utah Valley tory settings. Regional Medical Center, Provo. Utah; ^Department of Microbiology. Brigham Young University, Provo. Utah; 4 Department of Pathology, LDS Hospital, Salt Lake City, Utah;'Department ofPathology, UniverMATERIALS A N D METHODS sity of Utah School of Medicine, Salt Lake City, Utah. Settings and Reference Range Studies Three laboratories representing different practice settings and patient populations were selected to participate in the study. Characteristics of the laboratories have been reported in previous work7 and are briefly summarized in Table 1. The laboratories differed according to size, location, number of 806 807 CRITCHFIELD AND BENNETT Influence of Reference Mean PT on INR TABLE 1. CHARACTERISTICS OF LABORATORIES Laboratory Type Location Annual PT Determinations 1 Large reference Urban 31,882 2 Regional reference Suburban 14,615 3 Small hospital Rural Analytical 392 Methods Reference Population Means. Each laboratory submitted PTs from its reference population sample. The geometric reference population mean, M, was computed by the formula: ( \l/N N (2) The arithmetic mean, A, was also determined. The absolute and percent differences between the geometric and arithmetic means were determined according to: A= A-M, M M X 100, (4) where A is the arithmetic mean, and M is the geometric mean. Means Precision estimates for the means were obtained by two methods. From basic probability theory and the central limit theorem, 8 the standard deviation (SD) of the arithmetic mean of the reference population was estimated as: Sp 1.93 2.82 from the original data. Bootstrap analysis is a powerful nonparametric statistical technique. It permits computation of any statistic of interest based on the simple assumption that the original sample data themselves are representative of the population that is being studied, (which is an assumption that is fundamental for any statistical study). To estimate the reference geometric mean PT, the original data set of each laboratory was sampled with replacement* repeatedly to yield (generally different) bootstrapped data sets of the same size as the original sample. From each bootstrapped data set, a bootstrapped geometric mean was computed. The SD of the distribution of bootstrapped geometric means was used as an estimate of the SD of the actual geometric mean. Several examples of bootstrap analysis can be found in the biomedical literature to illustrate applications of the technique.""' 3 Effect on the INR of Substituting the Arithmetic for the Geometric Mean PT of the Reference Population The effect of using the arithmetic mean in lieu of the geometric mean was assessed graphically and numerically for the three laboratories. The International Normalized Ratio distributions were computed according to a previously validated statistical model using both the geometric and arithmetic means. 7,14 Effect on the INR of Errors in the Determination Mean PT of the Reference Population Precision of the Reference Population 1.93 (3) and %A Thromboplastin ISI MLA 900C/Organon Teknika Simplastin Automated Reagent, lot #102282 MLA 900/Organon Teknika Simplastin Excel, lot #102531 MLA Electra 750/Sigma Diagnostics Thromboplastin with Calcium, lot #52H-6104 yearly PT determinations, and reagent systems used to perform PT measurements. Each laboratory was asked to perform a reference study to determine the geometric mean, M, of the reference population. In each setting, the laboratories were asked to measure PTs in approximately 20 normal subjects who met the usual inclusion criteria: healthy nonpregnant adults not taking medications that may alter coagulation studies, with no personal or family history of bleeding tendencies or thrombosis. 7 The PT measurements were performed on automated instrumentation according to manufacturer's instructions. Statistical Method/Reagent System (5) where sP is the SD of the PT measurements in the reference population sample, and N is the number of subjects. Next, using bootstrap analysis,9"10 the SD of the geometric mean, M, of the reference population was estimated directly of the A Monte Carlo study was conducted to determine the effects on the INR of errors in the reference population mean. Between-run control data (n = 49) were used to estimate the mean and SD of PT measurements at a mid-level control in a laboratory with excellent PT precision (Laboratory 1). The determination of the geometric mean of the reference population has already been described. Simulated INR measurements were generated from random deviates of the PT distribution according to Equation 1 as previously described.7 The median, mean, SD, and coefficient of variation (CV) of the INR were calculated for the simulated INR measurements. Simulations were performed with the correct reference population mean and with reference population means with hypothetical errors of ±1%, ±2%, ±5% and ±10% (keeping the ISI, the control PT level and precision constant). The simulation software was written in the C programming language for generic UNIX Sys- * Sampling-with-replacement means drawing from a distribution of values, returning the value to the distribution of values, reshuffling or mixing, and redrawing another value. Vol. 102-No. 6 COAGULATION AND TRANSFUSION MEDICINE 808 Original Article TABLE 2. COMPARISON OF GEOMETRIC AND ARITHMETIC MEANS FOR LABORATORIES ON ACTUAL REFERENCE POPULATION DATA Laboratory Number of Subjects Geometric Mean (M)l Standard Deviation* (seconds) 1 2 3 18 20 22 10.61/0.07 12.22/0.14 12.27/0.10 Arithmetic Mean (A)/ Standard Deviation^ (seconds) Absolute Difference A—M (seconds) 'ercent Difference A-M 10.62/0.08 12.24/0.14 12.28/0.10 4.7 X 10"3 1.5 X 10"2 8.6 X 10"3 4.4 X 10"2 1.2 X 10"' 7.0 X 10~2 M * Estimated by bootstrap analysis. See text for details. t Usual formula for estimate of standard deviation of mean: sx = \Z/V tern V operating systems (IBM RS 6000 AIX, Austin, TX, and SCO UNIX, Santa Cruz, CA for an 80486 microprocessor). For each level of the reference mean simulations, 50,000 iterations were conducted. Graphs were constructed for all error models using commercial software (Excel, Microsoft, Redmond, WA and Maple V, Waterloo Maple Software, Ontario, Canada). Although in actual practice, the reported INR should be rounded to 1 decimal place, the statistical data in this study are presented to 2 decimal places to illustrate the level of agreement between the models. RESULTS Table 1 shows specific information on the types of laboratories and settings for PT determinations. The laboratories represented a wide spectrum of practice settings, including a large urban teaching hospital, a suburban hospital, and a rural health care facility. Table 2 shows the geometric and arithmetic means of the reference populations in each of the three laboratories. The arithmetic mean approximated the geometric mean well in all three laboratories, with an absolute error of approximation less than 0.02 s in all cases. The percentage difference in the estimates was less than 0.12%. Estimates of the precision of both the arithmetic and geometric means are also shown. The geometric mean showed insignificantly better estimated precision than the arithmetic mean in all three laboratories (a CV of 0.7% to 1.1% vs. 0.7% to 1.2%). Figure 1 shows two INR distributions, one using the geometric and the other, the arithmetic reference mean in computing an INR based on PT data for the mid-level control in Laboratory 1. To illustrate the analysis for Laboratory 1,7 women and 11 men were used as the reference population. The mean midlevel control PT was 19.31 sec, the SD was 0.38 sec and the ISI was 1.93. The geometric mean PT of the reference population was 10.61 sec, whereas the arithmetic mean PT on the same reference population data was 10.62 sec. The two probability density curves are virtually superimposable in this case, showing that using the arithmetic mean in lieu of the geometric mean produces essentially no difference in the resulting INR. Similar results were obtained on control data in Laboratories 2 and 3 (graphs not shown). Table 3 and Figures 2 and 3 illustrate the effect of reference mean errors on the INR for the same mid-level control data: ISI of 1.93, a mean PT level of 19.31 sec, and a PT SD of 0.38 sec. The median INR shows the degree of bias induced by errors in the reference mean, whereas the SD and confidence intervals illustrate the effect of errors on precision (Table 3). The magnitude of the INR bias increases with greater errors in the reference mean determination, but in an asymmetric fashion. The asymmetry in the INR bias resulting from negative and positive errors in the reference mean is due to the exponential INR transformation. Underestimation of the reference mean leads to a relatively greater absolute bias in the INR than does overestimation of the reference mean. For large errors in the reference mean determination, the bias can be substantial, as seen, for example, in the INR associated with ± 10% errors in the reference mean (right column in Table 3 and Figs. 2 and 3). Although the INR precision generally improves as overestimation errors in the reference mean increase, the accuracy deteriorates (Fig. 2). However, both the accuracy and the precision of the INR deteriorate when the reference mean is underestimated (Fig. 3). These phenomena are all illustrated by Figure 4, which shows the probability density surface, fllM (i\m), for the INR conditioned on reference means which range from 9.6 sec to 11.6 sec. In this figure, the INR levels lie between 2.5 and 4.0. For errors that decrease the apparent reference mean (and produce consequently higher INR values), the width of the probability density curves increases. For errors that increase the apparent reference mean (and produce lower INR values), the width of the probability density curves is narrower. Each of the curves in Figures 1 through 3 is a cross-section of Figure 4 at its respective reference mean level. All of the individual INR curves in Figures 1 through 3 had identical CVs of 3.8%. With either over- or underestimation of the reference mean, the SD and mean of the INR change in proportion so that the CV of the INR is constant. The independence of the reference mean and the INR CV is proven mathematically in the Appendix. DISCUSSION The mathematics of the INR yields several interesting insights into its behavior. A full mathematical model allows simultaneous computation of both mean (location) and SD (spread) in the distribution of the INR, and facilitates the assessment of the theoretical and practical effects of errors in any of the INR parameters.7 In this study, we report the effects on the INR of errors in the determination of the reference population mean PT. Although the original description of the INR includes the geometric mean PT of the reference population as one of the parameters, it is common practice for some clinical laboratories to use the more familiar arithmetic mean. Many commercially available spreadsheets now offer the geometric mean as a mathematical function. In the three laboratories in this study, the arithmetic mean PT of the reference population differed only slightly from the geometric mean. Substitution of the arith- A.J.C.P. • December 1994 809 CRITCHFIELD AND BENNETT Influence of Reference Mean PT on INR 4.5 -r 4.5 10% 4 4 -• -• 3.5 3.5 -• </> W 3 111 e 2.5 3 HI I" s< < + CD O '-5 cc 0. 1 -- ED O 1-5 + oc o. 1 + 2 2 0.5 2.5 3 3.5 4 2 4.5 INTERNATIONAL NORMALIZED RATIO 2.5 3 3.5 A 4.5 INTERNATIONAL NORMALIZED RATIO FIG. 1. Effect of using arithmetic in lieu of geometric mean. Two INR probability density curves are shown for a mean PT determination of 19.13 sec, with a SD of 0.38 sec in Laboratory 1. The ISI is 1.93 and the geometric mean PT of the reference population is 10.61 sec. The arithmetic mean PT on the same reference data is 10.62 sec. The two curves are virtually superimposable, showing that in this case, the INR is essentially not affected by substituting the arithmetic mean for the geometric mean. FIG. 2. INR probability density functions: Overestimating the reference mean. Theright-mostcurve is the actual probability density function for the INR of the medium level control in Laboratory 1. The curves to the left represent errors of 1, 2, 5, and 10% above the actual geometric mean reference PT in Laboratory 1. While the location of the INR biased by the reference mean errors, the precision (expressed as the SD) of the reported INR value improves slightly (Table 3). 4.5 0% - 1 % 2 2.5 3 3.5 4 4.5 INTERNATIONAL NORMALIZED RATIO FIG. 3. INR probability density functions: Underestimating the reference mean. The left-most curve is the actual probability density function for the INR of the medium level control in Laboratory 1 (same as right-most curve of Fig. 2). The curves to the right represent errors of 1%, 2%, 5%, and 10% below the actual geometric mean reference PT in Laboratory 1. Not only is the location of the INR biased by the reference mean errors, but the INR precision (expressed as the SD) worsens (Table 3). FIG. 4. INR probability density surface for INR errors in reference mean. The conditional probability density surface (PD) for the INR is plotted as a function of INR level and the PT reference mean value. As in Figs. 1 through 3, the mean PT for the measurement was 19.13 sec and the PTSD was 0.38 sec. The correct geometric mean was 10.61 sec. The probability density curves in Figures 1 through 3 are slices of the surface along respective reference mean levels. metic mean for the geometric mean produced negligible changes in the INR distributions (Fig. 1). For any set of non-negative numbers, the arithmetic mean is greater than or equal to the geometric mean. 15 Thus, theoretically, in laboratories that use the arithmetic mean of the reference population, the INR will always be less than or equal to INR values correctly derived using the geometric mean. Overestimation of the reference mean will cause INR determinations to be lower than their correct values. Consequently, patients will, on average, tend to be relatively over-anticoagulated. Despite these theoretical concerns, the error may be negligible as found in the three laboratories in this study, where substitution of the arithmetic for the geometric reference mean had no practical implications. Although this finding is specific to the laboratories studied, the varied sizes and settings of these laboratories suggest that use of the arithmetic reference mean may be generally acceptable. Using the methods outlined in this paper, it is possible for any clinical laboratory to determine Vol. 102-No. 6 COAGULATION AND TRANSFUSION MEDICINE 810 Original Article TABLE 3. EFFECT ON THE INR OF ERRORS IN REFERENCE MEAN BY MONTE CARLO| SIMULATION--LABORATORY 1 Statistic of Interest Geometric mean reference PT (seconds) INR median INR standard deviation INR 95% confidence interval width Percentage error (%)t Actual INR Statistics 0% Error 10.61* 3.12 0.12 0.47 — Hypothetical Errors in Reference Mean +i% +2% (-i°/o) 10.82(10.40) 3.00 (3.22) 0.11 (0.12) 10.72(10.51) 3.06(3.18) 0.12(0.12) 0.46 (0.47) -1.9% (2) (-2%) 0.45 (0.48) -3.8% (4) +5% (-5%) 11.14(10.08) 2.84 (3.44) 0.11 (0.13) 0.42(0.51) -9.0% (10.4) + 10% (-10%) 11.67(9.55) 2.60 (3.82) 0.10(0.15) 0.39 (0.57) -16.8% (22.5) * Data arc rounded for presentation purposes only. See text for details on Monte Carlo simulation parameters. t The percentage error is based on the actual median INR value. whether substitution of the arithmetic mean for the geometric mean is appropriate in its setting. The precision of the geometric mean estimated by bootstrap analysis was only slightly better than the precision of the arithmetic mean estimated using the central limit theorem. The similarity of the precision profiles indicates that the arithmetic mean is nearly as precise a statistical estimator as the geometric mean, and given the negligible effects on the INR of the bias in the arithmetic mean, there is no compelling argument on statistical grounds favoring either the geometric or the arithmetic mean in the laboratories studied. Due to the familiarity with the arithmetic mean and the ease with which it can be determined, use of the arithmetic rather than the geometric mean may simplify computation of the reference mean for laboratories in which no significant numerical difference in the two reference mean estimates has been demonstrated. For laboratories lacking software to determine the geometric mean, it may be an acceptable alternative for the arithmetic mean. The important issue appears to be not whether the reference mean is computed geometrically or arithmetically, but whether the reference population study was conducted correctly. Miscalculations or errors in the reference mean can drastically affect the resulting INR values (Table 3, Figs. 2 and 3). For example, an absolute error of 10% in the reference mean resulted in errors in a median INR of 3.12 ranging from approximately — 16% to 22%. Errors of this magnitude can cause patients to receive suboptimal anticoagulation therapy, leading to over- or underanticoagulation with attendant risks of hemorrhage and thrombosis, respectively.2 This can be true even in laboratories that have excellent PT precision as previously shown. Indeed, laboratories in the United States are even more susceptible to the introduction of errors in the INR because relatively low sensitivity thromboplastins that are in common use. Any difficulties in the computation of the PT ratio (eg, the reference mean) will result in a magnification of errors through the higher ISIs in the transformation of the PT ratio to the INR. This was evident in the INR performance of Laboratory 1 at the mid-level control with errors less than 10%. In addition to placing patients at greater risk, if such errors occur in US laboratories, they may decrease acceptance of the INR by clinicians, and slow the adoption of the INR, which currently offers the best hope for standardizing oral anticoagulation treatment monitoring. Errors in reference mean determinations can arise from several sources: measurement error in the PT determinations, substituting a control material PT mean numerically different from that of the mean of the reference population in the denominator of the INR equation, improper selection of individuals for reference range studies, and errors in computation of the reference mean. Whatever the cause, clinically significant errors can occur if the reference mean is not determined correctly. Careful attention to PT quality control, reference population sampling, specimen collection and handling, and numerical computations is required for the reference mean, and thus the INR, to be accurate. Interestingly, biases in the reference mean PT do not change the CV of the resulting INR (Appendix, Equation A9). This fact allows laboratories to use approximate formulas for INR imprecision to assess the effects of parameter errors on the performance characteristics of the INR.14,16 Laboratories can model the INR using commercially available software to assess the implications of INR errors in their settings. Acknowledgments. The authors thank the technologists in their laboratories for their assistance in performing the assays and collecting data; and the reviewers for their excellent comments. REFERENCES 1. WHO. Who Expert Committee on Biological Standardization. 31st Report. WHO Tech Rep Ser 1981;658:185-205. 2. Hirsh J, Dalen JE, Deykin D, Poller L. Oral anticoagulants: Mechanism of action, clinical effectiveness, and optimal therapeutic range. Chest 1992; 102:312S-326S. 3. Gogstad GO, Wadt J, Smith P, Brynildsrud T. Utility of a modified calibration model for reliable conversion of thromboplastin times to International Normalized Ratios. Thromb Haemost 1986;56:178-182. 4. Eckman MH, Levine HJ, Paucker SG. Effect of laboratory variation in the prothrombin-time ratio on the results of oral anticoagulant therapy. N Engl J Med 1993;329:696-702. 5. Loeliger EA, Lewis SM. Progress in laboratory control of anticoagulants. Lancet 1982;2:318-320. 6. van der Velde EA. Orthogonal regression equation. In: Van den Besselaar AMHP, Gralnick HR, Lewis SM, eds. Thromboplastin Calibration and Oral Anticoagulant Control. The Hague: Martinus Hijhoff, 1984, pp 25-39. 7. Critchfield GC, Bennett ST. The International Normalized Ratio and uncertainty: Validation of a probabilistic model. Am J Clin Pathol 1994;102:115-122. 8. Lindgren BW. Some parametric families of distributions. In: Statistical Theory, ed. 3. New York: Macmillan, 1976, pp 168-169. 9. Effron, B. Bootstrap methods: Another look at the jackknife. Ann Slat 1979;7:1-26. 10. Effron B, Gong E. A leisurely look at the bootstrap, the jackknife, and cross-validation. Am Stat 1983;37:36-48. A.J.C.P. • December 1994 CRITCHFIELD AND BENNETT 811 Influence of Reference Mean PT on INR 11. Critchfield GC, Wilkins DG, Loughmiller DL, Rollins DE. Nonparametric assessment of toxicologic assay linearity by bootstrap analysis. J Anal Toxicol 1992; 16:125-131. 12. Chen CH, George SL. The bootstrap and identification of prognostic factors via Cox's proportional hazards regression model. Stat Med 1985;4:39-46. 13. Suthers GC, Wilson SR. Genetic counseling in rare syndromes: A resampling method for determining an approximate confidence interval for gene location with linkage data from a single pedigree. Am J Hum Genet 1990;47:53-61. 14. Bennett ST, Critchfield GC. Imprecision in INR determinations: Comparison of exact and approximate formulas. J Clin Pathol 1994:47:635-638. 15. Phillips ER. Metric and normed linear spaces. In: An Introduction to Analysis and Integration Theory. New York: Dover Publications, 1984, pp 49-50. 16. Taberner DA, Poller L, Thomson JM, Darby K.V. Effect of international sensitivity index (1SI) of thromboplasins on precision of international normalized ratios (INR). J Clin Pathol 1989;42:92-96. APPENDIX: DERIVATION OF INR CV WITH BIAS IN REFERENCE GEOMETRIC M E A N From basic probability theory, the INR mean is7 soP iii \/s mi exp -oo sapl/lir L •r. 1 (w/-' / J UP)2 2 <JP exp (mi 2 Vs . -Up)' 1 p* exp rrf aPylir •I p \ ' sjf ' Tdp m (P - M 2 1dp. l (A3) laP The constant of the integration, \/ms can be taken out of the integral as _L r M/ I? aPVl s (P - MP)2 exp dp. (A4) m J-a For simplicity, substitute a for the integral. Note that o does not depend on m. The simplified expression for the mean is: ^"Hf'a = (A5) Hf The variance of the INR is given as o, ,1/s - 1di "i di, (P - M/>)2 2a,,2 J-ooW /: (i - M,) 2 m exp sap\J2n (mi'" - HPY Us l i ' di. laP2 (A6) Now substitute for / and for di and use the fact that \i, is equal to aim5. With a little more algebra, the expression for the variance becomes i-fiW m m Mi (Al) "e where / is the random variable for the INR, n, is the mean of the INR, / is a specific value for /, / i s the probability density function, m is the geometric mean PT of the reference population, 5 is the International Sensitivity Index of the thromboplastin, P is the random variable representing the PT measurement, UP is the mean of the PT, and dp is the standard deviation of the PT. J-X[\m) s m\ (p - tip),21 lap2 sapf2^ fV'1 i£2. s dp. mJ m (A7) This simplifies the expression 1 C" "i (Ps - «) 2 ap^lir exp (P - UP? dp lap2 i2 m 2s ' (A8) where for simplicity /S2 has been substituted for the integral. 0 2 is independent of m, the geometric mean of the reference population. The CV of the INR is given by Substituting / = (pfmf mil/s = p, dp = -—-dp (A2) m \m With a little algebra, the expectation of the INR can be expressed as CV, = -^ X 100% = ^lm] s X 100% = - X 100%, (A9) H, (a/m ) a which is therefore also independent of m. Vol. 102-No. 6
© Copyright 2026 Paperzz