Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Theory and Application of Benchmarking in Business Surveys
Susie Fortier, Benoı̂t Quenneville
Statistics Canada, Time Series Research and Analysis Centre
Abstract
Situations that require benchmarking are very common in repeated establishment surveys, whether they are related to the
coherence of an annual and sub-annual surveys on the same
target population or to the necessity of preserving annual totals
in case of seasonal adjustments. This paper gives an overview
of the benchmarking methodology used at Statistics Canada
and presents detailed examples. Issues such as preservation of
period-to-period change and availability of benchmarks, especially at the end of the series, are discussed. Other innovative
uses of the methodology, such as its use as a linkage method to
reconcile two sections of a time series, are also presented. The
software used to produce the examples is Statistics Canada
PROC BENCHMARKING from the Forillon package.
Figure 1: Graph in the original scale – Overlay of a quarterly
indicator series, the resulting benchmarked series, the annual
figures for the quarterly and the benchmarks values averaged
over each year.
Keywords: Benchmarking, Linking, Forillon.
1 Introduction
Benchmarking deals with the issue of combining a series of
high-frequency data with a series of less frequent data into
a consistent time series. In business surveys, high-frequency
series are typically from monthly and quarterly surveys. These
series are often the only source of explicit information about
the short-term movement in the variables of interest. Reliable
information on the overall level and the long-term movement
is typically provided by less frequent annual surveys. From
here on, the high-frequency series will be referred to as the
indicator series while the less frequent series will be called
the benchmarks.
There are two main issues at stake when benchmarking:
benchmarks’ constraints and is also providing an initial adjustment in the last 6 quarters when the official annual benchmark
is not yet known.
1.1 Notation and Sample Code
The following SASr statements read a seasonal quarterly time
series into a dataset called mySeries, and the corresponding
annual benchmarks into a dataset called myBenchmarks.
DATA mySeries;
INPUT @01 year
@06 period
@08 value;
• Preserve period to period movement of the indicator
CARDS;
(monthly/quarterly) series while simultaneously attain1998 1 1851
ing the level of the benchmarks (annual);
1998 2 2436
1998 3 3115
• Consider the timeliness of the benchmarks.
1998 4 2205
These two issues are illustrated in Figure 1 where the quar- 1999 1 1987
terly indicator, the resulting benchmarked series as well as 1999 2 2635
average indicator and benchmarks series are overlayed. In a 1999 3 3435
case like this – where benchmarks are annual – the average 1999 4 2361
series represent the annual figure from respectively the quar- 2000 1 2183
terly series and the benchmarks series, divided by the period- ... ;
icity of the data (4, in this case). They form two straight lines RUN;
over each year a benchmark value is provided and are useful to
give an overview of the original discrepancies between the two DATA myBenchmarks;
sources. The resulting benchmarked series follows closely the
INPUT @01 startYear
quarter-to-quarter movement of the indicator series given the
@06 startPeriod
422
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
@08
@13
@15
CARDS;
1998 1 1998
1999 1 1999
... ;
RUN;
endYear
endPeriod
value;
4 10324
4 10200
Records in mySeries are denoted by:
st , t = 1, . . . , T,
and those in myBenchmarks by:
am , m = 1, . . . , M.
Both series are associated with dates. The date of st is indicated with variables year and period and is assumed to be
mapped into the set of integers t = 1, . . . , T . In myBenchmarks, each benchmark value needs to be associated with its
coverage period defined by its starting date t1,m (using variables startYear and startPeriod) and its ending date t2,m (using variables endYear and endPeriod). Dates must be such that
1 ≤ t1,m ≤ t2,m ≤ T .
In the usual case of benchmarking flow series to annual figures, the coverage will be indicated as above; for a given year,
the startPeriod is 1 and the endPeriod is 4 (12 for monthly
data). Benchmarking can be more generalized; in fact, benchmarks can represent individual period as it would be the case
for stock series, or any aggregates of consecutive periods.
They can themselves be non-consecutive – say, with benchmarks every other year – and can even overlay, as long as
they are coherent .To simplify the notation, t ∈ m will mean
t1,m ≤ t ≤ t2,m .
With binding benchmarking, the benchmarked series is denoted as:
θ̂ = (θ̂t ), t = 1, . . . , T
such that:
X
t∈m
θ̂t = am , m = 1, . . . , M.
(1)
Figure 2: Illustration of the bias correction effect on a quarterly indicator series. The annual figures for the indicator and
the benchmarks values averaged over each year are given as
references. Note that the bias corrected series does not match
the given benchmarks.
and used as such to compute the bias corrected series:
s†t = st · b.
The bias correction is a preliminary adjustment to reduce,
on average, the discrepancies between the two sources of data.
It can be particularly useful for periods not covered by benchmarks. The effect of the bias correction using the ratio expression is illustrated in Figure 2.
2 Simple Benchmarking Methods
2.1 Pro-rating
A simple way to respect the constraints from (1) is to use the
well-known formula for pro-rating :


am
θ̂t = s†t  X †  , for t ∈ m.
st
One of the way to deal with the timeliness issue is with the
use of a bias parameter. The bias represents the expected discrepancies between an annual benchmark and its related subThe resulting benchmarked series is plotted in Figure 3. Noannual series. It can be used to pre-adjust the indicator series.
tice that the movement going from the last quarter of a year to
Estimated with:
X
XX
the first quarter of the next year is not well preserved. This is
am −
st
most apparent from year 2005 to 2006. With pro-rating, the
m
m t∈m
adjustment is constant within each year but independent from
XX
b=
,
1
one year to the next as illustrated by the step-function shown
m t∈m
in Figure 4. This step function represents the benchmarked
the bias is added to the indicator series to get a bias corrected to indicator (BI) ratio obtained with pro-rating. BI ratio series
gives a good indication of the movement preservation between
series:
the indicator and the benchmarked series. Horizontal BI ratios
s†t = st + b.
indicate that the movement is fully preserved. Breaks or steep
Alternatively, the bias can be expressed in terms of a ratio:
slopes in the BI ratio series will result in larger discrepancies
X
between the period-to-period growth rate of the benchmarked
am
series compared to those of the indicator series. Growth rates
m
b= XX
of the pro-rating example are illustrated in Figure 5 and exst
plicitly given in Table 1.
m t∈m
423
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 3: Graph on the original scale of a benchmarked series
using pro-rating.
Table 1: Growth rates in the indicator series and in the benchmarked series using pro-rating. Again, notice the difference in
the first quarter of a year.
Figure 4: Graph on the adjustment scale – Benchmarked to indicator ratio using pro-rating. The adjustment obtained with
pro-rating shows discontinuity between the years. This is referred to as the step problem.
Figure 6: Graph on the original scale of a benchmarked series
using the proportional Denton method.
Figure 5: Bar chart of growth rates in the indicator series
and in the benchmarked series using pro-rating. Notice that
the growth rates are exactly the same within a year but show
(sometimes) large difference when going from one year to another.
424
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 7: Graph on the adjustment scale – Benchmarked to
indicator ratio using the proportional Denton method. The
adjustment obtained is smooth under the constraints given by
the benchmarks. Notice that the adjustments at the end of the
series are repeats of the adjustment from quarter 4 in 2005,
the last period covered by a benchmark.
Figure 8: Bar chart of growth rates in the indicator series
and in the benchmarked series using the proportional Denton
method. The growth rates are not exactly the same within a
year but closer when going from one year to another that those
from the pro-rating method in Figure 5.
3
Main Method
Based on Dagum and Cholette (2006), the main method of
benchmarking presented here is a generalization of many
well-known benchmarking techniques such as the pro-rating
method shown in section 2.1 and the proportional Denton
method shown in section 2.2. It has been implemented at
Statistics Canada with a user-defined SAS procedure called
PROC BENCHMARKING as part of the Forillon project. Details of the procedures are presented in section 4.
3.1 Benchmarking Formulae
The benchmarked series is obtained as the solution of this minimization problem – for given parameters λ and ρ, and preadjusted series s†t , find the value θ̂t that minimizes the followTable 2: Growth rates in the indicator series and in the bench- ing function of θt :
marked series using the proportional Denton method. Again,

2
compare these rates with those from pro-rating in Figure 1.
†
 s − θ1 
f (θ1 , . . . , θT ) = (1 − ρ2 )  1¯ ¯λ  +
¯ †¯
¯s 1 ¯




2
2.2 Proportional Denton Method
T
†
†
X  s − θt 
 st−1 − θt−1 
t
 ¯ ¯λ  − ρ  ¯

¯
¯ †¯
¯ † ¯λ
A widely used technique for benchmarking is the proportional
t=2
¯s t ¯
¯st−1 ¯
Denton method. Figure 6 shows the benchmarked series using
the proportional Denton on the original scale. The smooth BI
under the constraints from (1) repeated here for convenience:
ratio series is presented in Figure 7 while growth rates can be
X
found in Figure 8 and Table 2. Notice how the growth rates
θ̂t = am , m = 1, . . . , M.
from the last quarter of a year to the first quarter of the next
t∈m
year in the benchmarked series are closer to those of the indicator series but they do slightly differ within a year. This
More details can be found in Quenneville, Fortier, Chen and
is a result of a smooth distribution of the annual discrepan- Latendresse (2006). The solution will depend on the paramecies – as opposed to the pro-rating method, the adjustment is ter ρ. For ρ < 1, the benchmarked series is:
not equally distributed within a year (Figure 7). Mathematical
¡
¢
details will be given at the end of section 3.2.
θ̂ = s† + Ve J 0 Vε a − Js†
425
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
where
• the model adjustment parameter λ (λ ∈ IR)1 ;
½
J
=
C
=
Ωe
=
1 if t ∈ m,
[jm,t ]M ×T , jm,t =
0 else;
µ¯ ¯ ¶
¯ ¯λ
diag ¯s†t ¯ ;
h
i
ρ|k−l|
, k, l = 1, . . . , T ;
Ve
Vε
=
=
CΩe C and
JVe J 0 .
• the smoothing parameter ρ (with 0 ≤ ρ ≤ 1) and
• the bias parameter (implied in s†t ).
3.2 Effect of The Parameter λ
T ×T
First, consider the case where λ = 0 and ρ = 1. Then, the
function to be minimized under the constraints (1) becomes
Notice in the solution that the term a − Js† represents the
remaining annual discrepancies between the two sources once
the indicator series is corrected for the bias. The preceding
matrix product will allocate these discrepancies depending on
the values of the parameters ρ and λ as well as the values from
the indicator series itself. Although the solution is directly derived from the minimization formula, a statistical model can
be built with exactly the same solution. As such, the technique is referred to as the Dagum and Cholette regressionbased model.
When ρ = 1, the benchmarked series is:
θ̂ = s† + W (a − Js† )
where W is the T × M upper-right corner matrix from:
· −1 0
¸−1 · −1 0
¸
C ∆ ∆C −1 J 0
C ∆ ∆C −1 0
J
0
J
IM
·
¸
IT W
=
;
0 Wν
IM and IT are respectively the M × M and the T × T Identity matrices, W ν is the M × M matrix associated with the
Lagrange multipliers and

 −1 if l = k,
1 if l = k + 1,
∆ = [δk,l ]T −1×T , δk,l =

0 otherwise.
f (θ) =
T h³
´ ³
´i2
X
s†t − θt − s†t−1 − θt−1
=
T h³
´
i2
X
s†t − s†t−1 − (θt − θt−1 ) ,
t=2
t=2
which aims at preserving the period-to-period changes in the
original series. That is the criterion of Denton (1971) modified by Cholette (1984), called the modified Denton method2 .
When λ = 0, the benchmarking adjustment will be referred to
as additive.
Next, consider the case where the bias corrected series is
strictly positive, λ = 1 and ρ = 1. Then, the function to be
minimized under the constraints (1) becomes
"Ã
! Ã †
!#2
T
X
st−1 − θt−1
s†t − θt
f (θ) =
−
s†t
s†t−1
t=2
"
#
2
T
X
θt
θt−1
=
− †
†
st−1
t=2 st
which, contrary to popular belief, does not preserve the
period-to-period growth rates, but, as explained in Bloem,
Dippelsman, and Mæhel (2001), is a variant of the proportional Denton criterion that seeks to minimize the change in
the ratios θt /s†t . The method that preserves growth rates
requires non-linear optimization. However, the proportional
It should be noted that the solution has the same form as when method does provide a fairly close approximation. When
ρ < 1 where the benchmarked series is the bias corrected se- λ 6= 0, the benchmarking adjustment will be referred to as
ries plus a certain portion of the remaining annual discrepan- proportional.
cies a − Js† .
To justify the minimization formula used, consider the case
3.3 Effect of The Parameter ρ
where the bias-corrected series is strictly positive, λ = 21 and
ρ = 0. Then, the function to minimize under the constraints In the minimization formula, the ρ parameter relates the
(1) becomes:
change between the indicator and the benchmarked series at

2
time t to the change at time t − 1 for t = 2, . . . , T . With
T
†
X
ρ = 0, the change of the preceding term at time t − 1 dis stq− θt  ,
f (θ) =
appears
from the formula such as in the pro-rating example.
t=1
s†t
With ρ = 1, the month to month movement is fully or approximately preserved such as in the Denton and proportional
with solution:
µ
¶

Denton examples.
 s† P am
Values of the BI ratio for various values of ρ and for λ = 1
for t ∈ m, m = 1, . . . , M ;
†
t
θt =
t∈m st
are
presented in Figure 9. Differences in the BI ratios are most

s†t
otherwise.
evident at the extremities, especially at the end for data points
For data points subject to a constraint, this is the well- not covered by benchmarks. In the Denton case (ρ = 1), the
known formula for pro-rating. The next three sections further
1 Typical values of λ are 0, 1 and 1.
2
2 Originally, Denton put 1 instead of (1 − ρ2 ) as the coefficient for the first
illustrate the effect of the three parameters at play in the minimization formula:
term in f (θ)
426
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 10: Overlay of the original series, the benchmarked Figure 12: Overlay of the original series, the benchmarked
series and their respective average annual values. The series and their respective average annual values. The benchbenchmarked series here is obtained without the bias pre- marked series here is obtained with the bias pre-adjustment.
adjustment.
Figure 11: BI ratios for the example in Figure 10 without the
bias pre-adjustment.
adjustment at the last quarter covered by a benchmark is repeated for data points not covered. For values of ρ < 1, the adjustment converges (at different speeds) towards the expected
discrepancies estimated by the bias as will be explained in the
next section.
3.4 Effect of The Bias Parameter
The effect of the bias parameter is related to the value of the ρ
parameter. As mentioned above, for ρ < 1, the adjustment for
periods at the end not covered by a benchmark will converge
towards the expected discrepancies. If the expected discrepancies are null (no bias is explicitly estimated), the benchmarked
series will thus converge to the indicator series as can be seen
in Figure 10. The BI ratios for this example are presented in
Figure 11 and are shown converging towards no adjustment –
towards a default BI value of 1.
If the bias is explicitly estimated and used to pre-adjust the
series, then the benchmarked series will converge to the biascorrected series. When compared to the original series, the
Figure 13: BI ratios for the series shown in Figure 12 with the
bias pre-adjustment
benchmarked series will seem to stay at a certain fixed distance from the indicator series (Figure 12). The BI ratios converging to the estimated bias value of 0.965 are shown in Figure 13. These two last figures should be compared respectively
to Figures 10 and 11.
It should be noted that when ρ < 1, the bias parameter has
a direct impact on the values of benchmarked series for data
points not covered by a benchmark, i.e. a way to deal with the
timeliness of the benchmarks.
3.5 Timeliness Issue
The previous section showed how the bias could be used to influence the preliminary benchmarked values created for data
points not covered by benchmarks. In the typical cases, these
values would be revised when an official benchmark becomes
available for the related periods. In the meantime, preliminary benchmarked values offer an implicit forecast of the next
benchmark. Implicit forecasts for 2006 with the proportional
Denton method and with the Dagum and Cholette regressionbased method (λ = 1, ρ = 0.9 and bias pre-adjustment) are
427
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 9: BI ratios for various values of the ρ parameter.
Table 3: Implicit forecast for the 2006 annual total and the
resulting year-to-year change from benchmarking with 2 different sets of options: one set is from the Dagum and Cholette
regression-based method with bias pre-adjustment, the other
gives the proportional Denton.
shown in Table 3.
An alternative for the preliminary benchmarking and the
timeliness issue is to use an explicit forecast of the next benchmark. If the forecast is of good quality, one could expect the
revision to be small once the true benchmark is known. Auxiliary information can be used to provide the explicit benchmark. One possibility is to use the annual growth rate from
the indicator series on the last known benchmark. The bench-
Table 4: Explicit forecast for the 2006 annual total using the
annual growth rate from the indicator series on the last known
benchmark (year 2005).
marks with explicit forecast for 2006 using this method are
presented in Table 4. The resulting benchmarked series is illustrated in Figure 14 and the corresponding BI-ratios are in
Figure 15. Using the annual growth rate from the indicator series to forecast the next benchmark is equivalent to using the
last known annual discrepancy as a forecast of the next. This
can be seen in Figure 14 where the distance between the average indicator series and the average benchmark is the same
in 2005 and 2006; and again in Figure 15 where the average
ratio is the same in 2005 and 2006 at 0.895.
428
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 16: Overlay of the original series, the benchmarked
series and their respective average annual values. The bias
estimated on recent years was used to pre-adjust the series.
Figure 14: Overlay of the original series, the benchmarked
series and their respective average annual values. The benchmarked series here is obtained with an explicitly forecasted
benchmark for 2006.
Figure 17: BI ratios for the series shown in Figure 16 where
the bias estimated on recent years was used to pre-adjust the
series.
A final alternative to deal with the timeliness issue is to use
the bias pre-adjustment but to compute the bias on the most
recent years only. The benchmarked series and the resulting
BI ratios using only the last three years to compute the bias are
illustrated in Figures 16 and 17. The computed bias is 0.939.
The two main methods to deal with the timeliness issue can
be compared. The use of explicit forecast based on the indicator series will, by construction, preserve the annual growth
rate of the indicator series. This can be a desirable feature especially when nothing else is available. The drawback of this
version of the explicit forecast is that the forecasted annual
Figure 15: BI ratios for the series shown in Figure 14 where discrepancies is based on only one year and can lack robustthe benchmarked series is obtained with an explicitly fore- ness. While the use of the bias will typically not keep the
casted benchmark for 2006.
annual growth rate of the indicator series, it is more robust.
Note that it can still be affected by non-representative (or nolonger representative) historical data. Using only the last few
years to compute the bias could be a good compromise. In the
end, the method that will minimize the revision once the true
benchmark is known will depend on the series themselves.
429
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
3.6 Summary of Methods
The methods presented in this section:
• Pro-rating;
• Denton (and proportional Denton);
• Dagum and Cholette regression-based method; (with or
without bias correction);
• (Proportional) Denton with explicit forecast;
are all obtained from the same numerical minimization formula and results can be produced with Statistic Canada userdefined procedure PROC BENCHMARKING (see section 4).
Other benchmarking methods not discussed here include
numerical methods revolving around different minimization Figure 18: Proc Benchmarking in Forillon - Input data pane.
functions. See annex 6.1 in Bloem, Dippelsman, and Mæhel
(2001) for variants and references. The regression-based
model from Dagum and Cholette (2006) can also be more
general than the restricted numerical version presented here.
Finally, Chen and Wu (2006) discuss the link between the numerical, regression-based and signal extraction methods.
4 Proc BENCHMARKING
As described in Latendresse, Djona and Fortier (2006), the following PROC BENCHMARKING statement performs benchmarking on the indicator series mySeries using annual constraints from myBenchmarks.
PROC BENCHMARKING
BENCHMARKS=myBenchmarks
SERIES=mySeries
OUTBENCHMARKS=outBenchmarks
OUTSERIES=outSeries
OUTGRAPHTABLE=outGraph
RHO=0.729 LAMBDA=1 BIASOPTION=3;
RUN;
Figure 19: Proc Benchmarking in Forillon - Parameter options
pane.
• BIASOPTION=3 Bias estimation – bias is estimated and
used to pre-adjust the sub-annual series.
The benchmarked series is stored in the SAS data set
called outSeries. The OUTBENCHMARKS option stores the
benchmarks that were actually used by the procedure and
OUTGRAPHTABLE names the output data set that contains the
data necessary to produce some analytical tables and graphs.
The RHO option identifies the smoothing parameter described
in section 3.3, here with the default value for quarterly3 series RHO= 0.93 = 0.729. The LAMBDA option refers to the
adjustment model parameter described in section 3.2.
Valid values for the BIASOPTION are:
• BIASOPTION=1 No bias estimation – default value4
will be used to pre-adjust the sub-annual series.
PROC BENCHMARKING is part of Statistics Canada Forillon5 package. The package comes with a user-friendly interface for SAS Enterprise Guider . The interface allows the user
to easily provide input files (Figure 18) and input parameters
(Figure 19) to the procedure. Output files can also be identified (Figure 20).
One of the main advantages of the interface is its embedded graphical capabilities. It will use the data saved with the
OUTGRAPHTABLE option to plot different types of graphs.
Figure 21 shows the menu to select which graphs are to be
plotted.
5 Other Uses of Benchmarking
• BIASOPTION=2 Bias estimation for information purpose only – bias value is estimated but not used to pre- Other uses of the benchmarking methodology can be found
adjust the sub-annual series.
in seasonal adjustment and linking or bridging different time
3 Default value for monthly series is RHO=0.9
segments of series.
4 The default value is 0 in the additive case (LAMBDA= 0) and 1 otherwise.
This can be changed with the optional keyword BIAS.
430
5 [email protected]
for more details.
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
5.1 Seasonal Adjustment
Seasonally adjusted series can be required to match given annual totals. This can be the case in complex production procedures such as those from systems of National Accounts. Another situation where forcing annual totals is necessary is in
the first step of the reconciliation strategy presented by Quenneville and Rancourt (2006).
As described in Quenneville, Fortier, Chen and Latendresse
(2006) and almost textually reproduced here for convenience,
forcing annual totals with the methodology of section 3.1 can
be achieved in the X-12-ARIMA seasonal adjustment program (Findley, Monsell, Bell, Otto, and Chen (1998)), version 0.3+ with the use of the spec FORCE and the argument
Type=regress. Results are provided in X-12-ARIMA’s table D 11.A.
In X-12-ARIMA, the bias parameter is replaced with argument target, which specifies which series is to be used as
the target for forcing the totals of the seasonally adjusted series. The choices are:
• Original;
Figure 20: Proc Benchmarking in Forillon - Results pane.
• Caladjust (Calendar adjusted series );
• Permprioradj (Original series adjusted for permanent prior adjustment factors );
• Both (Original series adjusted for calendar and permanent prior adjustment factors ).
In the following seasonal adjustment options, a multiplicative monthly seasonal adjustment is to be performed using
automatic outlier identification and ARIMA forecast extension on the Canadian Department Stores Retail Trade Series6 .
Trading-day and Easter adjustment factors are computed from
the regression spec. The annual totals of the seasonally adjusted series will be forced to equal the totals in the calendar
adjusted series with the regression-based method. The suggested default values7 are hard-coded in the FORCE spec. The
parameters are set to lambda= 1 to smooth the ratios (instead
of the differences) in the annual totals. The value rho=0.9 ensures that the BI-ratios in the incomplete years will converge
to 1, which is defined as the theoretical value for the ratio of
the calendar year total of the calendar adjusted raw series over
that of the seasonally adjusted series.
series{... save=a18}
transform{function=log}
regression{ variables=(TD easter[8])}
outlier{ ...}
arima{...}
forecast{...}
x11{... save=d11}
force{
lambda=1
rho=0.9
Figure 21: Proc Benchmarking in Forillon - Graph options
pane.
6 Trade
Group 170
7 The spec FORCE has another useful argument in usefcst.
value yes, it determines if forecasts are appended to the series.
431
With default
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 22: Overlay of a seasonally adjusted series (D 11) and
the same seasonally adjusted series with forced annual totals
(D 11.A) (in millions).
Figure 23: Difference between the seasonally adjusted series
(D 11) and the same seasonally adjusted series with forced
annual totals (D 11.A).
target=calendaradj
type=regress
save=saa }
Figure 22 displays the seasonally adjusted series (D 11) and
the corrected seasonally adjusted series (D 11.A). In this case
the indicator series is the seasonally adjusted series and the
benchmarks are the annual totals of the calendar adjusted raw
series. Not much can be seen in that figure so, the differences
in the two series are displayed in Figure 23 where one can
clearly see the effect of the smoothing parameter rho=0.9.
Figure 24 displays the growth rates in the seasonally adjusted
series before and after the FORCE spec. Clearly, benchmarking did not affect the growth rates in any noticeable way,
which was expected given the differences displayed in Figure 23.
By default, the FORCE spec implies that the calendar year Figure 24: Growth rates in a seasonally adjusted series (E 6)
totals in the seasonally adjusted series will be adjusted to be and in the same seasonally adjusted series with forced annual
equal to the calendar year totals of the target series. An totals (E 6.A).
alternative starting period for the annual total can be specified with the start argument. Annual totals starting at any
other period other that start may not be equal as shown in
Figure 25. Contrary to popular belief, benchmarking a seasonally adjusted series to annual totals via the methodology
implemented in the FORCE spec does not produce a seasonally adjusted series with constant seasonal factors.
5.2 Linking
Linking or bridging exercises are performed to join different
time series segments into a consistent time series. It can be
used to minimize the breaks caused by a survey redesign, the
introduction of a new classification system or the change of a
concept or method.
With proper identification of anchor points as benchmarks,
PROC BENCHMARKING can be used to link two segments of Figure 25: Difference between the sum of 12 consecutive
time series if they overlap. If they don’t overlap, then a model months computed on the seasonally adjusted series with
could be used to extend one of the two segments. The over- forced annual totals (D 11 A) and its target series (A 18).
lap periods are used to measure the difference in levels. The
432
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
Figure 26: Two segments of a series ; the historical segment
is used as the indicator while the overlapping part of the new
segment is used as anchor points (benchmarks).
smoothing parameter ρ can be used to gradually bridge the gap
between the two levels.
Figure 26 represents two segments of a series: a historical
segment – used as the indicator series – and a new segment
of 3 data points (say, from a redesigned sample) – used as the
”benchmarks”. We assume that the new level is the correct
one and the series must be adjusted to match this level. The
historical segment is used as the (only) indicator of the periodto-period movements.
Under the assumption that the difference between the two
segments would be constant over time, a linked series can be
created by benchmarking the historical series to the new series
with bias pre-adjustment, parameters λ = 1 and ρ = 0.9. A
change of concept – say the new segment now includes certain types of establishments that were excluded in the historical segment – could be a case where this assumption is acceptable. Note that other underlying assumptions are at play,
mainly that the historical segment is a ”good” proxy of the
new type of establishments’ behavior. Figure 27 shows the
resulting linked series as the ”benchmarked” series. The BIratios are shown in Figure 28. The last 3 points represent the
observed discrepancies between the two segments; the bias estimated at 1.046 is the expected discrepancy that is applied to
most of the historical series.
Another assumption is that the historical level was corrected
somewhere in the past. This is a typical assumption with sample restratification. We assume the sample gave correct level
estimates right after the sample plan was designed and applied;
then, with time, it gradually started to degenerate. A new restratified sample gives a better indication of the correct level.
In such a case, the gap observed during the segments’ overlap should be gradually smoothed in. To achieve this with
PROC BENCHMARKING, an extra benchmark representing a
time point from which the historical level is assumed correct
must be used. Benchmarking the historical indicator series
to the augmented benchmarks (”correct” historical series in
January 2004 and new data points in October, November and
December 2006) with no bias adjustment, λ = 1 and ρ = 0.9
Figure 27: Overlay of the (historical) indicator series and the
(linked) benchmarked series adjusted as a true level shift.
Figure 28: BI ratios from the linking exercise as a true level
shift of Figure 27.
433
Papers presented at the ICES-III, June 18-21, 2007, Montreal, Quebec, Canada
inclusion of more options provided in Dagum and Cholette
(2006) are also in the works.
References
Bloem, A. M., R. J. Dippelsman, and N. Ø. Mæhel (2001): Quarterly National Accounts Manual, Concepts, Data Sources and Compilation. International Monetary Fund, Washington DC.
Chen, Z. G., and Wu, K. H. (2006): “Comparison of Benchmarking Methods
With and Without Survey Error Model,” International Statistical Review,
74, 285–304.
Cholette, P. (1984): “Adjusting Sub-Annual series to yearly Benchmarks,”
Survey Methodology, 10, 35–49.
Figure 29: Overlay of the (historical) indicator series and the
(linked) benchmarked series adjusted as a gradual level shift.
Dagum, E.B. and Cholette, P. (2006). Benchmarking, Temporal Distribution
and Reconciliation Methods for Time Series Data. New York: SpringerVerlag, Lecture Notes in Statistics #186.
Denton, F. (1971): “Adjustment of Monthly or Quarterly Series to Annual
Totals: An Approach Based on Quadratic Minimization,” Journal of the
American Statistical Association, 82, 99–102.
Findley, D. F.,Monsell, B. C., Bell, W. R., Otto, M. C., and Chen, B. C.
(1998): “New Capabilities and Methods of the X-12-ARIMA SeasonalAdjustment Program,” Journal of Business and Economic Statistics, 16,
127–177.
Latendresse, E., Djona, M. and Fortier, S.(2006). Benchmarking Sub-Annual
Series to Annual Totals – From Concepts to SASr Procedure and SASr
Enterprise Guider Custom Task, Paper presented at the 2007 SAS Global
Forum, Orlando, April 2007.
Quenneville, B., Fortier, S., Chen, Z.-G. and Latendresse, E.(2006). Recent
Developments in Benchmarking to Annual Totals in X-12-ARIMA and
at Statistics Canada, Paper presented at the 2006 Eurostat conference on
Seasonality, Seasonal Adjustment and Their Implications for Short-Term
Analysis and Forecasting, Luxembourg, May 2006.
Figure 30: BI ratios from the linking exercise as a gradual
level shift of Figure 29.
Quenneville, B., and Rancourt, E. (2006): “Simple methods to restore the
additivity of a system of time series,” Proceedings of the Eurostat Workshop on Frontiers in Benchmarking Techniques and Their Application to
Official Statistics, Luxembourg, April 2005.
will produce the linked series shown in Figure 29. The corresponding BI-ratios are presented in Figure 30. Notice that the
adjustment goes from 1 ( no adjustment when the historical
series is deemed ”correct” – in January 2004 in the example)
to the adjustments required by the data from the new segment
at the end of 2006.
6
Conclusion
This paper presented an introductory overview of benchmarking with a restricted version of the Dagum and Cholette regression based method. Some variants of this benchmarking method that can be achieved through a numerical method
with PROC BENCHMARKING from Statistic Canada’s Forillon package were presented. Other uses of the benchmarking
methodology were presented with application to seasonal adjustment and linking exercises.
Future developments of PROC BENCHMARKING involve
technical improvements such as enhancement for batch processing with VAR and BY statements as well as for the use
of explicit forecasts. Improvement to the bias estimation and
434