SETTING THE OPTIMAL DECISION THRESHOLD AND ANALYSIS OF
IMPACT OF SAMPLE SIZE ON AUTOMATIC MODULATION
CLASSIFICATION BASED ON SIXTH-ORDER CUMULANTS
VLADIMIR D. ORLIC
“Vlatacom d.o.o.”, Belgrade, [email protected]
MIROSLAV L. DUKIC
Faculty of Electrical Engineering, Belgrade, [email protected]
Abstract: In this paper the optimal value of decision threshold in automatic modulation classification process, based on
the value of normalized sixth-order cumulants, is analyzed. Possibilities of positioning the decision threshold at
numerical values defined by variances of particular sixth-order cumulants (corresponding to observed modulation
formats) are considered. Performances of classification process organized in previously mentioned manner, along with
performances of classification with standard threshold setting approach (where threshold is always positioned in the
middle of successive numerical values of sixth-order cumulants), are presented. Since automatic modulation
classification performance is directly implicated by the size of data sample collected for the processing purposes, the
impact of sample size on optimal threshold setting is also considered. Expected classification performance is verified
through Monte-Carlo simulations.
Key words: automatic modulation classification, sixth-order cumulants, variance, decision threshold.
manner) and make decision from their observed values.
Feature-based algorithms are very simple, with nearlyoptimal performance, when designed properly. Their low
computational complexity recommends these algorithms
for practical applications: among many different
algorithms, those based on higher order statistics are
especially suitable for implementation, as being extremely
simple. Higher order statistics – moments and cumulants
can be used for AMC, with some advantage given to
cumulants because of their robustness on presence of
additive white Gaussian noise (AWGN).
1. INTRODUCTION
Implementation of advanced communication systems for
military applications in a crowded electromagnetic
spectrum is a challenging task: while friendly signals
should be transmitted and received securely, hostile signals
should be located, identified and jammed. Such
applications have emerged the need for intelligent and
flexible solutions, where the automatic recognition of the
modulation of a detected signal is a major task [1]. This
recognition process, commonly named – automatic
modulation classification (AMC), is an intermediate step
between detection and demodulation within a receiver.
AMC has been investigated in US military
communications-related research for years and would be
considered to be integrated with the future communication
standards involving adaptive modulation [2].
Employing normalized fourth-order cumulants as features
of interest in AMC was originally discussed in [3]. In the
same work authors explored the variance of the sample
estimates of normalized fourth-order cumulants, and
concluded that value of this variance directly depends on
the sample size. For AMC algorithm based on normalized
sixth-order cumulants, [4], expressions for the variance of
the cumulants’ sample estimates are derived in [5], and
numerical values for some modulation formats of interest
are calculated. In the same paper the direct dependence of
variance on the sample size is reported, and novel
criterion for mutual comparison between different
cumulant-based AMC algorithms is proposed.
In general, typical AMC approaches can be classified in
two major groups: decision-theoretic (likelihood-based)
and pattern-recognition (feature-based) algorithms.
Decision-theoretic algorithms are based on the likelihood
function of the received signal and the decision is made
through comparing the likelihood ratio with appropriate
threshold. Although these algorithms are optimal in
Bayesian sense, they suffer from significant
computational
complexity.
Pattern-recognition
algorithms, on the other hand, exploit several different
features of received signal (commonly chosen in ad hoc
In this paper we present results of further analysis of
normalized sixth-order cumulant’s variance and some
aspects of its impact on AMC procedure: simulated values
511
while the self-normalized sixth-order cumulant is defined as:
of sample estimate variances, the positions of optimal
comparison thresholds defined by variance values,
algorithm performance dependence on sample size and
some strategies for enhancement of AMC algorithm.
Cˆ 63, x = C63, x /(C21, x )3 .
We adopt the following relationships between the
cumulants of x and the cumulants of y (associated with
received sequence y(n)):
2. AMC ALGORITHM ON THE BASIS OF
SIXTH-ORDER CUMULANTS
The received signal sequence y (n) , corrupted by AWGN
only during propagation, can be represented by:
y ( n) = x ( n) + g ( n) ,
(1)
and g ( n) is AWGN with a zero mean and a variance of
transmitted data sequence x(n) , the second-order
cumulant C21, x = cum( x, x* ) is given by:
2
2
2
+12 E ( x 2 ) E ( x ) + 12 E 3 ( x )
Cˆ 63, x =
2
2
,
(6)
C63, y
(C21, y − σ g2 )3
.
(7)
while calculation of second-order and sixth-order
cumulant of received signal practically comes down on
calculation of mean-values over ensemble of collected
signal samples, and their further combining. If number of
samples is represented with N, equation (7) in practical
realization can be rewritten as:
(2)
be expressed as:
4
C21, y = C21, x + σ g2 .
The noise power σ g2 can be measured at receiving point,
The sixth-order cumulant C63, x = cum( x, x, x, x* , x* , x* ) can
6
(5)
Cˆ 63, x =
σ g2 . For zero-mean random variable x, associated with
C21, x = E ( x ) .
C63, y = C63, x ,
Consequently, we have:
where x(n) stands for transmitted modulated symbols,
C63, x = E ( x ) − 9 E ( x ) E ( x ) +
(4)
(3)
2
3
N
⎛ 1 N 2
⎞
⎛1 N
⎛1 N
1 N
1 N
6
4 1
2⎞
2
2⎞
⎜
⎟
−
⋅
+
⋅
+
y
n
y
n
y
n
y
n
y
n
y
n
(
)
9
(
)
(
)
12
(
)
(
)
12
(
)
∑
∑
∑
⎜ ∑
⎟
⎜ ∑
⎟
⎜ N∑
⎟
N n =1
N n =1
N n =1
⎝ N n =1
⎠
⎝ N n =1
⎠
⎝ n =1
⎠
1 N
2
( ∑ y (n) − σ g2 )3
N n =1
order k with m conjugations. For complex signals with N
samples error variance is given by:
2
2
2
2
− 54m4,2
N var(C63, x ) = [ m12,6 − m6,3
] + 9[ m2,1
(48m4,2 m2,1
Table 1. Theoretical normalized sixth-order cumulants
for some constellations
QPSK
16-QAM
64-QAM
Ĉ63
4.000
2.080
1.797
4
2
+ 96m2,1
− 64m6,3m2,1 ) + m4,2 (9m4,2
+ 16m6,3m2,1 (10)
− 2m8,4 ) + m2,1 (17 m8,4 m2,1 − 2m10,5 )]
As it can be noticed from eq. (9) and (10), error variances
are directly proportional with sample size N, and take
different values for different modulation formats [5].
However, limited precision in numerical calculations is
not the only source of dispersion of higher-order
cumulants’ values: dispersion is also implicated by
unequal number of different symbols in randomly
generated messages, and by the presence of the noise.
3. VARIANCE OF CUMULANT ESTIMATES
Theoretical values of higher-order cumulants in AMC
problems, like those shown in Table 1, represent only
expected values of cumulants; some portion of dispersion
around expected values is unavoidable in practical
calculations. This phenomenon was explored and
described in literature for fourth-order cumulants [3], and
for sixth-order cumulants [5]. The error variance due to
limited precision of calculation of C63, x , for real signals
Adopting the procedure described by eq. (8), decision
making process for the modulation recognition is based
on comparison of obtained values of estimates Cˆ 63, x with
with N samples, is given with:
2
2
2
2
4
− 126m4,2
+ 384m2,1
N var(C63, x ) = [m12,6 − m6,3
] + 9[m2,1
(384m4,2m2,1
2
− 128m6,3m2,1 ) + m4,2 (9m4,2
+ 16m6,3m2,1 − 2m8,4 )
(8)
where mk , m = E[ y k − m ( y * ) m ] represents mixed moment of
In Table 1 the theoretic values of the sixth-order
cumulants
for
some
well-adopted
modulation
constellations are shown.
Constellation
.
(9)
+ m2,1 (25m8,4m2,1 − 2m10,5 )]
512
predefined thresholds. Comparison thresholds are
commonly positioned at the middle of intervals between
expected (theoretical) values that correspond with
particular modulation formats. For example, from Table 1
it can be concluded that comparison threshold between
QPSK and 16-QAM signals should be placed at value
(4+2.08)/2=3.04.
However, it is a fact that this manner of thresholds’
positioning is, by theory, appropriate only if decision is to
be made between values having mutually equal variances,
and while occurring with equal probability, [6]. Statistics
of cumulants is considered to obey Gaussian law [3], but
their variances appear to be mutually unequal, for
different modulation formats. In case of decision making
between Gaussian parameters having different variances,
the position of optimal threshold can be derived from
theoretical condition for minimum probability of error:
−
1
e
Pr{s1}
2πσ 1
(VT − μ1 ) 2
2σ12
−
1
= Pr{s2 }
e
2πσ 2
proportion: for QAM signals this proportion
approximately holds under all considered SNR values,
while for QPSK signal higher sensitivity is shown for
changes of N value at higher SNR, and it converges to
~ 1/ N when the SNR value decreases. For all considered
modulation formats, higher values of SNR at fixed sample
size result with lower variances. This is, also, expected
way of behavior.
Table 2. Cˆ 63, x variance under SNR=20dB
(VT − μ 2 )2
2σ 22
QPSK
,
(11)
where Pr{si } stands for a priori probability of occurrence
of event si , whose statistics is Gaussian with mean μi
and variance σ i2 . Optimal threshold VT can under the
terms of mutually equal a priori probabilities Pr{si } be
calculated from equivalent equation:
−
σ1
=e
σ2
(VT − μ1 ) 2
2σ12
+
(12)
σ 2 μ − σ 2 2 μ1
VT = 1 22
σ1 − σ 22
σ1
+ σ 12σ 2 2 ( μ1 − μ2 ) 2 .
σ2
σ 12 − σ 2 2
±
2.2 ⋅ 10−3
N = 2.000
8.7 ⋅ 10−5
4.2 ⋅ 10−3
4.3 ⋅ 10−3
N = 1.000
3.1 ⋅ 10−4
8.3 ⋅ 10−3
9 ⋅ 10−3
N = 500
1.2 ⋅ 10−3
1.7 ⋅ 10−2
1.9 ⋅ 10−2
N = 250
5.2 ⋅ 10−3
3.8 ⋅ 10−2
4.1 ⋅ 10−2
Table 3. Cˆ 63, x variance under SNR=15dB
which after taking the natural logarithm and solving the
following quadratic equation gives this solution:
2(σ 2 2 − σ 12 )σ 2 2σ 12 ln
2.1 ⋅ 10
64-QAM
−3
2.5 ⋅ 10
QPSK
,
16-QAM
N = 4.000
(VT − μ 2 )2
2σ 22
−5
−5
16-QAM
2.5 ⋅ 10
−3
64-QAM
2.7 ⋅ 10−3
N = 4.000
9.7 ⋅ 10
N = 2.000
2.4 ⋅ 10−4
5 ⋅ 10−3
5.3 ⋅ 10−3
N = 1.000
6.7 ⋅ 10−4
1 ⋅ 10−2
1.1 ⋅ 10−2
N = 500
2 ⋅ 10−3
2 ⋅ 10−2
2 ⋅ 10−2
N = 250
6.4 ⋅ 10−3
4.3 ⋅ 10−2
4.7 ⋅ 10−2
Table 4. Cˆ 63, x variance under SNR=10dB
(13)
QPSK
The sign of square-root term in previous equation should
be chosen in the manner which results with solution
having numerical value from interval ( μ1 , μ 2 ) (under
assumption μ1 < μ 2 ).
Thus, it is of interest to determine overall variance of
normalized sixth-order cumulants, which represents the
product of synergy of several effects: limited numerical
precision, random symbol generation and presence of
noise, simultaneously. Having in mind results described
with equations (9) and (10), it is also of interest to explore
dependence of overall variance on sample size N.
−4
16-QAM
3.8 ⋅ 10
64-QAM
−3
4.3 ⋅ 10−3
N = 4.000
8.3 ⋅ 10
N = 2.000
1.8 ⋅ 10−3
7.9 ⋅ 10−3
8.7 ⋅ 10−3
N = 1.000
3.7 ⋅ 10−3
1.6 ⋅ 10−2
1.7 ⋅ 10−2
N = 500
8.8 ⋅ 10−3
3.3 ⋅ 10−2
3.7 ⋅ 10−2
N = 250
2.1 ⋅ 10−2
6.8 ⋅ 10−2
7.9 ⋅ 10−2
Table 5. Cˆ 63, x variance under SNR=5dB
QPSK
With this goal in mind, a number of computer simulations
is performed, and variances of Cˆ 63, x in AWGN channel,
for modulation formats belonging to the set {QPSK, 16QAM, 64-QAM}, through 2.000 mutually independent
experiments per each modulation format, are calculated
under various values of sample sizes N and various values
of signal-to-noise ratio (SNR). Achieved results are
presented in Tables 2 - 6.
−2
16-QAM
1.5 ⋅ 10
−2
64-QAM
1.5 ⋅ 10−2
N = 4.000
1 ⋅ 10
N = 2.000
2 ⋅ 10−2
3 ⋅ 10−2
3.3 ⋅ 10−2
N = 1.000
4.3 ⋅ 10−2
6 ⋅ 10−2
6.7 ⋅ 10−2
N = 500
8 ⋅ 10−2
1.3 ⋅ 10−1
1.4 ⋅ 10−1
N = 250
1.9 ⋅ 10−1
2.6 ⋅ 10−1
2.9 ⋅ 10−1
Table 6. Cˆ 63, x variance under SNR=0dB
QPSK
Results from Tables 2-6 show that variances of estimates
Cˆ 63, x change with sample size N and SNR values. As it
should be expected, larger values of N result with lower
variances. It is interesting to notice that this change is
been made with approximately following ~ 1/ N
513
−1
16-QAM
2.7 ⋅ 10
64-QAM
−1
2.8 ⋅ 10−1
N = 4.000
2.5 ⋅ 10
N = 2.000
5.1 ⋅ 10−1
5.7 ⋅ 10−1
5.5 ⋅ 10−1
N = 1.000
9.7 ⋅ 10−1
1.1
1.2
N = 500
2.2
2.3
2.4
N = 250
4.8
5.5
5.3
Mutual relations of variances corresponding with particular
constellations represent important result of described tests,
since they provide the information needed for deeper
considerations on optimal threshold values (eq. 13).
may result with improvement in overall complexity and
processing time. Precisely, simulations show that
successful (errorless) classification of QPSK signals can
be achieved even with very small sample sizes. This result
comes from the fact that Cˆ 63, x values for QPSK signals
4. DISCUSSION ON OPTIMAL THRESHOLDS
are distanced from Cˆ 63, x values of QAM signals enough to
From Tables 2 – 6 it can be noticed that variances of 16QAM and 64-QAM signals are mutually very close in
numerical values, for all sample sizes and signal-to-noise
ratio values. This means that optimal thresholds for
classification between 16-QAM and 64-QAM signals lay
at positions very close to the middle of interval between
theoretical values of Ĉ63 for these signal formats (i.e. at
3.04 exactly), for every N and SNR. Numerous
simulations of AMC algorithm on the basis of sixth-order
cumulants, which target these constellations (among
others), show that mutual distinguishing of signals
between 16-QAM and 64-QAM formats has the most
significant impact on overall AMC performance [4,5].
provide appropriate decision-making process even under
significant variances of sixth-order cumulants’ estimates.
On the other hand, mutual distinguishing of 16-QAM and
64-QAM signals requires adopting higher values of N.
Thus, as a strategy defined to provide lower algorithm
processing time (and relax memory requirements), twolevel AMC procedure can be defined in following
manner:
− First, estimate Cˆ 63, x value, according to eq. (8), by using
some relatively low number of samples N1 , and
compare it with “middle of the interval” threshold
between QPSK and 16-QAM signal’s cumulants (equal
to 3.04). If signal is within this step recognized as
QPSK, AMC procedure is over;
Further, it can be concluded that variance of QPSK signal
is significantly lower than variances of 16-QAM and 64QAM signals, for higher SNR values, and becoming
asymptotically equal with them as SNR decreases. So, it
is of interest to consider possibility of improving the
performance by adopting optimal thresholds (eq. (13)) for
the purpose of distinguishing QPSK from QAM signals.
We have tested this possibility in detail and simulations
show that, since they have been located at significant
− If estimated Cˆ 63, x value corresponds with QAM signals’
values, repeat the procedure from eq. (8) with higher
number of samples N 2 in order to provide necessary
precision in classification of QAM signals, and
compare it with “middle of the interval” threshold
between 64-QAM and 16-QAM signal’s cumulants.
distance from Cˆ 63, x values of QAM signals, Cˆ 63, x of
QPSK signals are successfully recognized with 100%
success at all SNR values from 5dB and above, with
“middle of interval” threshold setting. For SNR values
lower than 5dB ambiguity in classification evidently
occurs, but it is mainly caused by the noise, whose strong
presence makes variances of all considered modulation
formats to be approximately equal (Table 6). Thus, it can
be concluded that using optimal thresholds (in the
meaning of eq. (13)) does not lead to performance
improvement with QPSK signals neither, since the impact
of noise which causes overlapping in Cˆ 63, x values makes
these optimal values to be very near to the middle of
interval between theoretical values of cumulants.
Analysis of achieved results and given discussion on this
subject, clear the question of optimal threshold setting in
AMC algorithm on the basis of normalized sixth-order
cumulants. It can be concluded that threshold manipulations
due to variations in variance values do not improve
algorithm performance, and setting thresholds at the “middle
positions” between expected values is the most adequate
solution. We have confirmed given conclusion through large
number of Monte-Carlo experiments, which showed that no
significant improvement can be made by using threshold
values different than middle positions of theoretical intervals
shown in Table 1.
In order to evaluate the procedure given above, we have
simulated this procedure through 2.000 Monte-Carlo
experiments, under the terms of probability of correct
classification ( PCC ) versus signal-to-noise ratio (SNR).
Simulated modulation formats are taken from the set
{QPSK, 16-QAM, 64-QAM}. The simulation of proposed
two-level procedure is realized by using the values of
N1 = 500 for sample size in coarse classification of QPSK
signals in the first step, and N 2 = 2.000 samples for fine
classification of 16-QAM and 64-QAM signals in the
second step. Achieved values of PCC are presented in
Picture 1, along with values of PCC achieved by using the
fixed value of N = 2.000 for the number of processed
samples.
As it can be noticed from the Picture 1, proposed twolevel procedure truly follows the performance of
“classical” AMC approach very closely, at all SNR values
higher than, approximately, 5dB. Thus, it can be
concluded that proposed procedure should be expanded
with a pre-processing step for estimation of received
signal power (which is equal to second-order cumulant
C21, y ) and measurement of noise power σ g2 , and further
continuing with two-level classification if estimated SNR
is larger than 5dB, or using fixed higher value of N
otherwise. Since both estimation of C21, y and mesurement
of σ g2
5. MANIPULATIONS WITH SAMPLE SIZE
are already included in considered AMC
algorithm, described pre-processing step does not involve
any additional complexity in classification process.
Analysis of impact of sample size N on AMC algorithm
performance leads to some interesting conclusions, which
514
In scenario where values N1 = 500 and N 2 = 2.000 are
adopted for two-level classification, in comparison with
approach based on fixed value of N = 2.000 samples,
approximately 25% of processing time is saved at higher
SNR values, in average. Two-level classification with
N1 = 500 and N 2 = 4.000 samples, in comparison with
classical procedure with N = 4.000 sample size, results
with approximately 29.17% savings in processing time, in
average, as practically expected at values of SNR higher
than 5dB.
6. CONCLUSION
In this paper the overall variances of sample estimates of
normalized sixth order cumulants, for complex signal
constellations, under various sample sizes and signal-tonoise ratios, are presented. From empirical analysis it is
concluded that classical “middle of the interval”
thresholds should be used in implementation of
considered algorithm. New two-stage classification
procedure, based on manipulations with sizes of samples
being included in processing, is proposed, and its
appropriateness for adopting in considered algorithm is
verified through simulations. Selection of sample sizes
which are commonly considered in AMC literature, for
proposed two-level classification procedure, results with
significant savings in amount of time necessary for
numerical calculations. Future research in this field
should include multipath channel model, as being much
more realistic than channel with AWGN only.
Picture 1. Probability of correct classification of signals
from the set {QPSK, 16-QAM, 64-QAM} versus SNR: i)
new two-level procedure with N1 = 500 , N 2 = 2.000
(yellow), and ii) classical procedure with fixed value of
N = 2.000 (blue).
It is also interesting to estimate the savings in processing
time which should be expected with proposed two-level
procedure; implementation of proposed procedure can be
organized in two different ways, so we calculate expected
savings for both implementation methods. First, two-level
classification can be realized with mutually independent
calculations of formula from eq. (8) with N1 samples at
the first stage and, (if needed) with N 2 samples at the
second stage. For example, if values N1 = 500 and
N 2 = 2.000 are adopted, this means that statistically
33.3% of calculations is done by using only N1 samples,
while 66.7% of calculations require N1 + N 2 samples.
Comparison should be made with approach of using fixed
value of N = 2.000 samples for processing. If number of
calculations in two-level procedure is marked with CTL ,
and number of calculations in classical procedure with
fixed-number of samples is given by CC , we have:
CTL 1 N1 2 N1 + N 2 11
=
+
=
,
CC 3 N 3 N
12
References
[1] Dobre, O. A., Abdi, A., Bar-Ness, Y., Su, W.: A
Survey of Automatic Modulation Classification
Techniques: Classical Approach and New Trends,
IET Commun., Vol. 1, No. 2, pp. 137-156, April
2007.
[2] Wu, H.-C., Saquib, M., Yun, Z.: Novel Automatic
Modulation Classification Using Cumulant Features
for Communications via Multipath Channels, IEEE
Trans. Wireless Commun., Vol. 7, No. 8, pp. 30983105, August 2008.
[3] Swami, A., Sadler, B. M.:“Hierarchical Digital
Modulation Classification Using Cumulants, IEEE
Trans. Commun., Vol. 48, pp. 416-429, 2000.
[4] Orlic, V. D., Dukic, M. L.: Automatic Modulation
Classification Algorithm Using Higher-Order
Cumulants Under Real-World Channel Conditions,”
IEEE Commun. Lett., Vol. 13, Issue 12, pp. 917-919,
December 2009.
[5] Orlic, V. D., Dukic, M. L.: Properties of an
Algorithm for Automatic Modulation Classification
Based on Sixth-Order Cumulants, Proc. of ICEST
2009 Conf., Veliko Tarnovo, Bulgaria, 2009.
[6] Carlson, A. B., Crilly, P. B., Rutledge, J. C.:
Communication Systems, 4th edition, McGraw-Hill,
2002.
(14)
meaning that in average 8.33% savings in processing time
(number of calculations) can be expected from using twolevel classification procedure. In case when the same
method with N1 = 500 and N 2 = 4.000 parameters is
compared with classical method using fixed value of
N = 4.000 samples, achieved savings in processing time
rise up to 20.83% in average.
Also, two-level classification can be organized in such a
manner, that samples used within the first stage of
classification are also used within the second stage (if
needed), i.e. N1 samples from QPSK recognition are
included in N 2 samples for QAM signals recognition.
Since the number of calculations in eq. (8) exceeds all
other calculations in algorithm by far, ratio described in
eq. (14) becomes:
CTL 1 N1 2 N 2
≈
+
.
CC 3 N 3 N
(15)
515