CALIFORNIA STATE UNIVERSITY, NORTHRIDGE
QRS CLASSIFICATION USING SUPPORT VECTOR MACHINE
A graduate project submitted in partial fulfillment of the requirements
For the degree of Masters of Science in
Electrical Engineering
By
Triti Kadkhodayan
December 2014
The graduate project of Triti Kadkhodayan is approved:
Prof. Vijay J Bhatt
Date
Ramin Roosta, Ph.D.
Date
Xiyi Hang, Ph.D. Chair.
Date
California State University, Northridge
ii
ACKNOWLEDGEMENT
I would like to express my deepest appreciation to Professor Xiyi Hang for his support
and guidance throughout my graduate project. His continuous support throughout the years
I have been working towards this degree, led me to the right path. I would also like to thank
Professor Ramin Roosta and Professor Vijay Bhatt for their advice. I am grateful to thank
Dr. Ali Amini who took the time to talk to me about my project; I really appreciate his help
and support. Above all, I would like to thank my amazing husband Arash Bakhtmand and
the rest of my family for all of their love, support and encouragement. I could not have
completed this study without the kindness of the faculty and the campus community at
CSUN.
iii
TABLE OF CONTENTS
SIGNATURE PAGE ...................................................................................................................................... II
ACKNOWLEDGEMENT ............................................................................................................................ III
LIST OF FIGURES ........................................................................................................................................ V
ABSTRACT ................................................................................................................................................. VI
CHAPTER 1: INTRODUCTION ................................................................................................................... 1
CHAPTER 2: FEATURE EXTRACTION ..................................................................................................... 6
CHAPTER 3: SUPPORT VECTOR MACHINE............................................................................................ 9
CHAPTER 4: NUMERICAL EXPERIMENTS ........................................................................................... 13
CHAPTER 5: CONCLUSION ...................................................................................................................... 20
BIBLIOGRAPHY ......................................................................................................................................... 21
APPENDIX ................................................................................................................................................... 24
iv
LIST OF FIGURES
Figure 1. A Typical Normal ECG Signal ............................................................................ 2
Figure 2. Normal Sinus Rhythms ........................................................................................ 2
Figure 3. Premature Ventricular Contraction ...................................................................... 3
Figure 4. Atrial Premature Contraction ............................................................................... 4
Figure 5. Left Bundle Branch .............................................................................................. 5
Figure 6. Right Bundle Branch............................................................................................ 6
Figure 7. Analysis Filter Bank ............................................................................................ 7
Figure 8. Four level Filter Bank Decomposition ................................................................. 8
Figure 9. Two Class Classification using SVM with the largest margin .................................. 10
Figure 10. Mapping the data nonlinearly into a higher dimensional feature space via ϕ . 11
v
ABSTRACT
QRS CLASSIFICATION USING SUPPORT VECTOR MACHINE
By
Triti Kadkhodayan
Master of Science in Electrical Engineering
An electrocardiogram (ECG or EKG) is a test that shows the nature of heart conditions by
measuring electrical activity of the heart. This has been used as all effective method to identify
any heart malfunction prior to a cardiac arrest. Machine learning techniques are a good tool for
diagnosis since they are able to observe things that are not seen by naked eyes. Support vector
machine (SVM) is one of the most effective technique for learning from the provided data. Its
classification is vital to saving life. In this project the records were selected from the open source
MIT-BIH arrhythmia and the data is divided into training and testing sets. Norma and abnormal
heartbeats have been properly classified such as left bundle branch block (LBBB), right bundle
branch block (RBBB), atrial premature contractions (APC) and premature ventricular contractions
(PVC).
vi
CHAPTER 1: INTRODUCTION
Electrocardiogram (ECG or EKG) is a way to do measurements of the electrical activity of the
heart. Auto-rhythmic cells in the intrinsic conduction system creates action potential that goes
throughout the heart which makes contraction happen. The action potential goes through five
phases. A stimulus opens a few Na+ channels in the threshold phase. The influx of positive sodium
ions helps to depolarize the cell. The sodium ion channels start to inactivate at the depolarization
phase. The cell loses potassium and chloride ions and sodium influx slows down. When the muscle
cells depolarize, calcium is released from sarcoplasmic reticulum causing muscle cells to contract.
Once the calcium releases, it doesn’t let repolarization happen from happening and plateau phase
happens. Then at a certain point calcium goes back into sarcoplasmic reticulum causing muscle
cells to relax. At this point potassium leaves and causes repolarization to happen. Both sodium and
potassium channels are closed and membrane is in the resting state. This action potential can last
as much as 300 millisecond. The cycle goes on and repeats after a short period [8]. A typical ECG
consists of a P-wave, a QRS complex and a T-wave. In a normal sinus rhythm (N), the sinoatrial
(SA) node creates an electrical impulse which goes through the atrial muscles supplying electrical
changes. This causes atrial depolarization. P wave is the result of this depolarization event. The
impulses travel through atrioventricular (AV) node. As the result the right and left ventricles begin
to fill with blood. The Q waves is seen when the signal arrives at the bundle of His. The signal
leaves the bundle branches via Purkinje fibers. The signal goes across the ventricles causing them
to contract. The QRS complex is the result of ventricle depolarization and as a result blood spreads
throughout the body [2]. The signal that spread throughout the atria is relatively quick at 1 meter
per second but then it slows down at the AV node. The relaxation of electrical changes in the
1
ventricles is the result of ventricles repolarization and are represented by T-wave [4]. A normal
ECG signal is shown in Figure 1.
.
Figure 1: A Typical Normal ECG Signal
All p waves and QRS complexes have the same size and shape and they point in the same
direction. Each QRS has the same distance from the T waves and is the same shape throughout
the ECG.
Figure 2: Normal Sinus Rhythms (Record 100 of MIT-BIH database)
In this project the different patterns from the normal heartbeat pattern can be classified into
following categories: Pre-ventricular contraction (PVC), Atrial premature contraction (APC), Left
bundle branch block (LBBB), and Right bundle branch block (RBBB).
2
Premature ventricular contraction (PVC), results from a not mature contractions of the ventricles.
Sinus P waves appear earlier than expected in PVC also P waves are not uniform with PVC and
QRS complex is wide [3]. An ECG of a patient having PVC is shown in Figure 3.
Figure 3: Premature Ventricular Contraction (Record 233 of MIT-BIH database)
Atrial premature contraction (APC) is a single contraction that occurs sooner than the next
contraction that is supposed to happen. Pattern of heartbeat is not uniform whenever APC occurs.
P Waves can have a different shape. PR Interval normally have different shapes and QRS
interval is Normal. An ECG of patient with APC is shown in Figure 4.
Figure 4: Atrial Premature Contraction (Record 209 of MIT-BIH database)
3
Left bundle branch block (LBBB) is a cardiac abnormality seen on the ECG. In this condition, left
ventricle contraction is put off and the left ventricle contracts after contraction of the right
ventricle. The ST segments and T waves appear in the opposite direction to the main vector of the
QRS. An ECG of a patient having LBBB is shown in Figure 5.
Figure 5: Left Bundle Branch (Record 109 of MIT-BIH database)
Right bundle branch block (RBBB) happens when the QRS shows an extra deflection. A
depolarization of the left ventricle is followed by the slower depolarization of the right ventricle.
An ECG of a patient having RBBB can be seen in Figure 6.
Figure 6: Right Bundle Branch Block (record 118 of MIT-BIH database)
Correct classification of QRS complex helps doctors to see heart problems by seeing its ECG and
analyzing it and comparing it to the ECG of a healthy heart. At this point a holter monitor can play
an important role. A holter monitor is a movable electrocardiograph which monitors different
4
electrical activities of the heart for at least 24 hours. In this chapter the two main components of
heartbeat classification scheme is described. The process of feature extraction is described then
the classification method. Feature extraction is performed with wavelet transform for dataset
reduction. The technique used for classification of the heartbeats is a SVM classifier [7]. SVM
separates different classes from each other base on the margin between them [21]. Multiclass
classification is done with a multi-order polynomial kernel in this project. Chapter 2 presents
feature extraction. Chapter 3 includes SVM classification. In chapter 4 numerical experiment is
presented, and chapter 5 includes conclusion.
5
CHAPTER 2: FEATURE EXTRACTION
Feature extraction is used to cut the time series data and to use the important features that are
necessary. Extraction of the right features not only speeds up the calculation process but also
produces good results. A feature extraction technique for time series is Discrete Wavelet
Transform (DWT). Wavelet transform is a time scale processing method for signals that have
enough scale values and shifting in time [16]. One of the advantages of wavelet transformation is
having window sizes which is broad and narrow at low and high frequencies. This technique
shows signals in different resolutions. The extraction of ECG features in time domain can be
done using feature extraction methods using DWT. This technique extracts important data from
the signal and it can be used as an input for the classifier. DWT provides information for analysis
of the signal by reducing the time for calculation. In DWT based feature extraction scheme,
transformation of signals to time scale by multilevel wavelet decomposition happens first. DWT
decomposes signals into multiple bands by using filters which have different cutoff frequencies
[4]. DWT analyzes the high frequencies and low frequencies by passing the original signal
through a series of high pass and low pass filters respectively [6]. A low-pass filter is a filter that
passes low-frequency signals and lowers the range of signals with frequencies higher than
the cutoff frequency and a high-pass filter passes high-frequency signals but lowers the range of
signals with frequencies lower than the cutoff frequency. The actual amount of attenuation for
each frequency is different. It depends on specific filter design. Sampled signal is shown by x[n]
in equation-1 where n is an integer. Filtering a signal is equivalent to convolution of the signal
with the filter’s impulse response h[n] [5], [8].
∞
(𝑥 ∗ ℎ)[𝑛] = ∑ 𝑥[𝑘]ℎ[𝑛 − 𝑘]
𝑘=−∞
6
(1)
The procedure for discrete wavelet transform is to decompose the signal into approximation and
detail information and analyze the signal at different frequency bands. DWT uses scale and wavelet
functions related to low pass and high pass filters respectively. The original signal x[n] is passed
through a half band low pass filter h[n] and a high pass filter g[n]. Half of the samples are omitted
after filtering. The decomposition begins when the signal is passed through a half band low pass
filter with impulse response h[n]. All the frequencies that are above half of the highest frequencies
in the signal will be filtered by half band low pass filter. The outputs of the low pass and high pass
filters can be express as equations 2 and 3 [16]. The outputs from the high-pass filter give the detail
coefficients and the outputs form the low-pass filter give the approximation coefficients [16]. Low
feature frequency band is used for ECG classification.
∞
𝑦𝑙𝑜𝑤 [𝑘] = ∑ 𝑥[𝑛]ℎ[2𝑘 − 𝑛]
(2)
𝑛=−∞
∞
yℎ𝑖𝑔ℎ [𝑛] = ∑ 𝑥[𝑛]𝑔[2𝑘 − 𝑛]
(3)
𝑛=−∞
The filtering process at fundamental level can be showed as bellow.
G[n]
↓2
Detail Coeficients
H[n]
↓2
Approximation
Coeficients
Figure 7: Analysis Filter Bank
The filtering can be repeated for more decomposition levels. Each level of the decomposition
halves the number of samples and halves the frequency band from the signal. Moreover the entire
7
signal divides into two parts, half of samples for detail and the rest for approximation coefficients.
The time resolution halves due to the decomposition. But, the frequency resolution is doubled
since the uncertainty in the frequency is reduced by half. Figure 8 shows this procedure where x[n]
is the original signal to be decomposed.
G[n]
↓2
1st level DWT coefficinet
G[n]
H[n]
↓2
2nd level DWT coefficinet
↓2
H[n]
G[n]
↓2
H[n]
H0
↓2
3rd level DWT coefficinet
G[n]
4th level DWT coefficinet
H[n]
↓2
Figure 8: Four level filter bank decomposition
8
CHAPTER 3: SUPPORT VECTOR MACHINE
The support vector machine (SVM) is a technique for data classification. The idea is to use a linear
separating hyperplane which can maximize the distance between two classes. The linear SVM, can
be applied to those problems that can be separated linearly [11]. The SVM tries to find the separating
hyperplane with the largest margin between two classes. SVM classifiers are based on the class of
hyperplanes [10].
(w. x) + b = 0 wϵℛ N , bϵR
(4)
which w is weight vector and b is threshold. Corresponding to decision functions
f (x) = sign((w. x) + b)
(5)
In mathematics, the sign function is an odd function that extracts the sign of a real number. In
expressions the sign function is often represented as bellow.
−1 if x < 0
sign(x) ≔ { 0 if x = 0
1 if x > 0
(6)
The optimal hyperplane is orthogonal to the shortest line which connects the dot convex hulls of
the two classes and intersects it in half [17]. It can be constructed by solving a constrained
quadratic optimization problem which solution 𝑤 has an expansion 𝑤 = ∑𝑖 𝑣𝑖 𝑥𝑖 in terms of
training data that lie on the margin (Figure 1). There are two types of margins which are soft
margin and hard margin. Soft margin has the same margin maximizing as the hard margin
although soft margin has some constraints and bounded by C. Maximizing soft margin doesn’t
show the influence of any point on the decision boundary [11], [17]. Decreasing cost causes
classifier to loose linear separation. Two classes could be fully separated by (𝑤. 𝑥) + 𝑏 = 0.
9
Figure 9: Two Class Classification using SVM with the largest margin
The data in the class one, must locate in one side of (𝑤. 𝑥) + 𝑏 = 0, while data in the other class
must locate on another side. In two class classification, the distance between two classes of training
data should be large [14]. A line with parameters w and b can be found such that, the points closest
to the hyperplane satisfy |(𝑤. 𝑥𝑖 ) + 𝑏| = 1 and distance between (𝑤. 𝑥𝑖 ) + 𝑏 = ±1 is maximized.
The distance between |(𝑤. 𝑥𝑖 ) + 𝑏| = 1 is 2⁄‖𝑤‖ and maximizing 2⁄‖𝑤‖ is equal to
minimizing
‖𝑤‖⁄
2 . A form (𝑤, 𝑏) of the hyperplane with 𝑦𝑖 ((𝑤. 𝑥𝑖 ) + 𝑏) ≥ 1 can be obtained
[13]. The key to solve non-linear problems is to map the training data points into a higher dimension
and transform them into feature space. Then the linear separation can be done by applying a
transform function φ to sets of data points and construct a separating hyperplane with the maximum
margin.
10
Figure 10: Mapping the data nonlinearly into a higher dimensional feature space via 𝜙
This makes a nonlinear decision boundary in input space. The amount of calculation increases by
increasing the dimension space. In this case the Kernel Trick plays an important role. By using
this trick, calculating the separating hyperplane is possible. In Kernel Trick, the scalar product can
be calculated in low-dimensional space without transforming to the high dimensional feature space
[20], [21]. Kernel Trick is helpful since it makes it easy to calculate. Other kernel such as Gaussian
kernel also plays an important rule since it transforms the data points into an infinite dimensional
feature space. Both the quadratic programming problem and the final decision function 𝑓(𝑥) =
𝑠𝑖𝑔𝑛(∑𝑖 𝑣𝑖 . 𝑘(𝑥, 𝑥𝑖 ) + 𝑏) depend only on dot products between data points. This is what permits
to use the nonlinear case [17]. The scalar product between 𝑥1 and x2 can be expressed as (x1. x2)
and kernels can be used to calculate the scalar product. The property of the kernel can be expressed
as bellow:
𝑘(𝑥, 𝑦) ∶= (𝜑(𝑥). 𝜑 (𝑦))
(7)
For constructing a nonlinear classifiers, a 𝜑(𝑥𝑖 ) is used for each training example 𝒙 𝒊 and the
hyperplane algorithm is performed in F. Constraints premise training data not be on the correct
side of the separating hyperplane and minimize the training error in the function. Because kernels
are used, the result is nonlinear decision function of the form 𝑓(𝑥) = 𝑠𝑖𝑔𝑛(∑𝑖 𝑣𝑖 . 𝑘 (𝑥. 𝑥𝑖 ) + 𝑏).
11
In this equation both problem and the final decision function rely on dot products between data
points. This function will classify the inputs base on the limits which were assigned. This method
is helpful because it can be simplify to a linear function as it was discussed [23], [24].
The multi-category support vector machine is used which extends the binary SVM to the multiclass case. For multi-class case, two general strategies are used. The used methods are one-againstthe rest method using winner-takes-all strategy and one-against-one method implemented by maxwins voting [30], [32], [33]. In first case there are “n” classifiers and the class resulted is obtained
by majority vote of all classifiers.
LibSVM uses the one-against-one approach for multi-class problems. The one-against-one simply
has a shorter training time [31], [34]. In one-against-the rest method, N different binary classifiers
are used for training. Each one trained to distinct the examples in a single class from the examples
in all remaining classes. When the N classifiers are run, the classifier which outputs the largest
data is chosen. In one-against-one, every classifier gives the example to one of the two classes in
classification. Then one vote goes to that class. The votes increase over time. The class with the
more votes wins and determine the moment’s classification. As the result designing an optimal
multi-category SVM that continues to deliver the efficiency of the binary SVM is the motive [34].
12
CHAPTER 4: NUMERICAL EXPERIMENTS
Recorded physiologic signals (PhysioBank) and open-source software such as PhysioToolkit can
be obtained from PhysioNet website. The ECG files obtained from PhysioNet.org. From Twenty
patients of ages 24 to 87 years old, twenty records of one hour long were downloaded from the
website [1]. Each heart beat is marked and given a letter by annotation files and assigned a label
on the R peak. The Simulation environment with its different Matlab design is for faster growth
since allows different algorithms to be trained and tested. In this project, Matlab is the simulation
environment that is used. Feature extraction is done using Matlab software for calculation.
Implementation of SVM for QRS classification in the ECG is done by using LIBSVM. LIBSVM
is an integrated software package for SVM classification [2]. Since each record is a continuous
waveform it is necessary to extract only a single heartbeat [9], [11]. It is important to relate the
categories with individual extracted beats since the database have annotations for each heartbeat.
A total of 2970 beats were extracted with 594 beats belonging to each category such as Normal,
PVC, APC, RBBB and LBBBB. The training labels are assigned and SVM classification was
created using LIBSVM package. The test file is created in the same sort as the training file but
with 1558 samples. As mentioned in chapter 3, there are different functions φ to map data to higher
dimensions. Several types of kernels is used. The best kernel obtained during training is used to
test the datasets. In this project, we study linear, 2nd, 3rd and 4th order polynomial. 10-fold cross
validation is used to find the best values and the cost of different types of kernel is selected from
10-5 to 10+5. The samples in the ECG signal were reduced from 594 to 300, 153, 79, 41, 23, 14 and
10 in a seven level reconstruction. The first level reconstruction looks like the actual signal but has
half number of samples. 2nd level has half number of samples that of 1st level and 3rd level has
half number of samples than the 2nd level. This reduction in sampling size enables faster analysis.
13
The major advantage in using feature extraction is reducing the number of samples after the first
classification obtained [2].
Mixture of same class beats from different ECG datasets forms the training and testing for each
class and they are called positive examples. Mixture of identical number of beats from rest of all
other classes forms the negative examples [13].
Normal
RBBB
APC
PVC
LBBB
True Positive
False Negative
False Positive
386
59
0
543
3
57
176
2
13
81
4
2
297
7
3
Table 1: Table of Positive and Negative Example
True Negative
1113
955
1367
1471
1251
In this project, different rhythm of heartbeat such as normal (N), right bundle branch block
(RBBB), left bundle branch block (LBBB), premature ventricular contraction (PVC) and atrial
premature contraction (APC) are studied.
Class Label
2
4
5
6
7
Type of Beat
Normal
RBBB
APC
PVC
LBBB
MIT/BIH Data file
100,101,103,113,115,123
118,124,212,231
209,220,223,232
208,233
109,111,207,214
# of Beats/Data file
800
800
300,94,72,1200
960,233
800
Table 2: ECG Samples Used for Training and Testing
For SVM training, few parameters such as the penalty parameter C must be chosen. Choosing
optimal parameters for support vector machines is an important step in SVM. The technique used
to determine the performance of a selected parameter is n-fold cross validation. The training data
can be separated into different folds using cross validation. One fold can be used as the training
set and another fold can be used for testing set. Then the next fold is used for testing and the other
14
folds are used for training and this continues. The parameter C is changed from10-5 to 105. A cross
validation is performed for each C value and the C value with the highest value is approved [10],
[13], [18]. This parameter value is used to train the training data and the SVM is applied for
classification of test data. In this project the best value was calculated using a 10-fold cross
validation. The gamma in the kernel was also used with 10-fold cross validation. The performance
depends on the data selection and the variety of the ECG included in the training set and the quality
of the ECG signal [2]. Kernel function is set up for linear classification by setting the kernel flag
options with “-t 0”. By varying parameter t, the kernel type can be changed. “-t 1” is for polynomial
kernel and “-t 2” is for Gaussian type kernel. The results are shown in Table 3.
C
Linear
2nd
order
3rd
order
4th
order
10-5
10-4
10-3
10-2
10-1
100
10
102
103
104
105
94.42% 95.45% 87.68% 91.27% 88.32% 69.51% 98.77% 88.77% 87.72% 95.1% 95.1%
94.99% 95.1% 73.36% 94.99% 81.39% 94.99% 97.41% 90.69% 95.45% 95.1% 90.69%
95.1%
94.99% 81.39% 73.36% 74.27% 94.99% 97.99% 84.41% 84.09% 95.1% 95.1%
90.49% 92.1%
95.19% 94.99% 84.02% 94.99% 94.02% 84.09% 94.42% 95.1% 84.09%
Table 3: Accuracy on testing datasets without using feature extraction
Testing of the SVM for ECG classification is the next step. The sensitivity, specificity and
accuracy can be obtained by using the following formula.
Sensitivity is calculated in (17);
Sensitivity= True positive/ (True positive+ False Negative)
(17)
Specificity is defined in (18);
Specificity= True Negative/ ((True Negative+ False Positive)
(18)
Accuracy is defined in (19);
Accuracy = (True Negative+ True Positive)/ (True Positive+ False Negative+ True Negative+
False Positive)
(19)
15
The Accuracy for each class is written in Table 4.
Normal
Rest of the Class
Sensitivity=86.74%
Normal
386
0
Specificity=100%
Rest of the Class
59
1113
Accuracy= 96.21%
RBBB
Rest of the Class
Sensitivity=99.45%
RBBB
543
57
Specificity=94.37%
Rest of the Class
3
955
Accuracy= 96.14%
APC
Rest of the Class
Sensitivity=98.88%
APC
176
13
Specificity=99.06%
Rest of the Class
2
1367
Accuracy= 99.04%
PVC
Rest of the Class
Sensitivity=95.29%
PVC
81
2
Specificity=99.86%
Rest of the Class
4
1471
Accuracy= 99.61%
LBBB
Rest of the Class
Sensitivity= 97.70%
LBBB
297
3
Specificity= 99.76%
Table 4: Experimental Result
Rest of the Class
7
1251
Accuracy= 99.36%
The results show that the accuracy is high and SVM could classify the data. The performance of
the SVM is determined by running different type of kernel, including higher order dimension and
examined to achieve the best classification accuracy. Compressing and reducing the sample size
by feature extraction to classify each signal is shown in Table 5 [14]. It can see that the accuracy
changes in linear classification by varying the cost. On the fourth level changing the cost to 10 -4
gives an accuracy of 93.12% and 10-5 gives an accuracy of 89.23%. The accuracy drops when the
level number increases. The accuracy of level 5 is 92.85% with 23 samples and of level 6 is 91.77%
16
with only 14 samples [14]. By reducing the samples from 41 to 23 the accuracy drops and altering
the cost changes the accuracy [22], [24]. The best result in feature extraction can be seen in the 3rd
order polynomial at level 5 and the worst result is the 4th order polynomial at level 6 and 7.
17
Linear
2nd order polynomial
3rd order polynomial
4th order polynomial
Cost
decomp
ose
10-4
Level 1
10-3
10-2
10-1
100
10
102
103
95.4%
94.39%
94.10%
94.11%
88.4%
88.42%
80.42%
91.7%
Level 2
90.75%
94.33%
89.10%
89.63%
33.21% 92.85%
80.42%
80.8%
Level 3
94.33%
93.70%
92.71%
89.12%
66.62% 92.57%
80.42%
80.4%
Level 4
93.12%
93.11%
94.33%
88.3%
94.33%
94.11%
78.11% 78.83%
Level 5
94.41%
92.85%
93.16%
91.32%
90.75%
96.42%
91.53%
91.92%
Level 6
92.57%
91.77%
90.56%
91.09%
90.83%
92.6%
91.62%
91.41%
Level 7
91.07%
91.50%
92.6%
90.12%
90.64%
90.12%
90.02%
90.11%
Level 1
95.61%
95.11%
95.92%
81.54%
91.13%
95.16%
84.11%
94.8%
Level 2
91.53%
95.1%
84.4%
95.03%
82.19%
95.44%
95.4%
95.4%
Level 3
94.99%
91.9%
95.6%
95.72%
95.50%
91.12%
76.5%
84.9%
Level 4
92.16%
81.2%
84.1%
84.71%
95.55%
95.57%
76.7%
89.9%
Level 5
91.56%
91.15%
92.16%
91.87%
90.79%
92.55%
91.81%
91.45%
Level 6
91.47%
90.22%
91.46%
91.69%
90.33%
92.16%
92.20%
91.65%
Level 7
90.37%
90.51%
94.6%
91.12%
89.94%
94.99%
91.22%
91.81%
Level 1
94.99%
60.33%
84.02%
61.11%
93.96%
84.79%
91.21%
94.99%
Level 2
94.99%
60.33%
84.02%
61.11%
93.96%
94.99%
91.21%
94.99%
Level 3
94.99%
90.1%
84.02%
84.02%
81.02%
91.71%
84.79%
94.99%
Level 4
61.4%
71.20%
61.8%
77.82%
77.03%
91.20%
69.01%
61.4%
Level 5
96.21%
95.99%
95.12%
95.83%
96.37%
95.51%
96.87%
96.50%
Level 6
90.7%
91.25%
92.67%
91.39%
89.63%
90.19%
90.52%
91.61%
Level 7
89.77%
91.11%
92.64%
91.11%
88.69%
89.32%
91.20%
90.18%
Level 1
94.99%
92.19%
93.90%
84.02%
84.02%
94.99%
65.28%
94.99%
Level 2
93.39%
95.02%
94.99%
84.1%
30.61%
93.56%
62.58%
94.99%
Level 3
93.95%
60.33%
84.02%
61.11%
93.96%
84.79%
91.21%
94.99%
Level 4
91.20%
91.20%
91.20%
91.20%
91.20%
91.20%
91.21%
91.20%
Level 5
92.06%
92.11%
92.18%
91.34%
91.99%
92.35%
92.84%
92.41%
Level 6
87.71%
88.21%
84.66%
89.69%
89.33%
87.39%
84.23%
84.62%
Level 7
88.47%
90.10%
88.61%
89.11%
87.95%
89.2%
85.21%
86.82%
Table 5: Feature Extraction on linear, 2nd order, 3rd order and 4th order kernel
18
According to the above discussion, classification performance is compared with and without DWT
processing. Comparing the results show that using DWT prior to classification can lead to a higher
classification accuracy [10], [18]. Then the best feature extraction model is selected from Table 5
from different polynomial orders. Table 6 shows the best model with its corresponding cost and
level number. Testing Accuracy on each of these model is obtained and is written in Table 7.
Polynomial order
Level number
Cost
Best Feature Extraction Accuracy
Linear
5
10
96.42%
2nd order polynomial
4
10
95.57%
rd
5
10-4
96.21%
th
4 order polynomial
2
Polynomial order
Level number
Cost
Testing Accuracy
3 order polynomial
95.02%
10-3
Table 6: Best Feature extraction Model for linear, 2nd, 3rd, and 4th order kernel
Linear
5
10
91.42%
nd
2 order polynomial
4
10
93.57%
rd
3 order polynomial
5
10-4
98.24%
4th order polynomial
2
92.57%
10-3
Table 7: Testing Accuracy on the Best Feature extraction for linear, 2nd, 3rd, and 4th order kernel
The use of a Gaussian or polynomial kernel leads to a higher accuracy rate in classifier
performance when compared to the linear kernel. Accuracy is reduced for a large degree
polynomial and small width Gaussian kernel [15], [17].
19
CHAPTER 5: CONCLUSION
Many studies have shown that heart diseases can be linked to mortality. Appropriate classification
using SVM is important to save many lives. In this project the records were selected from the open
source MIT-BIH arrhythmia database for analysis and the data is divided into training and testing
sets. The effect of SVM classification on ECG signals with and without using feature extraction
can be seen. The samples in the ECG signal were reduced from 594 to 300, 153, 79, 41, 23, 14 and
10 through feature extraction. The software that is used for classification is Matlab and LIBSVM
software for analyzing each heartbeat. After labeling the heartbeats, training and classifying is
done and the accuracy rate is obtained. For better classification, different kernels are used.
The SVM used for classification of the ECG beat was trained, cross validated and tested with the
extracted features from discrete wavelet transform of the ECG signals. The performance of ECG
classification depends on the features extraction from the ECG data and the design of the classifier.
Computer simulations showed that our approach gives the excellent performances of successful
recognition.
20
BIBLIOGRAPHY
[1] Kadambe, S.; Murray, R.; Boudreaux-Bartels, G.F., "Wavelet transform-based QRS complex
detector," Biomedical Engineering, IEEE Transactions on , vol.46, no.7, pp.838,848, July 1999
[2] Rioul, O.; Vetterli, M., "Wavelets and signal processing," Signal Processing Magazine,
IEEE , vol.8, no.4, pp.14,38, Oct. 1991
[3] Millet-Roig, J.; Lopez-Soriano, J.J.; Mocholf, A.; Ruiz-Granell, R.; Chorro, F. J., "Study of
frequency and time domain parameters extracted by means of wavelet transform applied to ECG
to distinguish between VF and other arrhythmias," Computers in Cardiology 1998 , vol., no.,
pp.17,20, 13-16 Sep 1998
[4] Spaargaren, A.; English, M. J., "Analysis of the signal averaged ECG in the time-frequency
domain," Computers in Cardiology, 1999 , vol., no., pp.571,574, 1999
[5] Sierra, G.; Morel, P.; Le Guyader, P.; Trellez, F.; Nadeau, R.; Savard, Pierre, "Frequency
analysis of the signal-averaged ECG of postinfarction patients for prediction of cardiac
death," Engineering in Medicine and Biology Society, 1997. Proceedings of the 19th Annual
International Conference of the IEEE , vol.1, no., pp.76,77 vol.1, 30 Oct-2 Nov 1997
[6] C. Valens, "A Really Friendly Guide to Wavelets," vol. 2003, 1999.
[7] Qian Zheng; Chao Chen; Zhinan Li; Anpeng Huang; Bingli Jiao; Xiaohui Duan; Linzhen
Xie, "A novel multi-resolution SVM (MR-SVM) algorithm to detect ECG signal anomaly in
WE-CARE project," Biosignals and Biorobotics Conference (BRC), 2013 ISSNIP , vol., no.,
pp.1,6, 18-20 Feb. 2013
[8] Prasad, G.K.; Sahambi, J. S., "Classification of ECG arrhythmias using multi-resolution
analysis and neural networks," TENCON 2003. Conference on Convergent Technologies for the
Asia-Pacific Region , vol.1, no., pp.227,231 Vol.1, 15-17 Oct. 2003
[9] Castro, B.; Kogan, D.; Geva, A.B., "ECG feature extraction using optimal mother
wavelet," Electrical and ELectronic Engineers in Israel, 2000. The 21st IEEE Convention of
the , vol., no., pp.346,350, 2000
[10] Matsuyama, A.; Jonkman, M., "The Application of Wavelet and Feature Vectors to ECG
Signals," TENCON 2005 2005 IEEE Region 10 , vol., no., pp.1,4, 21-24 Nov. 2005
[11] Feng, C.; Liang, W.; Zhang, X.L. Feature extraction method for ECG signal based on HMM
in body sensor network. Inf. Control 2012, 10, 628–635.
[12] Szildgyi, S.M.; Szildgyi, L.; David, L., "Comparison Between Neural-Network-Based
Adaptive Filtering and Wavelet Transform for ECG Characteristic Points
detection," Engineering in Medicine and Biology Society, 1997. Proceedings of the 19th Annual
International Conference of the IEEE , vol.1, no., pp.272,274, Oct. 30 1997-Nov. 2 1997
21
[13] Stamkopoulos, T.; Diamantaras, K.; Maglaveras, N.; Strintzis, M., "ECG analysis using
nonlinear PCA neural networks for ischemia detection," Signal Processing, IEEE Transactions
on , vol.46, no.11, pp.3058,3067, Nov 1998
[14] Chih-Wei Hsu; Chih-Jen Lin, "A comparison of methods for multiclass support vector
machines," Neural Networks, IEEE Transactions on , vol.13, no.2, pp.415,425, Mar 2002
[15] http://www.mathworks.com/videos/introduction-to-wavelet-toolbox82002.html?form_seq=conf504&confirmation_page&wfsid=5030772 retrieve date,Oct. 2013
[16] Mousa, A.; Yimaz, A., "Neural network detection of ventricular late potentials in ECG
signals using wavelet transform extracted parameters," Engineering in Medicine and Biology
Society, 2001. Proceedings of the 23rd Annual International Conference of the IEEE , vol.2, no.,
pp.1668,1671 vol.2, 2001
[17] Chapelle, O.; Haffner, P.; Vapnik, V.N., "Support vector machines for histogram-based
image classification," Neural Networks, IEEE Transactions on , vol.10, no.5, pp.1055,1064, Sep
1999
[18] Conforto, S.; Laudani, A.; Oliva, F.; Fulginei, F.R.; Schmid, M., "Classification of ECG
patterns for diagnostic purposes by means of Neural Networks and Support Vector
Machines," Telecommunications and Signal Processing (TSP), 2013 36th International
Conference on , vol., no., pp.591,595, 2-4 July 2013
[19] Kher, R.; Vala, D.; Pawar, T.; Thakar, V.K., "Implementation of derivative based QRS
complex detection methods," Biomedical Engineering and Informatics (BMEI), 2010 3rd
International Conference on , vol.3, no., pp.927,931, 16-18 Oct. 2010
[20] Faziludeen, S.; Sabiq, P.V., "ECG beat classification using wavelets and
SVM," Information & Communication Technologies (ICT), 2013 IEEE Conference on , vol., no.,
pp.815,818, 11-12 April 2013
[21] Osowski, Stainslaw; Linh Tran Hoai; Markiewicz, T., "Support vector machine-based
expert system for reliable heartbeat recognition," Biomedical Engineering, IEEE Transactions
on , vol.51, no.4, pp.582,589, April 2004
[22] "PhysioBank," vol. 2004: Physionet.
[23] Mi Shen; Liping Wang; Kanjie Zhu; Jiangchao Zhu, "Multi-lead ECG classification based
on Independent Component Analysis and Support Vector Machine," Biomedical Engineering
and Informatics (BMEI), 2010 3rd International Conference on , vol.3, no., pp.960,964, 16-18
Oct. 2010
[24] Martis, R.J.; Acharya, U.R.; Ray, A.K.; Chakraborty, C., "Application of higher order
cumulants to ECG signals for the cardiac health diagnosis," Engineering in Medicine and
Biology Society,EMBC, 2011 Annual International Conference of the IEEE , vol., no.,
pp.1697,1700, Aug. 30 2011-Sept. 3 2011
22
[25] M. Misiti, Y. Misiti, G. Oppenheim, and J.-M. Poggi, Wavelet Toolbox User's Guide:
Mathwork, 2000.
[26] Cuiwei Li; Chongxun Zheng; Changfeng Tai, "Detection of ECG characteristic points using
wavelet transforms," Biomedical Engineering, IEEE Transactions on , vol.42, no.1, pp.21,28,
Jan. 1995
[27] F. Schlegelmilch, M. Helbig, V. Natchkova, G. Ivanova, K. Schellhorn, V. Detschew, D.
Mandler, and G. GrieBbach, "A Multi-Channel Approach for Online-Identification of QRSRegions Using Adaptive Recursive Estimators," presented at EMBEC 02, Vienna, 2002.
[28] http://www.webmd.com/heart-disease/electrocardiogram/ retrieve date, Nov. 2013
[29] http://www.nhlbi.nih.gov/health/health-topics/topics/ekg/ retrieve date, Dec. 2013
[30] Yoonkyung LEE, Yi LIN, and Grace WAHBA, “Multicategory Support Vector Machines:
Theory and Application to the Classification of Microarray Data and Satellite Radiance Data”
[31] Zhihua Zhang Bayesian, “Multicategory Support Vector Machines”
[32] GLENN M. FUNG, Computer-Aided Diagnosis & Therapy Solutions, Siemens Medical
Solutions, Inc, 51 Valley Stream Parkway,Malvern, PA 19355, “Multicategory Proximal Support
Vector Machine Classifiers”
[33] Yufeng Liu, “Fisher Consistency of Multicategory Support Vector Machines”
[34] G. von Wagner1, U. Kunzmann2, J. Schöchlin1, A. Bolzu, “SIMULATION METHODS FOR
THE ONLINE EXTRACTION OF ECG PARAMETERS UNDER MATLAB®/SIMULINK®”
[35] Robert Jenssen, Marius Kloft, Alexander Zien, Sören Sonnenburg,Klaus-Robert Müller, “A
scatter-based prototype framework and multi-class extension of support vector machines”
[36] Yann Guermeur, “A generic model of multi-class support vector machine”
[37] Zeng Fanzi, Ma Xiaolong, “The generalization error bound for the multiclass analytical
center classifier”
[38] Moody, G.; Mark, R. Development and evaluation of a 2-lead ECG analysis program. In
Proceedings of the Computers in Cardiology, Seattle, WA, USA, 18–21 September 1982; pp.
39–44.
[39] Nasiri, J.A.; Naghibzadeh, M.; Yazdi, H.S.; Naghibzadeh, B., "ECG Arrhythmia
Classification with Support Vector Machines and Genetic Algorithm," Computer Modeling and
Simulation, 2009. EMS '09. Third UKSim European Symposium on , vol., no., pp.187,192, 25-27
Nov. 2009
23
Appendix
Matlab code:
% This code just simply run the SVM on the example data set "heart scale",
% which is scaled properly. The code divides the data into 2 parts
% train and test
% Then plot the results vs their true class.
>> train = [normal(1:620,:);rbbb(1:900,:);apc(1:300,:);pvc(1:450,:); lbbb(1:700,:)];
>> labels = [2*ones(620,1); 4*ones(900,1); 5*ones(300,1);6*ones(450,1); 7*ones(700,1)];
>> test = [normal(621:1006,:); rbbb(901:1500,:); apc(300:488,:); pvc(451:533,:);
lbbb(701:1000,:)];
>> tlabels = [ 2*ones(386,1); 4*ones(600,1); 5*ones(189,1); 6*ones(83,1);7*ones(300,1)];
>> model = svmtrain(labels, train, '-t 1 -d 4 -c .001 ');
%% First Level
ca1 = mdwtdec('r',train,7,'sym4');
ca1t = mdwtdec('r',test,7,'sym4');
train2 = mdwtrec(ca1,'ca',1);
test2 = mdwtrec(ca1t,'ca',1);
model2 = svmtrain(labels, train2, '-t 1 -d 4 -c .001');
[predicted_label, accuracy, dec] = svmpredict(tlabels, test2 , model2);
%% Second Level
train3 = mdwtrec(ca1,'ca',2);
test3 = mdwtrec(ca1t,'ca',2);
model3 = svmtrain(labels, train3, '-t 1 -d 4 -c 1000 ');
[predicted_label, accuracy, dec] = svmpredict(tlabels, test3 , model3);
%% Third Level
train4 = mdwtrec(ca1,'ca',3);
test4 = mdwtrec(ca1t,'ca',3);
model4 = svmtrain(labels, train4, '-t 1 -d 4 -c .001 ');
[predicted_label, accuracy, dec] = svmpredict(tlabels, test4 , model4);
%% Fourth Level
train5 = mdwtrec(ca1,'ca',4);
test5 = mdwtrec(ca1t,'ca',4);
model5 = svmtrain(labels, train5, '-t 1 -d 4 -c 1000 ');
[predicted_label, accuracy, dec] = svmpredict(tlabels, test5 , model5);
24