Pattern Recognition 34 (2001) 2515}2522
Slant estimation algorithm for OCR systems
E. Kavallieratou*, N. Fakotakis, G. Kokkinakis
Wire Communications Laboratory, Electrical Computer Engineering, University of Patras, 26500 Patras, Greece
Received 13 December 1999; accepted 10 October 2000
Abstract
A slant removal algorithm is presented based on the use of the vertical projection pro"le of word images and the
Wigner}Ville distribution. The slant correction does not a!ect the connectivity of the word and the resulting words are
natural. The evaluation of our algorithm was equally made by subjective and objective means. The algorithm has been
tested in English and Modern Greek samples of more than 500 writers, taken from the databases IAM-DB and GRUHD.
The extracted results are natural, and almost always improved with respect to the original image, even in the case of
variant-slanted writing. The performance of an existed character recognition system showed an increase of up to 9% for
the same data, while the training time cost was signi"cantly reduced. Due to its simplicity, this algorithm can be easily
incorporated into any optical character recognition system. 2001 Pattern Recognition Society. Published by Elsevier
Science Ltd. All rights reserved.
Keywords: Slant estimation; Script writing; Wigner}Ville distribution; Character recognition
1. Introduction
Handwritten text is usually characterized by slanted
characters. In particular, the slanted characters slope
either from right to left or vice versa. Moreover, di!erent
deviations may appear not only within a text but also
within a single word. Some examples illustrating these
cases are shown in Fig. 1. Slanted characters constitute
a common feature of any natural language with a Latinstyle alphabet (e.g., English, Modern Greek, etc.). For
example, the percentage of slanted writing in IAM-DB
[1] database of English reaches 77% while the corresponding percentage in GRUHD, a database of Modern
Greek (1000 writers are included) [2], approaches 59%.
The above rates were provided by manually counting the
forms with apparent slanted writing either to the left or
to the right, according to the human judge. Furthermore,
slanted characters appear in both hand-printed and cursive writing.
* Corresponding author. Tel.: #30-61-991722; fax: #30-61991855.
E-mail address: [email protected] (E. Kavallieratou).
Consequently, a robust optical character recognition
(OCR) system has to be able to cope with slanted characters. Watanabe [3] conducted comparative experiments
showing that slant normalization minimizes the error of
recognition. To a further extent, the two most important
problems that may arise from the existence of slanted
characters, in regard to an OCR system, are the following:
E The application to slanted words of a character segmentation procedure, if such is required, that produces
vertical segment boundaries (e.g., based on histograms),
could result in defectively segmented characters as well
as in noisy segments.
E Both the computational cost of the training procedure
and accuracy of the recognition stage would be a!ected in a negative way. Indeed, concerning a single
character, the amount of the training data required for
covering as many slant angles as possible is substantial. Moreover, the classi"cation of a given character
into the correct class is much harder since the set of
possible classes is now bigger.
Therefore, the majority of recent OCR systems contain
a preprocessing stage dealing with slant correction [4}7].
0031-3203/01/$20.00 2001 Pattern Recognition Society. Published by Elsevier Science Ltd. All rights reserved.
PII: S 0 0 3 1 - 3 2 0 3 ( 0 0 ) 0 0 1 5 3 - 9
2516
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
Fig. 1. Examples of slanted word.: (a) a word slanted to right
(b) a word slanted to left (c) a variant-slanted word.
This stage is usually located before the segmentation
module, if it exists, or just before the recognition stage
otherwise.
The most commonly used method for slant estimation
is the calculation of the average angle of near-vertical
strokes [4}6]. This approach requires the detection of the
edges of the characters and its accuracy depends on the
particular characters included in the word. Shridar [7]
presents two more methods concerning slant estimation
and correction. In the "rst one, the vertical projection
pro"le is used while the second one involves making use
of the chain code of entire border pixels.
Vinciarelli [8] presents a technique based on a function S() that provides a measure of the slant absence
across the word. The calculation of such function relies
on the vertical density histogram that is easier to be
obtained than the direction of the strokes. For each angle
in an interval [!153, 153], a shear transform t is
?
applied and the following histogram is calculated:
h (m)
H (m)" ?
, 0)m(nCol,
?
y(m)
where h (m) is the value of the vertical density histogram
?
of the image shear transformed by the angle . y(m) is
the di!erence between the maximum and minimum y
coordinates of the foreground pixels in the m column and
nCol is the number of columns in the image. The number
of foreground pixels of each column is divided by the
distance between the highest and the lowest pixel giving
H (m)"1, if the column contains a continuous stroke,
?
and H H (m)3[0,1] otherwise.
?
Then, the maximum quantity is computed:
h (i).
?
G&? G
S()"
The value for which S() is maximum, is assumed as
slant estimate and the corresponding shear transform t ,
?Y
when applied on the desloped original image, gives the
deslanted image. Although time consuming in the shearing transformation procedure, the above method proved
to decrease the error rate by 12% compared to that
proposed by Bozinovic [9].
However, the evaluation of the slant correction approaches is di$cult since the slants may vary even within
a single word. Additionally, in the relevant literature a
slant correction procedure is rarely evaluated separately,
so that comparative results cannot be given.
In this paper a new method for slant estimation is
presented based on a combination of the projection pro"le technique and the Wigner}Ville distribution. Moreover, it uses a simple and fast shearing transformation
technique. Our approach is character independent and
can easily be adapted in order to satisfy the requirements
of any OCR system. Its performance is measured in
relation to the improvement of the results of a character
recognition system.
In the next section the Wigner}Ville distribution
(WVD) is brie#y presented. The algorithm is described in
Section 3. Finally, some experimental results are given in
Section 4 and the conclusions drawn by this study are
included in Section 5.
2. Wigner}Ville distribution
An important chapter of signal processing is the nonstationary signals, that is, signals whose characteristics
vary with time contrary to stationary signals that are
time-independent. Therefore, in order to succeed a better
representation of non-stationary signals joint timefrequency distributions are used.
A "rst class of time-frequency representations is the
atomic decomposition or linear time}frequency representations. These distributions decompose the signal on the basis
of elementary signals (i.e. the atoms) which have to be well
localized in both time and frequency. However, there is
a trade-o! between time and frequency resolutions, as the
decomposition is succeeded by windowing the signal.
A good time resolution requires a short window; on the
other hand, a good frequency resolution requires a long
window. This is a consequence of the time}frequency resolution relation via Heisenberg} Gabor inequality [10].
In contrast, the energy distributions, another class
of time}frequency representations, distribute the energy
of the signal over two description variables: time and
frequency. The starting point is that since the energy of
a signal x can be deduced from the squared modulus of
either a signal or its Fourier transform,
E "
V
>
>
x(t) dt"
X( f ) df,
\
\
(1)
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
we can interpret x(t) and X( f ) as energy densities,
respectively, in time and in frequency. It is, then natural,
to look for a joint time and frequency energy density
(t, f ), such that
V
E "
V
> >
(t, f ) dt df,
V
\ \
(2)
which is an intermediary situation between those described by Eq. (1). As the energy is a quadratic function of
the signal, the time}frequency energy distributions will
be, in general, quadratic representations.
A very well-known representative of the energy distributions and member of the Cohen's class [11], is the
Wigner}Ville distribution:
=(t, f )"
>
z(t#/2)z(t!/2)e\DO d,
\
>
Z( f#u/2)Z( f!u/2)e
SR du.
\
1. The word image is arti"cially slanted to both left and
right for di!erent slant angles. The maximum slant
angle is 453 approximately and the slant angle step
depends on the height of the word image, as described
below.
2. For each of the extracted word images, the vertical
histogram is calculated.
3. The WVD is calculated for all the above histograms.
4. The curves of maximum intensity of the WVDs are
extracted.
5. The curve of maximum intensity with the greatest
peak, corresponding to the histogram with the most
intense alternations, is selected. In Fig. 3 several
curves of maximum intensity are shown.
6. The corresponding word image is selected as the most
non-slanted word.
(3)
where z(t) represents the analytical signal associated with
the signal s(t). The WVD can also be expressed as a function of the spectrum of the signal, Z( f ), under analysis as
follows:
=(t, f )"
2517
(4)
The WV function is a particularly popular distribution
due to the large number of desirable mathematical properties it satis"es. Claasen and Mecklenbrauker [10] proved the uniqueness of the Wigner}Ville distribution `in
the sense that it is that single energy distribution that
possesses all the stated desirable propertiesa. This justi"es
the numerous applications of WVD in Pattern Recognition [12,13], Synthesis [14], Seismic signal [15], Optics
[16], etc.
3. The algorithm
The vertical projection pro"le of a non-slanted word
presents the most dips between the characters, even if
they are connected, and the highest peaks at the main
body of the characters. The latter is much more evident
when ascenders and descenders are included. In the case
of slanted words, the otherwise vertical strokes of the
characters cover now the intra-character gaps. Hence, the
dips of the histogram are less deep, while the peaks are
smoother. The alternations (i.e., between dips and peaks)
of the vertical projection pro"le of a certain word are
more intense when it is non-slanted than slanted. In
Fig. 2 the vertical projection pro"les for di!erent slants
of the same word are shown. These slanted words were
produced automatically by applying the technique described below.
The proposed algorithm for slant estimation and correction for a given word, consists of six steps:
In order to slant a word image to right or left we follow
the procedure below. The word image is segmented in
equally wide horizontal zones. The lowest zone is considered to be the base. The zone above the base is shifted
one pixel to the right or to the left. The next zone (if it
exists) is shifted two pixels to the same direction, etc.
Thus, each pixel p(x, y) of the image, is shifted to the point
(x, y):
x"x#i, Z(i(Z,
0(Z)h
(5)
and
y"y,
(6)
where successive zones are identi"ed by successive ordinals Z and h the height of the image. The corresponding
slant will be
tan "Z!1/h.
(7)
The more the zones, the greater the slant angle. The
maximum slant angle corresponds to one-pixel-wide
zones (i.e., when the amount of zones is equal to the
height h of the word image in pixels). In this case, the
higher zone is shifted by h!1 pixels and the corresponding slant angle is the maximum one and it can be
calculated as
tan "h!1+1N"453.
(8)
To illustrate the above procedure, the gradual slanting
of a vertical stroke is shown (in enlargement) in Fig. 4,
while in Fig. 5 we can see the maximum slant of a nonslanted word to left and right. Notice that the words
produced by that technique are very natural (as can be
seen in Fig. 2). A maximum slant angle of $453 covers
the vast majority of handwritings. Shifting each zone by
more than one pixel would increase the maximum slant
angle. However, it could cause undesirable disconnections between adjacent zones producing an unnatural
outcome.
2518
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
Fig. 2. Vertical projection pro"les of a word with various slants. For each slant the slant angle and the number of horizontal zones are as
follows: (a) !453!101, (b) !303!31, (c) !153!27, (d) 03!1, (e) 153!27, (f) 303!31, (g) 453!101.
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
2519
Fig. 3. Curves of maximum intensity that correspond to the histograms of Fig. 2.
Fig. 4. The gradual slanting of a vertical stroke in enlargement.
Fig. 5. The maximum slant of a word to left and right.
4. Experimental results
The presented slant removal algorithm has been tested
on a collection of word images taken from the IAM-DB
[1] and the GRUHD [2] databases comprising English
and Modern Greek unconstrained handwriting, respectively. In more detail, more than 1500 word images were
used taken from approximately 500 di!erent writers,
selected randomly. The word segmentation was performed automatically based on an OCR preprocessing
system [18].
As already mentioned, the evaluation of a slant removal method is di$cult since the selection of the most
appropriate result very often falls under subjective judgements (especially in case of dealing with variant-slanted
words). Moreover, a slant removal algorithm can
2520
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
Fig. 7. The system into which the proposed algorithm has been
incorporated.
Fig. 6. Some experimental results (leftdown: original word images, rightdown: corrected word images).
indirectly be evaluated by taking into account the improvement it provides to an existing OCR system.
Nevertheless, the application of our method to the
above mentioned data produced very satisfactory results.
Some examples are shown in Fig. 6. In general, it is clear
that even in the hardest cases the produced word is
considerably improved as regards its processing in
further stages (e.g., character segmentation, character
recognition, etc.).
It is worth noting that the already non-slanted words
are not a!ected in a negative way by applying this algorithm whether the characters are connected or not. The
ascenders and descenders, if they exist, play an important
role in slant estimation since the alternations of peaks
and dips in the histogram are getting more evident.
However, the proposed method can also handle words
without any ascender or descender. Regarding the
variant-slanted words, the slant of the majority of the
vertical strokes included in the word is more likely to
determine the "nal slant angle since more peaks of the
histogram will be generated.
The presented algorithm has been incorporated into
the character recognition system, shown in Fig. 7, aiming
at the automatic processing of document images. The
system, except of slant removing, includes six other main
modules, namely skew angle estimation and correction,
printed-handwritten text discrimination, line segmentation, word segmentation, character segmentation and
recognition, stemming from the implementation of already existing as well as novel algorithms. The proposed
technique could be applied to words or directly to the
text line images resulted from line segmentation. However, the uneven valleys of the vertical histogram, i.e. wide
valleys between words narrow valley between characters,
could give confusing results in some cases. This is the
reason that we preferred to use part of the text lines
instead. Assuming that every page corresponds to only
one writer, a skew angle is estimated per page. The longer
and the most solid the text parts, the more accurate the
estimation procedure will be, since they include more
information and even more histogram.
In order to select these parts, the valleys of the vertical
histogram of a text line, with width greater than a threshold
are considered to be the boundaries between the parts.
The 10 longest from the resulted parts were selected. As
a threshold, the 1/10 of the line height was used. However, any threshold small enough to sense word segment
valleys or smaller is appropriate. The resulted parts usually are entire words or parts of words. The slant estimation technique is applied to them and the average of the
estimated slants is considered to be the slant of the page.
In Fig. 8 a document image of IAM-DB as inserted
in the system and after the slant correction is shown. In
Fig. 9 the dependence of the recognition accuracy on the
amount of training samples per characters with and without the application of the slant removing algorithm are
presented, respectively. The results have been obtained
from tests applied to 200 forms (100 IAM-DB forms and
100 GRUHD forms). In the case of IAM-DB the system
was trained with samples taken by NIST [17] database,
while for the GRUHD, training sets, di!erent from the
testing sets, of the same database were used. As compared
with the initial recognition system (i.e., when no slant
correction was performed), the required training data for
achieving similar performance were reduced by more
than one-third.
The computational cost of using the proposed technique depends strongly on the size of the document.
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
2521
Fig. 8. An example of IAM-DB document image (a) as inserted into the system and (b) after the slant removing.
Fig. 9. Recognition accuracy versus the amount of training
samples per character, (a) before and (b) after the incorporation
of the slant removing algorithm.
However, the whole slant removing procedure for the
document of Fig. 8 requires 29.516 s using a Pentium III
at 300 MHz while just the slant estimation part demands
22.72 s.
5. Conclusions
In this paper, we presented an algorithm for slant
removal. In contrast to current techniques, our method
is character independent since it is based on the intent
alternations of the vertical histogram, indicating the
vertically oriented characters, rather than detecting the
almost vertical strokes that may be included in the word.
The WVD was used in order to estimate the slant angle
that can range between $453 according to the original
position.
The evaluation of our algorithm was made by both
subjective and objective means. First, the algorithm
was applied to isolated word image samples from both
English and Modern Greek databases. The extracted
results are natural, and almost always improved with
respect to the original image, even in the case of variantslanted writing. Then the algorithm was applied to entire
document images by incorporating it into an OCR system. The performance of the character recognition system was increased by up to 9% for the same data, while
the training time cost was signi"cantly reduced.
Almost any OCR system can bene"t from our algorithm since it requires little computational cost and it is
easily adapted. Cases where characters within a single
word have to be corrected by di!erent slant angles cannot be handled by our approach since it is based on the
dominant angle. However, we currently work on providing the most accurate results.
References
[1] U. Marti, H. Bunke, A full English sentence database for
o!-line handwriting recognition. Proceedings of the Fifth
International Conference on Document Analysis and Recognition, ICDAR'99, Bangalore, 1999, pp. 705}708.
[2] E. Kavallieratou, N. Liolios, E. Koutsogiorgos,
N. Fakotakis, G. Kokkinakis, The GRUHD database of
Modern Greek Unconstrained Handwriting, LREC2000,
Athens, vol. 3, 1999, pp. 1755}1759.
2522
E. Kavallieratou et al. / Pattern Recognition 34 (2001) 2515}2522
[3] M. Watanabe, Y. Hamammoto, T. Yasuda, S. Tomita,
Normalization techniques of handwritten numerals for
Gabor "lters, Proceedings of the International Conference
on Document Analysis and Recognition, ICDAR IEEE,
Los Alamitos, vol. 1, CA, 1997, pp. 303}307.
[4] G. Kim, V. Govindaraju, E$cient chain-code-based image
manipulation for handwritten word recognition, Proceeding of SPIE-The International Society for Optical
Enginering, Bellingham, vol. 2660, WA, USA, 1996,
pp. 262}272.
[5] S. Knerr, E. Augustin, O. Baret, D. Price, Hidden Markov
model based word recognition and its application to legal
amount reading on french checks, Comput. Vision Image
Understanding 70 (3) (1998) 404}419.
[6] A.W. Senior, A.J. Robinson, An o!-line cursive handwriting recognition system, IEEE Trans. Pattern Anal.
Mach. Intell. 20 (3) (1998) 309}321.
[7] M. Shridar, F. Kimura, Handwritten address interpretation using word recognition with and without lexicon,
Proceedings of the IEEE International Conference on
Systems, Man and Cybernetics, Piscataway, vol. 3, NJ,
USA, 1995, pp. 2341}2346.
[8] A. Vinciarelli, J. Luettin, O!-line cursive script recognition
based on continuous density HMM, Proceedings of the
Seventh International Workshop on Frontiers in Handwriting Recognition, Amsterdam, September 2000.
[9] R.M. Bozinovic, S.N. Srihari, O!-line cursive script word
recognition, IEEE Trans PAMI 11 (1) (1989) 68}83.
[10] T.A. Claasen, W.F. Mecklenbrauker, The Wigner distribution: a tool for time-frequency signal analysis, Phillips J.
Res. 35 (Parts 1}3) (1980) 217}250, 276}300, and 372}389.
[11] L. Cohen, Generalized phase-space distribution functions,
J. Math. Phys. 7 (1966) 781}786.
[12] B. Boashash, B. Lovell, L. White, Time frequency analysis
and pattern recognition using singular value decomposition of the Wigner}Ville distribution, advanced algorithms
and architecture for signal processing, Proc. SPIE 828
(1987) 104}114.
[13] G. Cristobal, J. Bescos, J. Santamaria, Application of
Wigner distribution for image representation and analysis,
Proceedings of the IEEE Eighth International Conference
Pattern Recognition, 1986, pp. 998}1000.
[14] K.B. Yu, S. Cheng, Signal synthesis from Wigner distribution, Proceedings of the IEEE ICASSP 85, 1985,
pp. 1037}1040.
[15] P. Boles, B. Boashash, The cross Wigner}Ville
distribution-a two dimensional analysis method for the
processing of vibrosis seismic signals, Proceedings of the
IEEE ICASSP 87, 1988, pp. 904}907.
[16] O. Kenny, B. Boashash, An optical signal processing for
time}frequency signal analysis using the Wigner}Ville
distribution, Journal of Electrical Electronic Engineering,
Australia, 1988, pp. 152}158.
[17] R. Wilkinson, J. Geist, S. Janet, P. Grother, C. Burges,
R. Creecy, B. Hammond, J. Hull, N. Larsen, T. Vogl,
C. Wilson, 1992. The First Census Optical Character
Recognition Systems Conference CNISTIR 4912. The
U.S. Bureau of Census and the National Institute of Standards and Technology. Gaithersburg, MD.
[18] E. Kavallieratou, N. Fakotakis, G. Kokkinakis, An Integrated system for Handwritten Document Image Processing, under reviewing.
About the Author*ERGINA KAVALLIERATOU was born in Kefalonia, Greece, in 1973. She received the Diploma in Electrical and
Computer Engineering in 1996 from the Polytechnic School of the University of Patras. She received her Ph.D. in Handwritten Optical
Character Recognition and Document Image processing from the same department. During the academic year 1997}1998 she was
a member of the Signals, Systems and Radiocommunications Laboratory of the Dept. of Telecommunications Engineering of the
Polytechnic School of Madrid, working on Image Processing. Her research interests include Optical Character Recognition, Document
Image Analysis and Image Processing.
About the Author*GEORGE KOKKINAKIS was born in Chios, Greece. He received the Diploma in Electrical Engineering
(Dipl.-Ing.) in 1961, the Doctor's Degree in Engineering (Dr.-Ing) in 1966 and the Diploma in Engineering Economics (Dipl.-Wirt.-Ing)
in 1967, all from the Technical University of Munich (Technische Hochschule Munchen). Since 1969 he is with the Department of
Electrical Engineering (Electrical and Computer Engineering since 1995) at the University of Patras, where he has organized and is
directing the Wire Communications Laboratory (WCL). His current activity in research and development includes, apart from
telecommunications subjects, the analysis, synthesis, recognition and linguistic processing of the Greek language. He has published
several books on Telecommunications and Electrotechnology and over 250 technical papers, articles and reports on Speech and
Language Technology and on Telecommunications. Dr. Kokkinakis is a senior member of IEEE and a member of the Technical
Chamber of Greece (TEE), the VDE (Verein Deutscher Elektrotechniker), ISCA (International Speech Communication Association), the
EURASIP (European Association for Signal Processing), the Linguistics Society of America (LSA) and the GORS (Greek Operations
Research Society). Since 1997 he is a member of the board of ISCA.
About the Author*DR. NIKOS FAKOTAKIS received the B.Sc. degree from the University of London (UK) in Electronics in 1978, the
M.Sc. degree in Electronics from the University of Wales (UK), and the Ph.D. degree in Speech Processing from the University of Patras,
(Greece), in 1986. From 1986 to 1992 he was lecturer in the Electrical and Computer Engineering Dept. of the University of Patras, from 1992
to 1999 Assistant Professor and since 2000 he has been Associate Professor in the area of Speech and Natural Language Processing and Head
of the Speech and Language Processing Group at the Wire Communications Laboratory. Dr. Fakotakis is author of over 100 publications in
the area of Speech and Natural Language Engineering. His current research interests include Speech Recognition/Understanding, Speaker
Recognition, Spoken Dialogue Processing, Natural Language Processing and Optical Character Recognition. Dr. Fakotakis is
a member of the Executive Board of ELSNET (European Language and Speech Network of Excellence), Editor-in-Chief of the
`European Student Journal on Language and Speecha, WEB-SLS. He is also a member of IEEE, TEE, EURASIP, ISCA.