Int. J. Electronic Security and Digital Forensics, Vol. 4, No. 4, 2012
Drunk person identification using thermal infrared
images
Georgia Koukiou* and
Vassilis Anastassopoulos
Electronics Laboratory,
Physics Department,
University of Patras,
26500, Greece
Fax: +302610997456
E-mail: [email protected]
E-mail: [email protected]
*Corresponding author
Abstract: Drunk person identification is carried out using thermal infrared
images. Two different approaches are proposed for distinguishing a drunk
person by means of radiometric values on its face. The features used in the first
approach are simply the pixel values of specific points on the face of the
person. It is proved that the cluster of a specific person moves in the feature
space as the person consumes alcohol. Fisher linear discriminant approach is
used for space dimensionality reduction. The feature space is found to be of
very low dimensionality. The majority of the clusters moves towards the same
direction and the feature space can easily be separated into the ‘sober’ and
‘drunk’ regions. Thus the ‘drunk’ feature space is introduced. In the second
approach thermal differences between various locations on the face are
evaluated and their value is monitored. It was found that specific areas in the
face of a drunk person present an increased thermal illumination. These areas
are the best candidates for identifying a drunk person. The concept behind this
second proposed approach relies on a physiology-based face identification
procedure.
Keywords: biometrics; thermal infrared signatures; drunk person verification.
Reference to this paper should be made as follows: Koukiou, G. and
Anastassopoulos, V. (2012) ‘Drunk person identification using thermal
infrared images’, Int. J. Electronic Security and Digital Forensics, Vol. 4,
No. 4, pp.229–243.
Biographical notes: Georgia Koukiou received her BSc in Physics in 2004 and
MSc in Electronics and Computers in 2006, both from the University of Patras.
She is a PhD student in Electronics and Computers and she is a member of the
research team of digital image and signal processing in the University of Patras.
Her research interests are in face identification using thermal infrared.
Vassilis Anastassopoulos received his BSc in Physics in 1980 and PhD in
Electronics in 1986, both from the University of Patras, Greece. He worked for
two years in Canadian Scientific Institutions. He is an academic staff member
in Physics Department, University of Patras, Greece since 1987. His
Copyright © 2012 Inderscience Enterprises Ltd.
229
230
G. Koukiou and V. Anastassopoulos
publication record contains over 90 journal and conference papers with over
than 400 citations. His research interests are within the scope of digital
signal, image and radar processing, remote sensing, SAR and infrared
imagery, signal detection, pattern recognition with emphasis in handwritten
analysis, and information fusion including image fusion and sensor fusion
architectures.
1
Introduction
Biometrics is a hot research area with numerous publications the last few years.
Applications are met in medicine, financial transactions, person identification and mainly
in security issues. Research has been carried out (Jain et al., 1999) in several biometric
problems, such as face and fingerprint recognition, facial expression classification and
iris identification with high rate of success.
Research in face recognition (Zhao et al., 2003) has been biased towards the visible
spectrum for a variety of reasons. Among those is the availability of low cost cameras in
the visible portion of the electromagnetic spectrum and the undeniable fact that face
recognition is one of the primary activities of the human visual system. However,
machine recognition using visible light is difficult due to the fact that the acquired images
change with the conditions of illumination. Recently, emphasis has been given to
acquiring information from faces in thermal infrared spectrum (Socolinsky and Selinger,
2002; Buddharaju et al., 2007; Khan et al., 2006; Zhao and Grigat, 2005). The main
reason for this is that the temperature on various positions of 3 the face depends on the
physiological as well as the psychological conditions of the specific person, which in turn
is strictly related to the distribution of the blood vessel network on it (Buddharaju et al.,
2007). A few publications are available in the literature regarding drunk person
identification (Hunicka et al., 2010; Wu et al., 2009; Kojima et al., 2009; Carswell and
Chandran, 1994; Koukiou et al., 2009). Several researchers have worked in person
identification and face recognition using thermal infrared imagery studying the
physiological and psychological properties of the person (Hunicka et al., 2010; Khan
et al., 2009). Nevertheless, these approaches have not been applied to drunk person
recognition and identification.
In this paper, we present two different approaches to the problem of identifying drunk
people using thermal images. The first approach was based on the fact that the
representation of a specific person into the feature space (cluster of feature vectors)
moves as the person consumes alcohol. The feature vector is formed by considering the
values of 20 specific pixels on the face (Koukiou et al., 2009). The feature space is easily
reduced to two dimensions and it is proved that the clusters of the persons move towards
the same direction with alcohol consumption. The feature space is separated into two
regions, i.e., the one containing the clusters of the sober and that with the clusters of the
drunk persons. Accordingly, this feature space is called ‘the drunk space’.
The second approach was based on the fact that the thermal differences between
specific locations on the face increase as the person consumes alcohol. For this purpose
each face image was separated into small square areas of 100 pixels (10 × 10) thus
Drunk person identification using thermal infrared images
231
producing a matrix (8 × 5) of squared regions for each person. For each squared region
the mean pixel value was evaluated for both the sober and the drunk person. Our purpose
was to identify the regions of the drunk person which presented increased temperature
with respect to other regions. Since relative temperatures are monitored, the identification
approach is independent of the auto-ranging capability of the infrared camera.
Experimental results revealed a temperature increase in regions near the mouth and nose
compared to regions on the forehead. More specifically, according to the second method
the image of the sober person is not needed. Only, the face of the drunk person is helpful
for deciding its situation since the temperature of the nose is always higher that on the
forehead.
The organisation of the paper is as follows. In Section 2, a description of the data
used is provided. The first identification procedure applied for the drunk person is
explained in Section 3. The cluster separability is addressed while the dimensionality of
the feature space is examined and dimensionality space reduction is carried out using a
linear transformation. The ‘drunk space’ is define. The second identification procedure is
presented in Section 4 together with the experimental results. Finally, the conclusions are
drawn in Section 5.
2
Data used
The infrared images used in this work were acquired by means of the Thermo Vision
Micron/A10 Model infrared camera of FLIR Company. The operating wavelengths are
from 7.5 μm to 13.0 μm, which means that the obtained information is in the thermal
infrared (max Wien curve at 9.5 μm for 300°K). In the experimental procedure 20 people
were involved. Each person consumed alcohol, drinking 330 ml of beer four times in
equal time intervals of 20 minutes. Before each beer consumption, a sequence of
50 frames was acquired from each person with time distance 100 msec between the
frames. A final acquisition of 50 frames for each specific person was obtained 20 minutes
after drinking the last beer. Thus, a total of 250 frames (5 × 50) were acquired for each
person in five acquisitions. In Figure 1, four consecutive frames for the same person are
presented from one of the five acquisitions. The mean value of the 50 frames of each
acquisition was evaluated using MATLAB, in order to work with frames having high
signal to noise ration. In Figure 2, such an image is demonstrated.
During the acquisition procedure, the lighting and temperature conditions in
the room were kept unchanged. Actually, a very dim light was available from a
neighbouring room for the researcher to be able to work. The Infrared camera
is not affected by this light. The distance of each face from the camera was kept
constant from acquisition to acquisition so that the images of the same person could be
compared.
The persons involved were healthy. The beer was chilled to the same temperature for
all persons, while no food was consumed by anyone, before or during the experiment.
The experiment was conducted once.
232
G. Koukiou and V. Anastassopoulos
Figure 1
Figure 2
Four thermal images of a specific person corresponding to the first acquisition
Twenty points were obtained on each face to monitor temperature changes with the
consumption of alcohol
Note: The samples were obtained at the same position for all acquisitions.
3
Feature formations using simple pixel values
In the first approach for drunk person identification a simple feature vector was formed
by simply taking 20 different points on the face of each person. These points were
obtained at the same position for all faces and acquisitions. Accordingly, samples were
obtained on the forehead, the nose, the eye brows, the chins and other points on the face
as shown in Figure 2. The selection of these points was made taking into consider in the
fact that these regions contain blood vessels which significantly contribute to the region
temperature. Therefore, each face-image corresponds to a 20-dimentional feature vector:
x i = [181 169 203 166 217 175 171 189 169 206 152
144 243 165 225 147 247 149 247 127]t
Drunk person identification using thermal infrared images
233
which corresponds to a point in the 20-dimentional space. Consequently, the set of
50 images for the same individual and a specific acquisition corresponds to a cluster of
50 points in the 20-dimentional space. Since, for the same person, five acquisitions were
obtained, we have in the 20-dimentional space five different clusters of 50 points.
3.1 Cluster separability
First of all, it is important to find out if the cluster which corresponds to the same person
moves in the feature space as the person consumes alcohol (Koukiou et al., 2009). Only
in this case the 20-point feature will be suitable for drunk person identification.
Simultaneously, we have to examine if the cluster of each person moves towards the
same direction with alcohol consumption.
In order to examine the separability of the clusters of the same person in the feature
space, we compare the scatter of each cluster with the distance of the centres of the
clusters. For demonstration purposes, we selected to examine three of the five available
clusters of a person, i.e., those corresponding to the first, third and fifth acquisitions.
Thus, the scatter of each cluster is expressed by means of the standard deviation of the
cluster in each direction. For the first cluster which corresponds to the sober person the
standard deviation is:
σ1 = [2.8238 3.0284 5.9333 2.0156 2.2467 2.0092 4.7983
4.3554 3.6372 1.3159 2.6174 2.2336
0.9625 5.6285 5.4040 2.7887 0 3.4632 0 2.7370]
while for the fifth cluster (person drunk after having consumed four beers) the standard
deviation is:
σ5 = [1.2994 7.4469 3.6503 2.1667 4.1975 3.3006
10.4082 3.4112 2.2395 1.5653 4.3237 1.8679
1.2622 14.1372 6.1686 3.8903 2.2873 6.2559 0.2132 7.2303]
On the other hand, the distance between the first and the last cluster is found to be:
20
m15 =
∑(m
1k -m5k
)2
(1)
k=1
where m is the sample mean of each cluster and k denotes the index of the dimension.
The obtained result is:
m15 = 98.52
This result is to be compared with:
s15 = max ( σ1 ) + max ( σ5 ) = 20.07
(2)
From this specific result it is obvious that the distance between the first and the fifth
cluster m15 = 98.52 is much larger than the summation of the clusters maximum scatter
s15 = 20.07. Thus, the two clusters are totally separable. The same is valid for all pairs of
234
G. Koukiou and V. Anastassopoulos
clusters according to Table 1. The conclusion from this experiment is that the cluster of a
specific person moves with alcohol consumption. This is valid for all 20 persons.
Table 1
Standard deviation (scatter) of its cluster and their distances
Cluster pair
Distance
Standard deviation
(1, 3)
99.5
17.5
(1, 5)
98.5
20.1
(3, 5)
72.9
25.7
However, in order this 20-dimensional feature (and the corresponding 20-dimensional
space) to be suitable for drunk person identification it is of crucial importance that every
cluster moves almost to the same direction as the person consumes alcohol. If the
direction of movement is different for different people then we would have many
directions in the 20-dimensional space, which would be of great significance. In this
case it would be difficult to demonstrate the space in a simpler way (preferably in
two dimensions). A dimensionality reduction procedure which is described in the
next subsection reveals that our problem is almost of two-dimensions, since the solution
of the generalised eigenvalue problem has given only two-eigenvalues with significant
value.
Bringing (next subsection) all clusters in the two-dimensional space, it is evident
from a simple observation of the space that the clusters move to almost the same
directions as the person consumes alcohol. The directions of movement are estimated and
it is proved that the space can easily be separated into ‘sober’ and ‘drunk’ regions.
3.2 Space dimensionality and dimensionality reduction
The dimensionality of the feature space is studied so that its projection into the two most
important dimensions gives the possibility to visually monitor the cluster movement for
each separate person (Koukiou et al., 2009). In our case, the feature space dimensionality
was examined by the statistics of the clusters of eight persons. The eight persons were
selected from the sample of 20, so that they are of the same sex (male) and almost of the
same weight (80 kgr). Only the initial (not drunk) and the final clusters (after having
consumed four beers) were used.
When projecting onto this 2-D-space the suitable directions for maximum separability
has to be found. To achieve that, the projection by means of a linear transformation
W in a new space is required. The vectors wi of W are the new directions where
(each image-vector) x will be projected. This linear transformation is:
yi = w t x i
(3)
where yi is the transformed xi. For energy preservation we have to assure for the
transformation that ||w|| = 1. An important criterion function that can be used for the
separation of the clusters is given by
J=
SB
SW
(4)
Drunk person identification using thermal infrared images
235
When J, becomes maximum the clusters are better separated. SW is called the
within-scatter-matrix while SB is called the between-scatter-matrix. We need SW to be
small and SB large. SW expresses how much each separate cluster is dispersed in the space
(cluster scatter) and is evaluated by summing up all individual cluster matrices Si as
follows
Sw = S1 + S2 +. . .+ S8
(5)
where
Si =
∑x
∗ x it
i
(6)
In the transformed space the within-scatter-matrix (SW) is given by
( SW ) = w t S W w
(7)
The between-scatter-matrix SB reveals how much the centres of the clusters are separated.
The evaluation of the between scatter matrix SB is realised as follows
SB =
∑m
i
∗ mit ,
i = 1, 2, … 8
(8)
where mi corresponds to each cluster centre. In the transformed space the between scatter
matrix (SB) will be given by
( SB ) = w t SB w
(9)
Therefore, we conclude that the function J in the transformed space is given by
J(w) =
w t SB w
w t SW w
(10)
The transformation vectors w that maximise the function J(w) are obtained from the
solution of the generalised eigenvalue problem:
SB Wi = λ i SW Wi
(11)
This solution provides us with the matrix W of eigenvectors wi, which constitute the
directions in the new space, on which we have to project the original image-vectors xi.
Simultaneously, it gives the eigenvalues which correspond to each of the above
eigenvector. The eigenvalues express the importance of each direction-eigenvector in the
feature space. The larger the eigenvalue the better the separability of the clusters we
obtain towards the corresponding eigenvector. The solution of (11) is actually the Fisher
linear discriminant (FLD) procedure. It is a general procedure taking into consideration
the matrices SB and SW with opposite effect. The conventional PCA is a particular case of
FLD, and as such it cannot resolve multiclass-multidimensional classification problem.
The generalised eigenvalue problem was solved, as we mentioned previously, for
eight persons (males) of the same weight. A total of 16 clusters are available in the 20-D
feature space, i.e., two clusters per person (sober and drunk). In Table 2, the five largest
eigenvalues are provided. From the table, it is evident that two of the directions are very
important (corresponding to the two largest eigenvalues) and that for maximum
separability of the clusters we have to project on the plane defined by these two
236
G. Koukiou and V. Anastassopoulos
directions. Actually, the sum of these two largest eigenvalues over the sum of all
eigenvalues gives the quality of cluster seperability in the reduced (2D) feature space. In
this experiment this ration was found equal 70% as it is evident from Table 2. The
resulting two-dimensional feature space is demonstrated in Figure 3 along with the
16 clusters. Furthermore, in the same figure, the direction of movement of the cluster of
each person is exhibited.
Table 2
1st
The five largest eigenvalues, which correspond to the 20-D feature space with eight
persons and 16 clusters
2.48
2nd
1.22
3rd
0.74
4th
0.33
5th
0.22
Figure 3
The 16 clusters of eight persons in the 2-D space formed by the two most important
directions (correspond to the first 2 largest eigenvalues) (see online version
for colours)
Notes: We call hereafter this space, the ‘drunk space’. The dotted line that separates the
drunk from the sober space has been drawn by connecting the mid points of the
vectors that connect each cluster of a sober person with the corresponding cluster
of the drunk person. No a specific algorithm was used for this purpose.
It is obvious from Figure 3, that there is a line in the feature space or otherwise decision
boundary, which separates the space into two regions: the ‘drunk’ and ‘sober’ regions.
Based on this figure we can decide whether an unknown person is drunk or not, from the
position of the corresponding cluster on this space, hereafter the ‘drunk space’.
After that, we examined thoroughly the stability of the two-dimensional ‘drunk space’
as far as the sensitivity of the two main directions to the participation of a specific person
(two clusters) is concerned. Accordingly, one of the eight persons was excluded from the
237
Drunk person identification using thermal infrared images
space dimensionality reduction procedure, and the previously described approach was
carried out employing seven persons (leave-one-out-method). The method was applied
eight times, excluding each time one of the eight persons and thus the ‘drunk space’ was
evaluated eight times. The question is: how different are these eight ‘drunk spaces’ from
the initial ‘drunk space’ evaluated using all eight persons?
In order to carry out the above sensitivity analysis of the ‘drunk spaces’, we
compared the two dimensions of the ‘drunk space’ with the eight persons with the
corresponding directions of the eight ‘drunk spaces’ with seven persons. The correlation
measure used was the cosine of the angle formed between the first most important
direction of the ‘drunk space’ with eight persons and the corresponding first most import
direction of each one of the eight ‘drunk spaces’ with the seven persons:
cos θi =
a1 ⋅ bi1 + a2 ⋅ bi 2 +
a12 + a22 +
+ a20 ⋅ bi 20
2
+ a20
⋅ bi21 + bi22 +
+ bi220
, i = 1, … ,8
where aj, j = 1, …, 20 are the components of the main direction of the 2-D space with
eight persons while bij, j = 1, …, 20 corresponds of the main direction of the ith 2-D space
with seven persons. The angles θi are presented in Table 3. From this table, it is obvious
that the main direction of the space does not change significantly when one of the eight
persons is excluded. The same procedure was applied for the second most important
direction of the ‘drunk space’ with eight persons and the corresponding second most
important direction of the eight ‘drunk spaces’ with the seven persons. The corresponding
angles ϕi are depicted in the same table. The main conclusion is that the second important
direction does not change significantly when a person is excluded.
In conclusion, the created drunk space is suitable for drunk person identification.
Specifically, if a person has a feature vector in the ‘drunk’ region, then it has to be further
examined for alcohol consumption.
Table 3
The angles θi formed between the first most important directions of the ‘drunk space’
with eight persons and the corresponding first most import direction of each one of
the eight ‘drunk spaces’ with the seven persons
cos
θi in degrees
cos
ϕi in degrees
1st
0.9534
17.5
0.9814
11.1
2nd
0.9862
9.5
0.9913
7.6
3rd
0.7491
41.5
–0.8575
149.0
4th
0.9581
16.6
0.9877
9.0
5th
0.9736
13.2
0.5333
57.8
6th
0.9640
15.4
0.9397
20.0
7th
0.9835
10.4
0.8721
29.3
8th
0.9105
24.4
0.9886
8.6
Person
Note: Accordingly, the angles φi formed between the second most important directions of
the ‘drunk space’ with eight persons and the corresponding second most import
direction of each one of the eight ‘drunk spaces’ with the seven persons.
238
4
G. Koukiou and V. Anastassopoulos
Face thermal changes in drunk people
In the second proposed approach for drunk person identification, the thermal differences
between various locations on the face are examined. The final goal of the approach is to
determine if some regions of the face become hotter when consuming alcohol. Moreover,
it is very important to examine and find regions, which initially are cold, compared to
specific other regions for the sober person and become hotter for the drunk person. For
example, we do not care about the absolute temperature of the eye but we monitor if the
temperature of the eye increases with respected to the temperature of the other location of
the drunk person (e.g., mouth).
4.1 Face thermal change detection
In this second identification approach the face of each person was partitioned into a
matrix of 8 × 5 squared regions. Each region (i, j) is of 10 × 10 pixels and is located on
the same position of the face for every image (see Figure 4). The identification approach
is based on monitoring the temperature difference between all possible pairs of squared
regions as the person consumes alcohol. As it is shown in Figure 4, the thermal difference
of the two black regions will be examined for the sober and the drunk person. It is very
important to identify the pairs for which the thermal difference increases too much or
even better changes sign from the sober to drunk person.
Figure 4
These face-image results as the mean of the 50 frames of a specific acquisition
Notes: The face is afterwards partitioned into a matrix of 8 × 5 squared regions. Each
region is of 10 × 10 pixels and its temperature with respect to the other regions is
monitored. Two of these regions (forehead and nose), which present a large
change in their thermal difference from sober to drunk person, are shown in black.
The region on the nose becomes hotter, for the drunk person, compared to the
region on the forehead.
In the experimental procedure 50 frames were obtained per person and acquisition.
Accordingly, firstly is evaluated the mean of these 50 frames and after that we evaluated
the mean pixel value in each squared region using its 10 × 10 pixels. A total of 40 values
were calculated on the face of a specific person, which correspond to the squared regions
for a specific acquisition.
The next step in the procedure was to evaluate the thermal differences among all
values of the 40 squared regions, thus creating a matrix 40x40 which contained their
thermal differences on the image. A thermal difference matrix was formed, Figure 5, for
each person and acquisition. Accordingly, five difference-matrices correspond to each
person. In our approach we had to compare the difference-matrices which correspond to
the same person when he was sober and in the case he consumed four beers. Comparing
Drunk person identification using thermal infrared images
239
these two matrices we recorded the elements which presented the maximum variation of
the thermal differences. It was evident that the thermal difference between specific
regions on the face increased for the drunk person. Specifically, we observed that the
thermal difference between the forehead and the region of nose and mouth increases for
the drunk person. That is, the nose and the mouth became hotter compared to the
forehead for the drunk person.
Figure 5
Three difference matrices, (a) for the sober person and (b) for the drunk person
(c) the difference of the difference matrices (values normalised to full grey-scale)
(see online version for colours)
(a)
(b)
(c)
Notes: In this last matrix, white pixels indicate the coordinates of the squared regions,
which present large change in their thermal difference. The largest of these
differences, in this example, is shown in the white circle and its value is 29.8.
In Figure 4, two locations are demonstrated on the face of a drunk person that present
thermal difference. The temperature of the nose is increased compared to that of the
forehead. The nose was found to become hotter than the forehead for the drunk person
while their temperature was almost the same for the sober person. Since the whole
identification procedure was based on the thermal difference matrices, and the
localisations of their maximum changed, we present such difference matrices as a
grey-scale 40 × 40 image in Figure 5. The first matrix [Figure 5(a)] corresponds to sober
person. The second matrix [Figure 5(b)] corresponds to the drunk person having
consumed four beers, while their difference is presented as a third matrix [Figure 5(c)]
where the maximum changes can be localised. The maximum variation of the difference
matrices in Figures 5(a) and 5(b) are represented with the white pixels in the matrix of
Figure 5(c). The coordinates (i, j) of these white pixels show those pairs of squared
regions on the face that have maximum thermal change.
240
G. Koukiou and V. Anastassopoulos
4.2 Experimental results
In Figure 6, the regions that exhibit the largest change in thermal differences after alcohol
consumption are demonstrated for two different people. These regions were indicated by
the corresponding difference matrices as the one in Figure 5(c), as being candidates for
revealing a drunk person in a potential alcohol test.
For Person A, eight squared regions on the forehead (in black) present change in
thermal difference larger than 55 with respect to three squared regions around nose. A
similar image corresponding to Person B is shown in the same figure. For this person we
have almost the same regions (in black) of the face that present large change in thermal
differences. For Person A the threshold for identifying thermal differences was set to 55.
A total of 17 different pairs of regions were found to present thermal difference larger
than the threshold. In Table 4, the five largest differences are given for all pairs and
Person A. For Person B the threshold for identifying thermal differences was set to 25.
A total of 21 different pairs of regions were found to present thermal difference larger
than the threshold. Additionally, in Table 4, the five largest differences are given for all
pairs and Person B.
Table 4
The five largest differences for Person A and B depicted in Figure 6
Region pairs
Person A
Person B
1
46.7
(1, 14)
29.8
(1, 23)
2
46.2
(1, 9)
29.6
(10, 23)
3
46.2
(1, 13)
28.8
(2, 23)
4
46.2
(1, 18)
28.7
(3, 23)
5
45.9
(1, 15)
28.1
(1, 40)
Note: In parentheses are given the pairs of the squared regions with the largest
differences.
Figure 6
The regions that present the largest change in thermal differences for two different
persons are illustrated
Notes: For Person A, eight squared regions on the forehead present thermal differences
with respect to three squared regions around mouth. For Person B, six squared
regions on the forehead present thermal differences with respect to five squared
regions around mouth.
241
Drunk person identification using thermal infrared images
A simple automatic procedure was implemented in software, in order to demonstrate the
regions with maximum thermal change. Figure 4 is a snapshot of this procedure. Only the
regions with thermal difference larger than the threshold, are presented to reader. This
threshold is operationally adjustable. The thresholds are chosen each time (depending on
the person) according to the final number of pairs of squares which present high thermal
differences. The threshold is lowered when the obtained number of pairs is small, so that
an adequate number of pairs of squared regions are recorded.
The increase in the thermal difference between the forehead and the nose for six
persons is shown in Table 5. This difference was recorded for all 20 persons and its mean
was found 41.3 and its standard deviation 19.8. All 20 values were tested using the z-test,
whether they follow the Normal density with the above mean and variance. Using
MATLAB we found that this hypothesis cannot be rejected, with 5% significance level.
Table 5
Persons
The temperature of nose increases for the drunk person for all people, and becomes
larger than that of the forehead
Sober
Forehead
Drunk
Nose
Forehead
Nose
Increase in thermal
difference between nose
and forehead
Person A
230.5
221.1
214.7
233.9
19.2 – (–9.5) = 28.7
Person B
230.4
226.5
197.2
226.7
29.4 – (–3.9) = 33.3
Person C
229.7
183.8
193.4
232.3
38.9 – (–45.9) = 84.8
Person D
220.0
218.7
190.8
227.4
36.5 – (–1.3) = 37.8
Person E
233.7
186.4
220.1
218.3
–1.8 – (–47.3) = 45.5
Person F
212.7
219.2
200.0
232.6
32.6 – 6.4 = 26.2
Notes: The fact that in two of the drunk persons the temperature of nose is not higher
than the forehead (Table 5) does not mean that the generality of the conclusion is
lost, since, even in this case the relative temperature of nose increases
significantly.
In conclusion, the region of nose, almost in all cases, becomes hotter compared to the
forehead, for persons who have consumed alcohol. This fact can be used as the basic
indication that a person is drunk. A drunk person identification system will present a
person as drunk, if the nose is hotter than the forehead. Thus, an alcohol test procedure
would involve a thermal camera and a simple software embedded to detect the
temperature difference between nose and forehead. This system has the advantage that it
will not come in contact with the tested person (non-invasive procedure).
5
Conclusions
In this paper, two different approaches were presented to the problem of identifying
drunk people using thermal images. The first approach was based on the fact that the
representation of a specific person into the feature space (cluster of feature vectors)
moves as the person consumes alcohol. The feature vector was formed by simply
considering the values of specific pixels on the face. The feature space was easily
reduced to two dimensions and it was proved that the clusters of the persons move
towards almost the same direction after alcohol consumption. The feature space is
242
G. Koukiou and V. Anastassopoulos
separated into two regions, i.e., the region containing the clusters of the sober people and
that with the clusters of the drunk people. Accordingly, we named this feature space ‘the
drunk space’. The ‘drunk space’ can be used for discriminating drunk people by simply
checking the position of the persons’ cluster in the transformed 2-D space.
The second approach was based on the fact that the thermal differences between
specific locations on the face change as the person consumes alcohol. Regions on the face
of the drunk person which present increased temperature with respect to other regions
were detected. Experimental results showed a temperature increase in regions near the
mouth and nose with respect to regions on the forehead.
Both methods reveal that a drunk person presents thermal differences from a sober
person while these differences are prominent between specific regions on the face. More
specifically, according to the second method the image of the sober person is not needed.
Only, the face of the drunk person is helpful for deciding its situation since the
temperature of the nose is always higher that on the forehead. The fact that in two of the
drunk persons the temperature of nose is not higher than the forehead (Table 5) does not
mean that the generality of the conclusion is lost, since, even in this case the relative
temperature of nose increases significantly.
Acknowledgements
This research has been co-financed by the European Union (European Social
Fund – ESF) and Greek national funds through the Operational Programme
‘Education and Lifelong Learning’ of the National Strategic Reference Framework
(NSRF) – Research Funding Programme: Heracleitus II.
Investing in knowledge society through the European Social Fund.
References
Buddharaju, P., Pavlidis, J.T., Tsiamyrtzis, P. and Bazakos, M. (2007) ‘Physiology-based
face recognition in the thermal infrared spectrum’, IEEE Trans. on PAMI, April, Vol. 29,
No. 4, pp.613–626.
Carswell, B. and Chandran, V. (1994) ‘Automated recognition of drunk driving on highways from
video sequences’, IEEE International Conference on Image Processing, Proceedings ICIP-94,
November, Vol. 2, pp.306–310.
Hunicka, B., Laurell, H. and Bergman, H. (2010) ‘Psychosocial characteristics of drunk drivers
assessed by addiction severity index, prediction of relapse’, Scandinavian Journal of Public
Health, February, Vol. 38, No. 1, pp.71–77.
Jain, A., Bolle, R. and Pankanti, S. (1999) Biometrics: Personal Identification in Network Society,
1st ed., Kluwer Academic, New York.
Khan, M.M., Ingleby, M. and Ward, R.D. (2006) ‘Automated facial expression classification and
affect interpretation using infrared measurement of facial skin temperature variations’,
ACM Transactions on Autonomous and Adaptive Systems, September, Vol. 1, No. 1,
pp.91–113.
Khan, M.M., Ward, R.D. and Ingleby, M. (2009) ‘Classifying pretended and evoked facial
expressions of positive and negative affective states using infrared measurement of skin
temperature’, ACM Transactions on Applied Perception (TAP), February, Vol. 6, No. 1,
pp.1–22.
Drunk person identification using thermal infrared images
243
Kojima, S., Maeda, S., Ogura, Y., Fujita, E., Murata, K., Kamei, T., Tsuji, T., Kaneko, S. and
Yoshizumi, M. (2009) ‘Noninvasive biological sensor system for detection of drunk driving’,
9th International Conference on Information Technology and Applications in Biomedicine,
ITAB 2009, November, pp.1–4.
Koukiou, G., Panagopoulos, G. and Anastassopoulos, V. (2009) ‘Drunk person identification
using thermal infrared images’, 16th International Conference on Digital Signal Processing,
5–7 July, pp.1–4.
Socolinsky, D.A. and Selinger, A. (2002) ‘A comparative analysis of face recognition performance
with visible and thermal infrared imagery’, Proc. 16th International Conference Pattern
Recognition, Vol. 4, pp.217–222.
Wu, Y.C., Xia, Y.Q., Xie, P. and Ji, X.W. (2009) ‘The design of an automotive anti-drunk driving
system to guarantee the uniqueness of driver’, International Conference on Information
Engineering and Computer Science, ICIECS 2009, December, pp.1–4.
Zhao, S. and Grigat, R.R. (2005) ‘An automatic face recognition system in the near infrared
spectrum’, Proc. 4th International Conference on Machine Learning and Data Mining in
Pattern Recognition, July, pp.437–444.
Zhao, W., Chellappa, R., Phillips, P.J. and Rosenfeld, A. (2003) ‘Face recognition: a literature
survey’, ACM Computing Surveys, December, Vol. 35, No. 4, pp.399–458.