2012 7th International ICST Conference on Communications and Networking in China (CHINACOM)
Rock Classification via a Mobile Device
1,4
Chia-Hsiang Wu1
Department of Biomedical Engineering
I-Shou University
Kaohsiung, TAIWAN
[email protected]
Jiann-Shu Lee2, Chin-Yin Shie3, and Mei-Yun Su4
Department of Computer Science and Information
Engineering, National University of Tainan
Tainan, TAIWAN
[email protected]
2,3
by using co-occurrence matrix to analyze and classify rocks by
using texture features. Edge is one of the key components in
texture and useful in rock image analysis. Xu et al.[13] propose
a fracture edge detection method from rock images by using
quaternion convolution at different scales as well as gray level
difference to obtain the monochromatic edges. In addition to
texture, color is also an important factor to identify rocks [9].
For example, Bruno et al. [7] use color histograms and sizeintensity diagrams to characterize ornamental stone samples.
Lebrun et al. [8] analyze rock samples based on L*a*b* color
space. Lepistö et al. [11] combine textural features and spectral
features to classify rock images based on color parameters and
co-occurrence matrix. Support vector machine (SVM) is one of
the popular classifiers in image analysis. Yang et al. [15]
present a image classification method by using SVM combined
with spatial pyramid matching (SPM) method to reduce the
complexity of SVMs. Neuro-fuzzy approaches are helpful in
decision making and classification. A hierarchical neurofuzzy
system was presented for classification of microscopic rock
texture based on texture descriptors [14].
Abstract— Mobile devices have been widely used to facilitate
information exchanges and help our daily lives. In this paper, we
present a rock classification system based on mobile
environment. This system consists of a mobile phone and a
remote server. First, the mobile phone is used to image rocks, due
to the convenience in viewing and checking the captured images.
However, it is difficult to automatically recognize the rocks in the
phone because of insufficient computing capability. Therefore,
the users can select and submit crucial images to a remote server
in order to classify the type of rocks. The classification task is
carried out by feature extraction, followed by a neural network–
based classifier. The texture features consists of color,
directionality, and granularity. With the extracted features, an
ANFIS classifier is used to accomplish the recognition task. The
experimental results show that our system can successfully
classify rocks and achieve the ubiquitous rock classification task.
Keywords-ubiquitous
classification
I.
computing,
mobile
phone,
rock
INTRODUCTION
In recent years, the development of ubiquitous computing
[1] makes intelligent human-computer interaction becoming
popular. At the beginning, human computer interaction is based
on mainframe computers, then personal computers, and then
mobile devices. Nowadays, it is common that people use more
than one computing device from place to place and time to time
such that the interaction has becoming a part of our daily lives.
One of the key in this success is the popularity of mobile
devices. Mobile devices, such as mobile phones, facilitate
convenient remote information exchange and alleviate the
limitation in space and time. Also, they help to improve
learning and group collaboration [2]. Moreover, the
applications in scientific research, especially in data collection
and observation are also possible [3-5] via a variety of mobile
devices.
In this paper, we present a mobile phone based system for
rock classification. In order to achieve on-line classification of
rocks, as shown in Fig.1, the rock image is captured by the
mobile phone. Then, the phone uploads the image to the server.
Next, the server identifies the type of the rock and submits the
results as well as relevant information of the rock to the phone.
In this way, the computation is shared by the remote server
such that the operation is fast enough to approach real time
interaction.
Successful rock classification is one of the keys to the
proposed system. Here, we propose a rock classification
method based on rock image features and an adaptive-networkbased fuzzy inference system. We designate membership
functions for each of the features in order to train the neural
network in a supervised way.
In scientific research and education, observation and data
analysis is essential to obtain deeper understanding. For
example, it is common that a dermatologist carries a handheld
imaging device to capture the appearance of the target site, in
order to evaluate or trace the effectiveness of the treatments.
Similarly, earth scientists try to understand how the Earth
system works via a variety of tools. One of the activities to
build the understanding is the recognition of rock and mineral.
The training of the recognition of rocks usually relies on books
and in-classroom teaching, and it is necessary to extend the
training to be out of the classroom. Therefore, we decide to
build a mobile phone based rock classification system.
Mobile phone
User interface
Server
Rock images
Identification
system
Classification results
Figure 1.
System flowchart.
To analyze and identify the type of rocks, several methods
have been proposed. In [6], Autio et al. extract texture features
663
978-1-4673-2699-5/12/$31.00 © 2012 IEEE
II.
and the surface of the rock is rough. Suppose that the rock is on
the user’s hand, as shown in Fig. 2(a). The captured image
should locate around the center of the image. If the image is of
size M × M, an appropriate ROI(region of interest) could be
M/2 × M/2. In this study, ROI with this size works well, as
shown in Fig. 2(b). We can evaluate the texture of the rock in
the ROI image rather than the original image, which suffers
from the interference from background objects such as the
hand.
FEATURE EXTRACTION
Earth scientists can identify the type of rocks according to
image features, such as color, directionality and granularity.
Therefore, we choose these three features to facilitate the
classification.
Color is one of the most important descriptors in real world.
In a computer, it is common to specify colors via a color
model. In RGB color model, each color is represented by the
amounts of red(R), green(G), and blue(B). However, the
difference of two RGB triplets could not reflect the actual color
difference. Therefore, we choose the HSI color model instead
of the RGB model. HSI is the abbreviation of hue, saturation,
and intensity. Because intensity represents the degree of
brightness, we use hue and saturation as the color features in
this study. Given a pixel color (r,g,b), the hue h and saturation s
can be obtained by
b≤g
b>g
⎧t
h=⎨
⎩360 − t
s = 1 − 3(min( r , g , b )) /( r + g + b )
(1)
(2)
where
(a)
1
⎛
⎞
(( r − g ) + ( r − b))
⎜
⎟
2
⎟ ,
t = cos ⎜
2
1/ 2
⎜⎜ (( r − g ) + ( r − b)( g − b)) ⎟⎟
⎝
⎠
−1
(3)
The ROI image is then filtered with a median filter. The
granularity is evaluated by comparing the number of extremes
of the gray level of the original ROI image and the filtered
image. Similar numbers indicate low granularity because the
filter does not change the image too much, and vice versa.
and min(r,g,b) denotes the minimum among r, g, and b. The
color features hue and saturation are used to identify whether
the imaged object is color or grayscale.
III.
The direction of texture is another clue for rock
identification. More specifically, the direction of texture is
represented by the direction of line segments, suppose that we
consider the texture is formulated by a group of line segments.
To evaluate the directions of line segments, edge detector is
necessary. Most of the edge detectors are based on the
derivatives of gray levels in the horizontal and in the vertical
directions. To evaluate the derivatives Ix and Iy in the two
directions for a given pixel (x, y), finite difference is used to
approximate the derivative. With the derivatives in the two
directions, the direction of the edge passing through the pixel is
evaluated by
⎛ Iy ⎞
⎟⎟
⎝ Ix ⎠
θ = tan −1 ⎜⎜
(b)
Figure 2. Background object removal. (a) Original image. (b) ROI.
CLASSIFICATION
ANFIS (adaptive-network-based fuzzy inference system)
[12] is based on FIS (fuzzy inference system) and neural
network. The fuzzy inference system employs if-then rule to
describe the knowledge and inference process of human beings,
and the neural network is capable of self-learning. ANFIS
combines the fuzzy inference system and the neural network to
compensate for each other, and it is the basis for our rock
classification.
A. ANFIS
Here, we explain the ANFIS based on fuzzy rules of Takagi
and Sugeno’s type [11]. Suppose that there are two inputs, x1
and x2, and one output f. The fuzzy rule can be expressed as
(4)
Rule 1： if x1 is A1 and x2 is B1 then f1=α1x1+β1x2+γ1
After the direction of each pixel has been calculated, we check
the histogram of the directions. For the texture with no specific
direction, the histogram is close to a uniform distribution. This
indicates that the texture is non-directional. On the other hand,
there should have at least one peak for the texture with specific
direction.
where Ai and Bi are linguistic labels of x1 and x2 , and they
can be represented by membership functions Ai ( ) and Bi ( ) ,
In this study, the degree of granularity is also useful in
determining the type of rocks because the rock is not polished
such as triangular, trapezoidal, Gaussian, …, etc. The
membership function indicates the degree about how the input
Rule 2： if x1 is A2 and x2 is B2 then f2=α2x1+β2x2+γ2
664
satisfies the linguistic label, and the parameters that
determining the membership functions are called premise
parameters. The framework of ANFIS consists of five layers.
The first layer is input layer, whose outputs are the membership
function values
estimator. If the premise parameters are known, but not optimal,
the non-optimal premise parameters can be refined by steepest
descent method and re-estimate the consequent parameters can
be estimated again.
(5)
C. Membership functions
With the features, including color, directionality, and
granularity, as described in section II, we establish the
membership functions of the ANFIS. In determining the
membership functions for color feature, the rock images are
categorized as color and grayscale. Ten color samples and ten
grayscale samples were acquired to establish the membership
functions, where Fig. 4 demonstrates two examples. Because a
color can be referred by hue and saturation, we build the
membership functions based on the two features, where each
feature is associated with two membership functions to
distinguish between color and grayscale. Therefore, there are
four functions for color features. The membership functions for
directionality and granularity were established similarly, in
which they are associated with one and three functions.
O1i = Ai ( x1 ), O2i = Bi ( x2 ), i = 1,2
Every node in the second layer multiplies the incoming inputs
O11, O12, O21, and O22, and sends the product out by using Tnorm operator :
wi = O1i × O2 i
(6)
The product represents the firing strengths of fuzzy rules. The
third layer normalize the firing strengths by calculating the
ratio of firing strengths of the ith rule to the sum of all rules :
Wi =
wi
w1 + w2
(7)
The fourth layer multiplied the output of layer three by the ifthen rules:
Wi f i = Wi (α i x1 + β i x2 + γ i )
(8)
where α i , β , γ i are called consequent parameters. The fifth
i
layer computes the output by
f =
w1 f1 + w2 f 2
= W1 f 1 + W2 f 2
w1 + w2
.
Figure 3. Membership function (Hue).
(9)
The above framework is based on two fuzzy rules. The
ANFIS can vary by changing the number of fuzzy rules, the
number of inputs/outputs, the number of membership functions
such that the complexity changes.
B. Learning
According to the computation of the output in the fifth layer,
we have
f = W1 (α1 x1 + β1 x2 + γ 1 ) + W2 (α2 x1 + β2 x2 + γ 2 )
=α1 (W1 x1 ) + β1 (W1x2 ) + γ1 (W1 ) + α2 (W2 x1 ) + β2 (W2 x2 ) + γ 2 (W2 )
(10)
Figure 4. Exampls of (left) color and (right) grayscale rocks.
Reformulating this equation, we can obtain a matrix form
AB=f
D. Rock classification
The ANFIS for rock classification are based on three kinds of
features: color, granularity, and directionality, and their
membership functions. According to the number of
membership functions, there are 12 rules in the rule bank. With
the inputs, membership functions, fuzzy if-then rules, the
ANFIS is ready for rock classification.
(11)
where f are outputs, B are consequent parameters, and A is the
matrix collecting the weighted parameters of B based on (10).
Assume that there are sufficient number of training data, the
consequent parameters can be estimated by a least-square
665
IV.
Communications of the ACM, vol. 42, no. 8, pp. 21–27,
1999.
[5] C. Staudt and S. Hsi, “Synergy projects and pocket
computers,” Concord Consortium Newsletter, 1999.
[6] J. Autio, S. Lukkarinen, L. Rantanen, and A. Visa, “The
classification and characterisation of rock using texture
analysis by co-occurrence matrices and the hough
transform,” in Proc. International Symposium Applications
in Geology, pp. 5-8, 1999.
[7] R. Bruno, S. Persi paoli, P. Laurenge, M. Coluccino, F.
Muge, V. Ramos, P Pina, M. Mengucci, M. Cica Olmo,
and E. Serrano Olmedo, “Image analysis for ornamental
stone standard’s characterisation,” in Proc. International
Symposium on Imaging Applications in Geology, pp.29-32,
1999.
[8] V. Lebrun, C. Toussaint, and E. Pirard, “On the use of
image analysis for the quantitative monitoring of stone
alteration,” in Proc. Weathering, 2000.
[9] V. Lebrun and L. Macaire, “Aspect inspection of marble
tiles by colour line-scan camera,” in Proc. QCAV'2001,
2001.
[10] L. Lepistö, I. Kunttu, J. Autio, and A. Visa, “Rock image
classification using non-Homogenous textures and spectral
imaging,” in Proc. 11th International Conference in
Central Europe on Computer Graphics, Visualization and
Computer Vision, 2003.
[11]T. Takagi and M. Sugeno, “Derivation of fuzzy control
rules from human operator’s control actions,” in Proc.
IFAC Symp. Fuzzy Inform, Knowledge Representation and
Decision Analysis, pp. 55-60, 1983.
[12]J.-S.R. Jang, “ANFIS: adaptive-network-based fuzzy
inference system,” IEEE Trans. System, Man and
Cybernetics, vol. 23, no.3, pp.55-60, 1993.
[13] J. Xu, W. Wang and L. Ye, "Rock fracture edge detection
based on quaternion convolution by scale multiplication,"
Optical Engineering, vol. 48, no.9, 2009.
[14]L.B. Goncalves, F.R. Leta and S.C. de Valente,
"Macroscopic Rock Texture Image Classification Using an
Hierarchical Neuro-Fuzzy System," in Proc. 16th
International Conference on Systems, Signals and Image
Processing( IWSSIP), pp.1-5, June 2009.
[15]J. Yang, K. Yu, Y. Gong, and T. Huang, "Linear spatial
pyramid matching using sparse coding for image
classification," in Proc. IEEE CVPR 2009, pp.1794-1801,
June 2009.
EXPERIMENTAL RESULTS
To evaluate the performance of rock classification, there are
25 types of rocks in our database, where each type consists of
15 images. Two examples are shown in Fig. 5. Total
25*15=375 images were evaluated in our experiments. The
performance of the classification is evaluated by the number of
sample successfully classified divided by the total number of
images. There are 291 images were successfully identified and
84 images failed. Therefore, the rate of success is 77.6%, which
is satisfied by the collaborative earth scientists.
Figure 5. Example of the testing data.
V. CONCLUSION
In this paper, we present a mobile phone based rock
classification system. We have shown that this system can
successfully transmit image from a mobile phone to a remote
server, recognize the type rocks, and send the results back to
the phone. The experimental results demonstrate that the
success rate is around 78%. In the near future, we will
investigate techniques that can deal with low image quality of
the camera equipped to the phone, lighting variation in the
outdoor environment, and image blurring resulting from hand
movement.
ACKNOWLEDGMENT
This research was supported by the National Science
Council of the ROC, under Grant No. NSC 100-2221-E-024 001 and NSC 100-2221-E-214-009.
REFERENCES
[1] M. Weiser, “The computers for the twenty-first century,”
Scientific American, vol. 265, no. 3, pp. 94-104, 1991.
[2] J. Roschelle and R. Pea, “A walk on the WILD side: How
wireless handhelds may change,” in Proc. CSCL, 2002.
[3] R. Rieger and G. Gay, “Using mobile computing to
enhance field study,” in Proc. CSCL, pp. 215–223, 1997.
[4] E. Soloway, W. Grant, R. Tinker, J. Roschelle, M. Mills,
and M. Resnick, “Science in the palms of their hands,”
666