Kansei Retrieval Method using Visual Pattern Image Coding for Virtual Reality
Space Presentation
Kaoru Sugita1, Tomoyuki Ishida2, Akihiro Miyakawa2, 3, Leonard Barolli1, Yoshitaka Shibata2
1
Fukuoka Institute of Technology
2
Iwate Prefectural University
3
Planning Section, Tatsuruhama, Ishikawa Prefecture
Abstract
In this paper, we propose a Kansei Retrieval method
based on the design pattern of traditional Japanese
crafting object to provide a user with the desired
presentation space in digital traditional Japanese crafting
system. The visual quantitative feature values are extracted
by using Visual Pattern Image Coding (VPIC). These
values include the total number, the frequency, the
dispersion rate and the deviation rate for different edges.
The quantitative feature values for traditional Japanese
crafting objects are registered in the multimedia database
and the relation between Kansei words and the visual
feature of traditional Japanese crafting objects are
analyzed by using the questionnaire. Then, the visual
features are compared with the quantitative feature values.
Through the above process, we want to find the relation
between the design pattern components and edge types
using VPIC. By finding this relation, the Kansei retrieval
method can be realized.
1. Introduction
In Japan, there are many traditional Japanese crafting
such as fittings, furniture, textile, etc., which are closely
related to Japanese culture and life. However, these
industries are on the decline in recent years. In order to
keep traditional Japanese crafting in the future, it is
necessary to attract people and educate many successors.
From 1980 to 1990 some database systems for traditional
Japanese crafting are proposed to introduce information
technology in traditional Japanese crafting or local industry.
This data are shown on World Wide Web (WWW) [1] and
include shoji, fusuma, ranma and so on. However, it is
difficult to represent the impression and functions for most
of the crafting objects by existing hypermedia composed of
text, images and movies. Most of the crafting objects are
very important elements for living space and the fittings
should be fit in the Japanese rooms. So, it is necessary to
represent the impression of the room by open/close the
fitting from different places. In order to overcome such
problems, it is required to realize the virtual reality
presentation system including presentation space and
crafting objects by 3D Computer Graphics (CG) for
traditional Japanese crafting.
We have proposed a Digital Traditional Japanese Crafting
System (DTJCS) [2] providing a presentation space for
traditional Japanese crafting. We also introduced a Kansei1
1
Kansei is a Japanese word which shows people feelings
to the objects, such as “Gorgeous”, “Calm”, etc.
0-7803-8603-5/04/$20.00 ©2004 IEEE.
Retrieval Method (KRM) [3, 4] to provide the presentation
space reflected by people’s feelings. In this paper, we
propose Kansei Retrieval Method using Visual Pattern
Image Coding for virtual reality space presentation to
provide the users with the desired presentation space.
The organization of this paper is as follows. In the next
Section, we will introduce the related works. We describe
the DTJCS in Section 3. We explain KRM in Section 4.
The extraction method using Visual Pattern Image Coding
(VPIC) [5] is treated in Section 5. We show the
experimental results of extraction by using VPIC in
Section 6. Finally, some conclusions are given in Section 7.
2. Related works
A Kansei retrieval method for textile design image
database system was proposed by Fukuda et.al. [7, 8]. The
proposed image retrieval method uses the relation between
Kansei word and the feature of the image which may be
the color or the pattern shape. This method was applied to
retrieve the textile images, but it can not be applied to the
traditional Japanese crafting object.
The shape future space constructing method for retrieving
image database was proposed by Harada et. al. [9]. This
method uses for retrieving the relevance between Kansei
word and the shape features.
A retrieval method on the image database using sensitive
word reflecting user’s subjectivity was proposed by Kurita
et. al. [10]. These methods did not consider the affinity of
the design pattern with the traditional Japanese crafting
object.
3. DTJCS
In order to learn the existing products, the craftsmen and
the successors should see the products during the design
process. Also, the customers and craftsmen should see the
products during the design process too. Especially,
products such as fittings are important elements of the
designed products. For this reason, the designed products
should be very close to the desired ones. But using the
conventional design methods, it is very difficult to achieve
this goal. On the other hand, by using computers, the
presentation of traditional Japanese crafting products is
possible in a digital way by CG and Virtual Reality (VR)
technologies. In this way, it is possible to see the virtual
products during the design process. For this reason, we
developed the DTJCS which is shown in Fig.1. The
DTJCS can realize a 3D CG presentation of traditional
Japanese crafting for both classic and modern fittings. Also,
the proposed system can be used by the craftsmen and the
successors to study traditional Japanese crafting which
include the existing products and their impressions. On the
other hand, the proposed system can be used by architects
and designers to plan and construct creative and original
Japanese-style hotels, houses, event halls, etc. Furthermore,
using this system, the users can construct their own space
reflecting their feelings and can walk through the space and
exchange the interior fittings using PCs or WSs connected
to the Internet.
4. KRM
It is difficult to attain the desired design or data based on
user’s feeling or Kansei by using the key-word retrieving
and index retrieving methods. This is because the keywords and the indexing processes are carried out by
persons who register the database and they are influenced
by their subjectivity. In traditional Japanese crafting not
only the key-words and indexes but also the feeling process
is very important. For this reason, we developed a
Knowledge Agent (KA) to retrieve suitable designs based
on each user feeling.
For retrieving information, the KA uses a knowledgebase which is based on the relation between people feeling,
the presentation space, and the objects. In order to realize
this retrieval method, we need to find the relation between
Kansei word and the quantitative feature values. However,
it is difficult to find this relation directly. So, in this study,
we classified the feature as the sensitive feature and the
quantitative feature value. Then, we consider the sensitive
feature as the design pattern of traditional Japanese crafting
object and the quantitative feature values are processing
parameters extracted by VPIC.
We show the flow of Kansei retrieval method using the
feature of traditional Japanese crafting object in Fig 2.
When user requests a presentation space by using a Kansei
word, the CA sends a Kansei word to KA. When the KA
receives a Kansei word, the KA converts it to a sensitive
feature. The KA retrieves MDBs by using the quantitative
feature values corresponding to a sensitive feature and
sends the retrieved objects to the CA. Finally, retrieved
objects are put on the presentation space by CA. For
instance, when a user wants a “Calm” space, he sends a
request to its CA. The CA sends the “Calm” to KA. The
KA converts “Calm” to sensitive features such as
“Brightness is low”, “Density is rough”, and “Periodicity is
high”. After that, the KA transforms sensitive features into
parameters in order to be processed by the KA. Based on
the parameter values, the KA sends its query to multimedia
database and the required objects are retrieved. The
retrieved objects are represented in the presentation space
by the CA. Thus, a “Calm” space is presented to the user.
We select Kansei words, which are adjective words used
often in the interior design and the fitting design. We use
the relation between the Kansei word and the sensitive
feature of the fitting getting by the questionnaire [3]. The
relation between the Kansei word and the sensitive feature
of the fitting are shown in Table 1. The “Black” or
“Brown” in the sensitive feature corresponds to the color
and “Straight” or “Curve” to the line. The pattern includes
the density, the geometric pattern, the periodicity and the
size of pattern. These relations between the Kansei word
and the quantitative feature values are stored in the
knowledge base of KA.
Figure1: Digital Traditional Japanese
Crafting System (DTJCS).
Figure 2: Kansei Retrieval Method (KRM).
Table 1: Relation between Kansei words and
design patterns of fittings.
I❧q❝❣✣mp❜
❈❥❝❦❝❧r✣m❞✣❇❝q❣❡❧✣Nrr❝p❧
❇❝❧q❣r✇
Nrr❝p❧
❇p✐
❏❣❡❢r
❋❣❡❢
❏m✉
◗osp❝✣❧❜✣❋❝✈❡m❧
❊mp❡❝msq
◗❣❦n❥❝
❋❣❡❢
❏m✉
◗osp❝✣❧❜✣❋❝✈❡m❧
◗osp❝
❆❥❦
❅p❣❡❢r
❏m✉
❋❣❡❢
◗osp❝
◗osp❝
❙❧❣os❝
❘p❜❜❣r❣m❧❥
❏m✉
❏m✉
◗osp❝
❆m❥mp
◗❣x❝
N❝p❣m❜❣❛❣r✇
❏❣❧❝
❏p❡❝
❋❣❡❢
❏m✉
◗rp❣❡❢r
❆spt❝
❅❥❛✐
❢❣r❝
❏p❡❝
❋❣❡❢
❋❣❡❢
◗rp❣❡❢r
◗rp❣❡❢r
❅❥❛✐
❢❣r❝
❏p❡❝
❋❣❡❢
❋❣❡❢
◗rp❣❡❢r
◗rp❣❡❢r
❏p❡❝
❏p❡❝
❏m✉
❋❣❡❢
❆spt❝
◗rp❣❡❢r
❢❣r❝
❑m❜❝p❧
❅❥❛✐
❆❥qq❣❛
❏m✉
◗osp❝
❏p❡❝
❋❣❡❢
◗rp❣❡❢r
❢❣r❝
N❥❣❧
◗❥❝❧❜❝p
❏m✉
❋❣❡❢
◗osp❝
◗osp❝✣❧❜✣❋❝✈❡m❧
❏p❡❝
◗❦❥❥
❋❣❡❢
❋❣❡❢
◗rp❣❡❢r
◗rp❣❡❢r
❢❣r❝
❅❥❛✐
❋p❜
◗m❞r
❏m✉
❏p❡❝
❋❣❡❢
❏m✉
◗rp❣❡❢r
❆spt❝
❅❥❛✐
❢❣r❝
❙❧p❝❞❣❧❝❜
◗❦pr
Pms❡❢
❏p❡❝
❋❣❡❢
❏m✉
◗rp❣❡❢r
❆spt❝
❝❥❥m✉
❢❣r❝
◗rp❣❡❢r
❅pm✉❧✣❧❜✣Mp❧❡❝
❢❣r❝
p❦
❆mm❥
◗osp❝
◗osp❝
Pms❡❢
◗osp❝
❏p❡❝
❋❣❡❢
5. Extraction method for quantitative feature
values
In order to record the traditional Japanese crafting object
in MDB, the sensitive features are converted to the
quantitative parameter. To extract the quantitative feature
value including the pattern, we use VPIC[5]. To find
relation between the visual feature and the quantitative
feature values, we use the type and number of edge
patterns in the object.
The design of traditional Japanese crafting includes the
general pattern which recognizes the design from far sight
and the exquisite pattern which recognizes the design from
near sight. However, VPIC can encode not only the general
pattern but also the exquisite pattern by changing the
encoded block size. Furthermore, this method can extract
the quantitative feature values quickly by converting from
the VPIC edge to several edge patterns and expressing the
edge pattern as a vector with size and direction.
Table 2: Pattern relations.
The Pattern Components The Quantitative Feature Value
Density
Total Number
Periodicity
Frequency
Geometric Pattern
Size of Pattern
Dispersion Rate
Deviation Rate
5.1. VPIC
The edge pattern is extracted by dividing the encoded
block of an image as shown in Fig.3. There are two types
of pattern: the edge pattern and uniform pattern without the
edge. In the visual pattern the average of positive element
and negative element with encoded block becomes zero.
We use 8 types of edge patterns to represent 8 directions. In
the process of the edge pattern, we divide the encoded
block into 4 sub-blocks as shown in the left side of Fig.4.
Then, we calculate an average intensity in the encoded subblocks. Next, we calculate the difference of intensity in the
diagonal encoded sub-blocks v0, v1 based on the equations
(5.1) and (5.2). After that, we calculate the angle and the
size of edges between v0 and v2 based on the equation (5.3)
and (5.4). Finally, we convert the angle of edge to the edge
pattern as shown in the center of Fig.4 and decide the edge
pattern for the encoding block as shown in the right side of
Fig.4.
The difference of intensity for the diagonal encoding is
calculated by following formula.
v 0 = Ave3 − Ave0
(5.1)
v1 = Ave1 − Ave2
(5.2)
The angle of edge ∠ v and M are represented by the
following formula.
V =
( v 0 ) + ( v1 )
2
2
(5.3)
v 
∠V = tan −1 1 
 v0 
(5.4)
Encoded Block
Input Image
Encoded Block
Height
Encoded
Subblock
5.2. Expression method for pattern components
In order to represent the pattern components in the design
of traditional Japanese crafting, we try to find the relation
between the pattern components and the edge types using
VPIC as shown Table 2. The density is represented by total
number of edges. The periodicity can be represented by
frequency of edges because the regular pattern includes the
equivalent intervals in the space and non-regular pattern
includes different edge intervals in the same direction. The
geometric pattern is represented by the equivalent number
in each edge direction. When the geometric pattern consists
of square and is equivalent to the number in each direction,
the dispersion rate is high. But, the dispersion rate is low
when the geometric pattern has only circles. The size of
pattern is represented by the deviation rate of number of
edges when the encoded block size is changed. When the
encoded block size is small, a large size of edge pattern is
lost from the noise. While, when the encoded block size is
large, a small size of edge pattern is lost by averaging pixel
intensity.
6. Experimental results
In order to evaluate the relation between the pattern
components and the VPIC quantitative feature, we
extracted the VPIC quantitative feature from test images
including triangles, squares or circles shapes and then
analyzed the relation between the pattern components and
the VPIC quantitative features.
We analyzed the influence to the number of edges by
changing geometric pattern. In Fig.5 is shown the relation
between the number of square and the total number of
edges. The horizontal axis shows the number of square and
the vertical axis shows the number of edges. The result
shows that there is a proportional relation between the
number of geometric patterns and the number of edges.
Therefore, the density can be derived from the total
number of edges.
Edge Direction
2
3
1
v1
Extraction
of Edge Pattern
Ave0Ave1 v
x
4
0
Ave2Ave3
v0
5
6
7
Extracted Edge Pattern
Edge Pattern
2
3
1
4
0
25
Number of Edge(%)
Fig 3: Division of the image by encoding block.
y
2x2
4x4
16x16
32x32
64x62
128x128
256x256
30
Encoded Block Width
20
15
10
5
0
5
6
7
4
9
16
Number of Geometric Pattern
Fig 4: Extraction of edge pattern using VPIC.
Figure 5: Total number.
10
1
Dispersion Rate(%)
9
8
2
7
6
finding the relation between the design pattern components
and the edge types using the VPIC, the Kansei retrieval
method can be realized. Now, we are analyzing the relation
between Kansei words and quantitative feature values of
traditional Japanese crafting objects.
3
5
References
4
3
2
1
0
2x2
4x4
16x16
32x32
64x62
[1]
The Society of Industrial Promotion for Traditional
Japanese Crafting, “Traditional Japanese Crafting in
Japan”,
URL:
http://www.wnn.or.jp/wnncraft/w_index.html.
[2]
K.Sugita, A.Miyakawa and Y.Shibata, “VR Digital
Traditional Japanese Crafting System on Japan
Gigabit Network”, IPSJ (in Japanese), Vol.43, No.2,
pp.633-646, 2002.
[3]
A.Miyakawa, K.Sugita, M.Hosokawa and Y.Shibata,
“Statistical Analysis of the Relation Between Digital
Traditional Japanese Crafting Presentation and
Kansei Words”, IPSJ DPSWS’2001 (in Japanese),
pp.43-48, 2001.
[4]
K.Sugita, A.Miyakawa and Y.Shibata, “Relation
Between Kansei Words and the Room Space in
Digital Traditional Japanese Crafting System”, The
International Conference on Advanced Information
Networking and Applications (AINA), pp.159-162,
2003.
[5]
D. Chen and A.C. Bovic, “Visual Pattern Image
Coding”, IEEE Transactions on Communications,
Vol.38, No.12, pp.2137-2145, 1990.
[6]
D. Chen and A.C. Bovic, “Hierachical Visual Pattern
Image Coding”, IEEE Transactions on
Communications, Vol.40, No.4, pp.671-675, 1992.
[7]
M.Fukuda, M.Katsumoto and Y.Shibata, “Kansei
Retrieve Method based on User Model”, IPSJ DPS,
DPS-68 (in Japanese), Vol.95, No.13, pp. 43-48,
1995.
[8]
M.Fukuda and Y.Shibata, “Kansei Retrieval Method
Reflecting Shape Pattern of Design Images”, IPSJ
DPSWS’1996 (in Japanese), Vol.96, No.1, pp.267274, 1996.
[9]
S.Harada, Y.Itoh and H.Nakatani, “On Constructing
Shape Feature Space for Interpreting Subjective
Expressions”, IPSJ (in Japanese), Vol.40, No.5,
1999.
128x128 256X256
Block Size
Figure 6: Dispersion rate.
60
1
Frequency Rate(%)
50
2
40
21
30
22
20
31
10
32
0
2x2
4x4
16x16
32x32
64x62
128x128 256x256
Block Size
Figure 7: Frequency Rate.
In Fig.6 is shown the dispersion rate extracted by each
geometric pattern. The horizontal axis shows the block size
and the vertical axis shows the dispersion rate. The
dispersion rate makes difference for each geometric pattern.
The dispersion rate is low when the geometric pattern is
similar to the circle, because in the circle pattern the
number of edges in each direction is the same. But, the
dispersion rate is high when the geometric pattern is similar
to the square, because in the square pattern the number of
slant edges is small. While, the triangle has low dispersion
rate because slant line is expressed by not only the slant
edge but also the vertical edge in discrete coordinates.
Therefore, it can be concluded that the dispersion rate
corresponds to the geometric pattern.
We analyzed the influence of the edge frequency rate that
was extracted from the regular geometric pattern and the
irregular geometric pattern. In Fig.7 is shown the relation
between the regularity and the frequency rate of the edges.
The horizontal axis shows the block size and the vertical
axis shows the frequency. The solid lines show the
frequency rate extracted from the regular geometric pattern.
The dotted lines show the frequency rate extracted from
irregular geometric pattern. The frequency rate is high
when the geometric pattern is regular. But, the frequency is
low when the geometric pattern is irregular. By these
results, it is clear that the frequency corresponds to the
regularity of geometric pattern.
7. Conclusions
In this paper, we proposed a using VPIC for virtual reality
space presentation. We described Kansei retrieval method
and the quantitative feature values extraction method by
using VPIC. Then, we showed the result of extraction
using VPIC. The experimental results show that by
[10] T.Kurita, T.Kato, I.Fukuda and A.Sakakura, “Sense
Retrieve on a Image Database of Full Color
Paintings”, IPSJ (in Japanese), Vol.33,No.11,
pp.1373-1383, 1992.