Evaluation 2011: Values and Valuing in Evaluation
November 2-5, Anaheim, California
Conjoint analysis for Contract Strategy for Culture
Technology Enhancement Program in Korea
Yun Jong Kim*, Uk Jung **
E-mail: [email protected], [email protected]
*: KISTEP, Seoul, Korea, **: Dongguk University, Seoul, Korea
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Overview (I)
 As Culture Technology becomes one of the leading industries in its
contribution to the economy, many countries are increasing national
research investment through government-sponsored research projects.
 Thus it becomes important to measure the importance that research
participants attach to government-supported Culture Technology research
project attributes.
 Through some focus group discussions, a list of research project attributes for Culture
Technology was identified as important for participants for more successful project results.
 This study uses conjoint analysis based on survey results to show that there
is preferential difference in research project attributes for different affiliations
of participants.
 While the research was based on Korean experience, the research technique
can be generalized to research policy designs in other countries.
1
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea
 The concept of CT in Korea was introduced in July 2001. It was immediately
adopted by the Korean government as one of the next six core technological
engines for economic growth in the 21st century.
 The six technology (6T), high value-added technology-intensive industries expected to
lead Korea's economic growth, were appointed for intensive support by the Korean
government.
 Information Technology (IT), Biotechnology (BT), Nanotechnology (NT),
 Space Technology (ST), Environmental Technology (ET), Culture Technology (CT)
 CT is defined as “…technologies used in the value chain of culture content
from the planning, commercialization, loading to media platforms, to
distribution, and in a wide sense, complex technologies which are necessary
for enhancing the added value to cultural products, including knowledge and
knowhow from humanities and social science, design, and arts as well as
science and engineering”
 Similar concepts are Entertainment Technology and Creative Technology in
other countries.
2
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea : Movies
3
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea : Movies
4
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea : Games
5
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea : Screen Golf
6
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technologies in Many Countries
IRCAM (France)
MIT Media Lab(America)
3C Research(England)
ATR(Japan)
ITP(America)
7
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology
 Especially, CT is the most typical technology field which is requiring the
convergence and fusion of several technologies.
 Apple’s DNA
 It’s in Apple’s DNA that technology is not enough. It’s tech married with liberal arts and the
humanities. (2011. 3. 3. Keynote)
8
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Culture Technology in Korea
 Most government-supported CT research projects in Korea require different areas
of knowledge to be put together and also require many different experts to
work closely together.
 Different areas of knowledge include humanities, Korean studies, engineering, marketing
and management and so on. As the results, the research collaboration is a key
mechanism for both knowledge production and diffusion in CT.
 Thus, most CT research projects in Korea have basic attributes of inter-affiliated or
interdisciplinary research requirements for research collaboration.
 The main objective of this study is to explore how the research participants
feel about the collaboration issue with inter-affiliated or interdisciplinary
research requirements.
 For this purpose, this research uses conjoint analysis, which is appropriate in
measuring the importance level or utility that research participants attach to the
government-supported research project attributes including interaffiliated or
interdisciplinary research requirements.
9
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Conjoint Analysis (I)
 Conjoint analysis is often used to understand how consumers develop
preferences for products or services with multi-attribute levels in the
marketing field.
 Two basic assumptions are made in conjoint analysis.
 1st, a product or service can be described as a combination of levels of a set of attributes.
 2nd, these attribute levels determine the respondents’ overall judgment of the product or
service.
 The attraction of using conjoint analysis is that
it asks the respondents to make choices between products or services
defined by a unique set of product/service attributes in a way resembling
what they normally do-by trading off features, one against the other.
10
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Conjoint Analysis (II)
 Conjoint analysis produces two important results :
 ① Utility of attribute
 [It] is a numerical expression of the value the respondents place in an attribute level.
 It represents the relative “worth” of the attribute.
 Low utility indicates less value; high utility indicates more value.
 ② Importance of attribute
 [It] can be calculated by examining the difference between the lowest and highest utilities
across the levels of attributes.
11
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Attributes in Conjoint Analysis Model
 Attributes of inter-affiliated or interdisciplinary research requirements are our
main interest in this study.
 Inter-affiliated research means the research in which researchers from different affiliations
collaborate no matter whether researchers’ specialized research fields are same or not,
 while interdisciplinary research means the research in which researchers with different
research fields of specialization work together.
 To design the tradeoffs to represent the utility of inter-affiliated or
interdisciplinary research requirements,
we put into the government-supported research project attributes the
research fund size and period as important attributes in this study.
12
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Hypotheses
 We will explore the utility of different levels of attributes and the importance of
attributes the overall respondents perceived.
 Then those utilities will be investigated in different participants group.
 Using these results, we will try to address the existence of preferential difference
among different participants group on the government-supported CT research
program attributes with the following research hypotheses;
 Hypothesis 1.
CT research participants from different affiliations would have different
preference on the levels of public funding size and period.
 Hypothesis 2.
CT research participants from different affiliations would have different
preference on whether the inter-affiliated or interdisciplinary research was
required.
13
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (I)
 We set up the research project attributes as Research Period, Research Fund
Size, Inter-affiliated Research Requirement, and Interdisciplinary Research
Requirement, in order to study suitable supporting policies for CT.
Factor
Factor level
Short Period (< 1 yr)
Research Period
Medium Period (1~2 yrs )
3 levels
Long Period (3+ yrs)
Small Fund (< 100 million won)
Research Fund Size
Medium Fund (100~ 299 million won)
3 levels
Large Fund (300+ million won)
Inter-affiliated
Requirement
Research With Inter-affiliation
Without Inter-affiliation
Interdisciplinary Research With Interdisciplinary research
Requirement
Without Interdisciplinary research
14
2 levels
2 levels
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (II)
 Despite a careful selection of factors, there were still too many (3322 = 36)
possible profiles for the respondents.
 The SPSS generated a parsimonious orthogonal array of 9 profiles.
Profile
Research
Period
Research
Fund Size
Inter-affiliated Research Interdisciplinary Research
Requirement
Requirement
1
Medium
Small
No
Yes
2
Long
Small
Yes
No
3
Medium
Large
Yes
No
4
Short
Medium
No
No
5
Short
Large
Yes
Yes
6
Short
Small
Yes
Yes
7
Long
Medium
Yes
Yes
8
Medium
Medium
Yes
Yes
9
Long
Large
No
Yes
15
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (III)
 It was decided it would be useful to study the utilities perceived by the
following three different groups of affiliation participating in the research
project; 1) industries, 2) academia, and 3) government-supported research
institutes(GRIs).
 There were a total of 128 respondents.
<Breakdown of affiliations>
Affiliation
Counts
Industries
30
Academia
76
GRIs
22
16
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (IV)
 The relative importance levels of the various attributes are summarized.
 Overall researchers generally consider the research period as the most
important attributes, following by research fund size, inter-affiliation
requirement, and interdisciplinary research requirement.
<Graph of averaged importance of various attributes>
17
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 One-way ANOVA analysis with a Least Significant Difference (LSD) Test at the 0.05
significance level was performed to compare the preferences of researchers
from different affiliations.
<Comparison of means and ANOVA results for different affiliations>
Items
Constant
Short Period
Medium Period
Long Period
Small Fund
Medium Fund
Large Fund
With Inter-affiliation
Without Inter-affiliation
With Interdisciplinary Research
Without Interdisciplinary Research
[A]
[B]
Overall
Industry( Academia
(n=128)
n=30)
(n=76)
4.88
5.15
4.81
-0.37
0.14
-0.54
0.17
0.30
0.19
0.19
-0.44
0.36
-0.85
-0.90
-0.70
0.64
0.46
0.62
0.21
0.44
0.08
0.27
-0.18
0.41
-0.27
0.18
-0.41
0.08
-0.27
0.16
-0.08
0.27
-0.16
18
[C]
GRI
(n=22)
4.78
-0.45
-0.05
0.50
-1.30
0.95
0.35
0.38
-0.38
0.27
-0.27
ANOVA
significance
level
AB, AC
~
AB, AC
~
~
~
~
AB, AC
AB, AC
AB, AC
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 One-way ANOVA analysis with a Least Significant Difference (LSD) Test at the 0.05
significance level was performed to compare the preferences of researchers
from different affiliations.
<Comparison of means and ANOVA results for different affiliations>
Items
Constant
Short Period
Medium Period
Long Period
Small Fund
Medium Fund
Large Fund
With Inter-affiliation
Without Inter-affiliation
With Interdisciplinary Research
Without Interdisciplinary Research
[A]
[B]
Overall
Industry( Academia
(n=128)
n=30)
(n=76)
4.88
5.15
4.81
-0.37
0.14
-0.54
0.17
0.30
0.19
0.19
-0.44
0.36
-0.85
-0.90
-0.70
0.64
0.46
0.62
0.21
0.44
0.08
0.27
-0.18
0.41
-0.27
0.18
-0.41
0.08
-0.27
0.16
-0.08
0.27
-0.16
19
[C]
ANOVA
GRI
significance
(n=22)
level
No statistical difference
AB, AClevel
at4.78
0.05 significance
-0.45
~
-0.05
AB, AC
0.50
~
-1.30
~
0.95
~
0.35
~
0.38
AB, AC
-0.38
AB, AC
0.27
AB, AC
-0.27
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 One-way ANOVA analysis with a Least Significant Difference (LSD) Test at the 0.05
significance level was performed to compare the preferences of researchers
from different affiliations.
<Comparison of means and ANOVA results for different affiliations>
Items
Constant
Short Period
Medium Period
Long Period
Small Fund
Medium Fund
Large Fund
With Inter-affiliation
Without Inter-affiliation
With Interdisciplinary Research
Without Interdisciplinary Research
[A]
[B]
[C]
ANOVA
Overall
Industry( Academia
GRI
significance
(n=128)
n=30)
(n=76)
(n=22)
level
4.88
5.15
4.81
4.78
AB, AC
-0.37
0.14
-0.54
-0.45
~
0.17
0.30
0.19 difference
-0.05 in means
AB, AC
at the
0.19
-0.44
0.36 0.05 level
0.50 for industry
~ ([A])
-0.85
-0.90
-0.70
-1.30
~
and
academia ([B])
0.64
0.46
0.62
0.95
~
0.21
0.44
0.08
0.35
~
0.27
-0.18
0.41
0.38
AB, AC
-0.27
0.18
-0.41
-0.38
AB, AC
0.08
-0.27
0.16
0.27
AB, AC
-0.08
0.27
-0.16
-0.27
AB, AC
20
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 The one-way ANOVA results can be interpreted as follows:
<Comparison of means and ANOVA results for different affiliations>
[A]
[B]
[C]
Overall
Items
Industry( Academia
GRI
(n=128)
n=30)
(n=76)
(n=22)
Constant
4.88
5.15
4.81
4.78
Short Period
-0.37
0.14
-0.54
-0.45
Medium Period
0.17
0.30
0.19
-0.05
Long Period
0.19and GRI’s
-0.44 researchers,
0.36
0.50
Compared to academia
Small Fund
-0.90 constant
-0.70 value
-1.30
industry researchers-0.85
had a higher
Medium Fund
0.64
0.46
0.62
0.95
Large Fund
0.21
0.44
0.08
0.35
With Inter-affiliation
0.27
-0.18
0.41
0.38
Without Inter-affiliation
-0.27
0.18
-0.41
-0.38
With Interdisciplinary Research
0.08
-0.27
0.16
0.27
Without Interdisciplinary Research
-0.08
0.27
-0.16
-0.27
21
ANOVA
significance
level
AB, AC
~
AB, AC
~
~
~
~
AB, AC
AB, AC
AB, AC
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 The one-way ANOVA results can be interpreted as follows:
<Comparison of means and ANOVA results for different affiliations>
[A]
[B]
[C]
Overall
Items
Industry( Academia
GRI
(n=128)
n=30)
(n=76)
(n=22)
Constant
4.88
5.15
4.81
4.78
Short Period
-0.37
0.14
-0.54
-0.45
Medium Period
0.17
0.30
0.19
-0.05
Long Period
0.19
-0.44
0.36
0.50
Small Fund
-0.85
-0.90
-0.70
-1.30
Medium Fund
0.64
0.46
0.62
0.95
Large Fund
0.44
0.08 to 0.35
Industry researchers0.21
assigned
higher utility
With Inter-affiliation
0.27
-0.18
0.41
0.38
Medium research periods,
Without Inter-affiliation
0.18
-0.41
whereas researchers-0.27
from academia
and
GRIs -0.38
With Interdisciplinary
Research
0.08
-0.27
0.16
assigned
higher utility
to Long
research
periods 0.27
Without Interdisciplinary Research
-0.08
0.27
-0.16
-0.27
22
ANOVA
significance
level
AB, AC
~
AB, AC
~
~
~
~
AB, AC
AB, AC
AB, AC
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 The one-way ANOVA results can be interpreted as follows:
<Comparison of means and ANOVA results for different affiliations>
[A]
[B]
Overall
Items
Industry( Academia
(n=128)
n=30)
(n=76)
Constant
5.15
4.81 to
Industry researchers4.88
assigned
lower utility
Short Period
-0.37 Requirements,
0.14
-0.54
Inter-affiliation Research
Medium Period whereas researchers0.17
0.30
0.19GRIs
from academia
and
Long Period
0.19
-0.44
0.36
assigned a higher utility.
Small Fund
-0.85
-0.90
-0.70
Medium Fund
0.64
0.46
0.62
Large Fund
0.21
0.44
0.08
With Inter-affiliation
0.27
-0.18
0.41
Without Inter-affiliation
-0.27
0.18
-0.41
With Interdisciplinary Research
0.08
-0.27
0.16
Without Interdisciplinary Research
-0.08
0.27
-0.16
23
[C]
GRI
(n=22)
4.78
-0.45
-0.05
0.50
-1.30
0.95
0.35
0.38
-0.38
0.27
-0.27
ANOVA
significance
level
AB, AC
~
AB, AC
~
~
~
~
AB, AC
AB, AC
AB, AC
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (V)
 The one-way ANOVA results can be interpreted as follows:
<Comparison of means and ANOVA results for different affiliations>
[A]
[B]
Overall
Items
Industry( Academia
(n=128)
n=30)
(n=76)
Constant
4.88
5.15
4.81
Short Period
-0.37
0.14
-0.54
Medium Period Industry researchers0.17
0.30
0.19 to
assigned
lower utility
Long Period
0.19
-0.44
0.36
Interdisciplinary Research
Requirements
Small Fund
-0.90
-0.70
whereas researchers-0.85
from academia
and
GRIs
Medium Fund assigned higher utility.
0.64
0.46
0.62
Large Fund
0.21
0.44
0.08
With Inter-affiliation
0.27
-0.18
0.41
Without Inter-affiliation
-0.27
0.18
-0.41
With Interdisciplinary Research
0.08
-0.27
0.16
Without Interdisciplinary Research
-0.08
0.27
-0.16
24
[C]
GRI
(n=22)
4.78
-0.45
-0.05
0.50
-1.30
0.95
0.35
0.38
-0.38
0.27
-0.27
ANOVA
significance
level
AB, AC
~
AB, AC
~
~
~
~
AB, AC
AB, AC
AB, AC
AB, AC
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (VI)
 Hypothesis 1.
CT research participants from different affiliations would have different
preference on the levels of public funding size and period.
 One can say that CT research participants from different affiliations have
different preference on the levels of public funding period.
 Industry researchers prefer medium research period to short or long period, whereas
researchers from academia and GRIs prefer long research periods.
 However, as for research fund size, there are no significant differences in
preference among research participants from different affiliations.
25
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Empirical Study (VII)
 Hypothesis 2.
CT research participants from different affiliations would have different
preference on whether the inter-affiliated or interdisciplinary research was
required.
 Industry researchers are reluctant to participate in inter-affiliated and
interdisciplinary research projects, whereas researchers from academia and
GRIs prefer them.
 These barriers that industry researchers feel about the collaborations are
discernible from the analysis regarding inter-affiliation and interdisciplinary
research requirements.
26
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Conclusion
 This research illustrates the usefulness of conjoint analysis in determining
the utility values of government-supported CT research project attributes.
 The study demonstrates how evaluators can use this research technique to
reveal and measure the hidden needs of participants.
 The segmentation of participants into different groups according to affiliation
has many practical applications.
 With other research techniques such as cluster analysis and multidimensional scaling, using conjoint analysis offers extremely interesting
academic and evaluating research opportunities.
27
【 Conjoint analysis for Contract Strategy for Culture Technology Enhancement Program in Korea 】
Thank you very much!!