Some Notes on Utility of Information and Communication Technologies
Nekaj izhodišč za študij koristnosti in uporabnosti
informacijskih in komunikacijskih tehnologij
Tomaž Turk
University of Ljubljana, Faculty of Economics, Kardeljeva pl. 17, 1000 Ljubljana, Slovenia,
Tel: +386 1 5892400, Fax: +386 1 5892698, [email protected]
Abstract
In this paper we focus on the ICTs adoption framework (the term ICT - information and communication
technologies - is understood in a broad sense, from user interfaces to complex devices), developed to investigate
the relationships between ICTs and users, and the comparison of this framework to the Sen's capability
framework, which introduces the well-being approach. A combination of both frameworks can give us the
toolset to explore the reasoning behind the adoption and usage patterns of ICT devices and services, and users'
decision processes. This toolset is based on the concept of well-being, as a common denominator of the
evaluation of choices users are doing every day, achieving their "being". We also discuss the definitions of
terms such as utility, usability and usefulness, which are used in the research of a quality of user interfaces
nowadays. The question of a definition of utility is especially interesting, since in this paper we combine several
disciplines, microeconomic analysis and the technological aspects of the quality of human interfaces. Both
disciplines use the term utility, but define it in different ways. We also introduce some basic models of human device interactions and their combinations, which can be the basis of the system dynamics approach in further
research work, but need some empirical tests of relevance.
Keywords: ICTs, adoption, user interface, utility, usability, well-being
Povzetek
V pričujočem zvezku soočamo dve ogrodji, ki nam omogočata razumevanje privzemanja informacijskih in
telekomunikacijskih izdelkov s strani uporabnikov. Prvo ogrodje (angl. ICTs adoption framework) je bilo
razvito za razumevanje odnosa naprava - uporabnik, drugo pa se nanaša na Senovo delo, njegov t.i. "zmožnostni
pristop" (angl. capability approach), kjer opusti klasično razumevanje koristnosti, saj jo opredeli širše, namreč
kot "dobrobit" (angl. well-being). Kombinacija obeh ogrodij in pristopov nam lahko služi kot orodje, s katerim
lahko pojasnimo dogajanje pri odločanju uporabnika o uporabi naprav, ki sodijo v t.i. informacijske tehnologije.
Dobrobit (v mikroekonomskem smislu v pomenu koristnosti) je uporabljena kot skupni imenovalec, uporabljan
vsakokrat, ko se uporabnik odloča o določenem ukrepu v zvezi z uporabo informacijske tehnologije (npr. pritisk
na gumb, nakup nove naprave). V zvezku so prikazani tudi nekateri modeli, s katerimi je možno opisati razmere,
v katerih se uporabnik odloča o določenem ukrepu.
Ključne besede: informacijska tehnologija, privzemanje, uporabniški vmesnik, koristnost, uporabnost, dobrobit
1.
Introduction
The relationship between people and different kinds of commodities has been the most
intriguing question for social scientists and economists for ages. Despite the long history of
the research work about this relationship there are still a lot of interesting issues and every
day new technological possibilities are creating new issues on design, adoption and ethics, to
name just a few of them. In this paper we focus on the adoption process of modern ICTs,
more precisely on the way people see new devices, the value they give to new services and
how they feel about their achievements.
In the search for better development and design methodologies and approaches, several
authors are focusing on new proposals for frameworks and models to assess the usability and
quality of different information and communication technologies (ICTs). In several published
books, papers and presentations various definitions and notions of terms usability, usefulness,
functionality and utility are mentioned with different interrelations among them (Nielsen,
2000; Pribeanu, 2002; Kragelj, 2003). Despite the fact that a number of models for usability
assessment are being proposed, it seems that a clear understanding and definitions of these
terms and concepts are not clear yet. Some models and definitions are not consistent with
each other and even worse; some of them are also self-contradictory.
This paper is not meant to be a critique of the previously proposed models, which can be
accepted (in the author's opinion), if they give appropriate metrics to their users and uses, but
is rather giving a "one step back" theoretical view on the values users of ICTs really
appreciate. Generally, this question is the most intriguing one, and it has been investigated
throughout the history of economic research. In economic studies, the reasoning behind
human behaviour has mostly been explained by the concept of utility, the common
denominator of all the values behind the users' choices. But historically, this concept was
mainly used in economic studies such as the investigation of demand for goods and services
and of consumer behaviour, deviating the concept from the real nature of utility, which
should measure well-being.
"The trouble with traditional utilitarian notion of welfare is that it can render only a partial
picture of human well-being. In fact this conception relies only on the welfarist criteria of
utility (in theory) and income (in application). The need to move towards such a broader
notion of well-being has been strongly advocated, among others, by Amartya Sen, whose
major contributions all stress the centrality of individual entitlements, opportunities, and
rights as conceptual foundations of economics and social choice. Sen has in fact gradually
developed an approach focused on the freedom of individuals to pursue their own project of
life, in which well-being is seen 'in terms of a person’s ability to do valuable acts or reach
valuable states of being' (Sen, 1993). This is the core of the so-called capability approach."
(Grasso, 2002).
Grasso (Grasso, 2002) continues his work with the operationalization of the Sen's capability
approach with the system's dynamics methodology and simulation. It is the author's belief
that the capability approach can be exploited for different levels of human projects,
depending on different dimensions like space or environment (e.g. person's project vs. project
of society) and time (e.g. person's project of a lifetime, project of a day, project of a particular
moment). The purpose of this paper is to give a speculation of a different operationalization
possibility of Sen's approach, the one which would give us a possibility of aggregation of
human well-being on different levels, and thus to reveal the true meanings of usability,
usefulness, functionality and utility, which are the many reincarnations of well-being goals on
different levels of human activity and courses of action. The Sen's capability approach and
operationalization will be illustrated with cases from ICT usage and development.
Firstly, we focus on the framework of ICTs adoption, developed to investigate the
relationship between ICTs and users, and the comparison of that framework to the Sen's
capability framework, which introduces the well-being approach. Secondly, the examination
of this framework introduces basic models of human - device interactions and their
combinations.
2.
People and commodities, users and ICTs
2.1. The ICT adoption framework
The ICT adoption framework was developed by Heres and others (Heres et al., 2002). The
framework tries to give a structure to the factors influencing adoption. The structure is based
on two theses:
1) In order to adopt an ICT there should be a ‘fit’ between the ICT and the
potential user (person). This fit depends on the specific ICT-characteristics of the
ICT on the one hand and the specific characteristics of the person on the other
hand.
This first thesis implies that factors influencing adoption could be divided into three
categories:
-
The fit between ICT and user (direct influence).
-
The user (indirect influence).
-
The ICT (indirect influence).
2) In order to adopt an ICT there should be a need or a necessity for the user to
adopt the ICT on the one hand and an absence of relevant barriers to adopt the
ICT on the other hand.
The second thesis implies that factors influencing adoption could be divided into two new
categories:
-
The user has a need or necessity to adopt the ICT.
-
It is easy for the user to adopt the ICT (ability or absence of barriers).
The framework is best described with the schematic as on Table 1.
Wants
Possibilities
Person
Personal preferences and
lifestyle
Personal abilities
ICT — person
Person wants to adopt the ICT:
perceived need or necessity
Person has the possibility to
adopt the ICT:
perceived easiness to adopt ICT
ICT
ICT functionalities
ICT constraints
Table 1: Adoption of ICT, a two-dimensional framework (Heres et al., 2002)
Different factors which influence the adoption process, such as price, user friendliness,
knowledge, etc., can relatively easily be placed into the right cell of the table. A more
intriguing question is: how does a particular factor influence the adoption? Is it favourable,
non-favourable or mixed? Consider, for instance, the price of a particular device. Combined
with the income of a person, it can present a barrier, but on the other hand it can present a
luxury. The framework can give a deeper insight into different connections between factors,
and many of those interactions can be researched in interdisciplinary fashion.
Important to note is also the test whether the framework could deal with two characteristics,
which are used nowadays in the research and design of a user interface:
-
Utility - "will it do what is needed functionally?" - functionality of the system required
for the task; is there a need for such a system?
-
Usability - "will the users actually work it successfully?" - the ability to use the utility
(Gaines et al., 2003)
These two characteristics were noted and defined as above by Nielsen (Nielsen, 1993).
Keates and Clarkson (2001) commented from his work that the system acceptability is
achieved by meeting the social and practical acceptability objectives for the system.
Usefulness, constituting usability and utility, is then perceived as the key objective for
providing practical acceptability. There are also other characteristics, like likeability,
effectiveness, learnability, flexibility and attitude, mentioned by others (see, for instance,
Gaines et al., 2003).
Nevertheless, one is wondering whether there is a common denominator, with which we can
in some way explore the person's decision process, which factors are considered as important
and of the valuation process behind the actual choice. Some types of choices a user should
deal with can be described as follows (Heres et al., 2002):
-
Whether to buy an ICT product or not, or to subscribe to a service or not
-
Which brand of an ICT product or a service to buy
-
Which ICT product or service to use in a particular situation
-
Which bundle of products and services to buy
-
The amount of products to buy and services to use.
In economic studies, questions about consumer behaviour are often examined with the
concept of the utility maximization. In fact, the core set of microeconomic laws is
investigated under this concept (see, for instance, Varian, 1992). According to that, one's
utility function is often a very convenient way to describe their preferences, but it should not
be given any psychological interpretation. The only relevant feature of a utility function is its
ordinal character. The utility evaluation performed by consumers should give them the clear
picture of which option would give them higher utility. If this is the only factor for making
the decision, what about usability? On the other hand, is the microeconomic analysis wrong,
because it stresses only utility?
Clearly, utility is not understood and defined in the same way in different disciplines. This
holds true even within the economic studies. This problem is described and addressed in the
works of Sen (Sen, 1999). Being in a way disappointed with different meanings of utility,
mostly coming from different contexts of scientific research, he proposed a different
approach and speaks not of utility but of well-being.
2.2. Sen's capability approach
Utility was firstly used by utilitarian economists. Utility has been seen as satisfaction or
happiness (classical utilitarianism) or as desire-fulfilment (modern utilitarianism). Nowadays,
the term utility serves for other purposes as well, standing for whatever the person
maximizes. Sen also comments that "... mathematical exactness of formulation has proceeded
hand in hand with remarkable inexactness of content." (Sen, 1999). In his work a person's
motivation behind the choice is treated as a parametric variable of well-being, which is
connected to that person's achievement. "The various interpretations of utility can be seen as
different ways of representing well-being (and they have indeed been seen in these ways)".
Sen's approach is used in many ways, for instance for the inter-regional comparisons of living
standard (Grasso, 2002). The approach gives a general framework for a possible exploration
and is intentionally left without notions of an exact model. Models should be developed by
researchers in different fields and aspects (contents), but with a common definition and
understanding of the well-being concept.
It is not the purpose of this paper to go deeply into the arguments and the interpretation of
well-being, as the reader can find those in published works by Sen. Our purpose is to show
the similarities between two frameworks, Sen's capabilities framework and the ICT adoption
framework, as described above.
Sen suggests that well-being should be considered in terms of human functionings and
capabilities. "Functionings relate to what a person may value doing or being: they are the
living conditions achieved by an individual and represent a set of interrelated activities and
states ("doings" and "beings") that form her life. Capabilities concern the ability of an
individual to achieve different combinations of functionings, and define the freedom to
choose the life that she prefers." (Grasso, 2002).
A formal description, given by Sen, is as follows:
xi
the vector of commodities
c(.)
the function converting a commodity vector into a vector of characteristics of those
commodities,
fi(.)
a personal utilization function of i reflecting one pattern of use of commodities that i
can actually make (functionings)
Fi
the set of utilization functions fi, any one of which person i can in fact choose
If the person chooses the utilization function fi(.), then with his commodities represented by
the vector xi, the achieved functionings will be given by the vector bi:
bi = fi(c(xi))
If vi(.) is the valuation function of person i, then the value of that vector of functionings bi is
given by
vi = vi(fi(c(xi)))
We can see that the commodity is seen as a set of characteristics, which are given by a
commodity and the present state of technology. The utilization function brings into the view
the personal and social conversion factors. The valuation function measures the well-being as
percept by a person.
commodities
commodity characteristics
personal and social
conversion factors
Fig. 1: The capability approach.
2.3. A comparison of frameworks
We can clearly see that there are similarities between both frameworks. If we consider the
capability approach as the basis for the research of ICT adoption phenomena, the same
elements can be seen in both frameworks:
-
a person investigating their needs, ICTs and possibilities
-
ICTs with their characteristics, which are not important without a user context or without
personal and social conversion factors
-
the valuation of the achievement by a person.
The adoption is not seen as a discrete event in a particular moment, but as a process within
both frameworks. ICT characteristics and the social and personal views are the key
components of the evaluation process.
If we accept Sen's capability approach, two achievements can be reached. Firstly, we can get
a clear understanding of different meanings of utility. The utility as a common denominator
of decision-making, as described according to the ICTs adoption framework, can in fact be
described as a well-being maximization problem. Utility as in "a functionality of the system
required for the task" is only one of possible interpretations.
If we follow the dynamics of consumers’ decisions according to the ICTs adoption
framework, they firstly consider their needs or wants. This is the starting point of the decision
process, when consumers ‘investigate’ their needs. The next step is the evaluation of their
options. The common denominator or ‘measure’, which is used in this decision process, is
well-being. Consumers try to predict their well-being deriving from functionalities which
emerge from the product and its characteristics, social and personal conversion factors by
exploring each factor among their needs and possibilities.
Both views can enrich our analysis in two ways:
1) We can refine the definitions of utility and usability, as used in user interface studies,
taking into account different contents of the term "utility".
2) We can further create the toolset for investigation of the relationships among different
factors, which influence the ICT adoption process. Due to the broad definition of well-being
emotions can also be investigated with the help of the framework.
3.
Well-being and the decision process
3.1. Evaluation of options
The basic mathematical formulation of the decision process can be expressed similarly as in
(Heres et al., 2002). Here we are using the term "well-being" instead of "utility" to make a
distinction between two different contexts of "utility".
If a consumer chooses between two similar ICT products, services or even his courses of
action, we can mathematically represent his decision process as:
V1 > V2
where V1 is the value of the first product according to the capabilities approach (his
evaluation of well-being) and V2 is the value of the second product. In this case, where the
value of the first product is greater than the value of the second product, the consumer
chooses the first product. Similar, when the consumer wants to replace an old product for a
new one, he considers their values and he might buy a new one, if he finds:
Vnew ICT > Vold ICT
We can state that the consumer adopts a specific ICT as a tool when the value gained from
the usage of a new tool is higher than the value emerging from the usage of the existing (old)
tool. When a user does not own a product (but considers buying one), he or she tries to get
some information about it and in this way tries to estimate the utility that could be gained
from it. The above formula becomes
V'new ICT > Vold ICT
V'new ICT is expected value, estimated according to the user's knowledge of the new ICT and
his or her experiences. How do users estimate the value? They consider different characteristics of it:
VA = kpVp(pA) + knVn(nA) + kwVw(wA)
(1)
where A is particular product, p, n and w are characteristics, or attributes of A, and k are
weights. With weights we can express the different degrees of importance of particular attributes for different people. Someone considers the colour of an ICT, while for another user it
is more important for an ICT product to have an infrared communication port. The overall
value is a sum of partial values, which are derived from different attributes. Every attribute
has its own conversion function V. The weights can be negative, thus expressing nonfavourable influences.
It is interesting to consider different courses of actions, which users make with ICT products.
For instance, if a user has a mobile phone and also a fixed one, which one will he choose to
call a friend? The user compares the value of different means of satisfying his wants.
Some other notions should also be considered in this framework:
a)
There is a distinction between achieved well-being and expected well-being. If we
look at the decision process dynamically, users are firstly trying to estimate their well-being
after the selection before the selection is made with all of its consequences. Thus, expected
well-being is the basis for the decision. Their estimation is corrected a posteriori, after their
decision while using the chosen ICT device as they gain certain experience, which is then
used in further estimations and probably shared with others.
b)
The importance of information. People decide between known options. If an option
is not known to a user, but is technically possible to achieve, this option cannot be considered
as relevant in their decision process in the time being. The quality of information is also
crucial for their decision.
c)
People tend to maximize their well-being in time. They are prone to lower their
well-being in the short run, if such action presented a possibility of gaining a higher level of
well-being in the future.
d)
Estimation of well-being means effort. The process of decision making about
different options is an additional effort and a time consuming activity, which is "invested"
into gaining additional knowledge and experience. Sometimes spending time and effort on
making the decision is not worth the amount of gained well-being. Here emotions come
forward as well, thus for dropping the postulate of rational behaviour.
e)
"Doing" is also knowledge gaining. When making a choice, taking a specific course
of action or while using a particular commodity, people are gaining knowledge and
experience, which can be used in decision-making processes in the future, while repeating the
action or usage in the future (thus spending less time with it), and it can be also shared with
others.
f)
Two faces of time. Normally, time is a scarce asset, which is used for taking courses
of actions, making decisions, gaining information, etc. (Making a decision is also taking a
specific course of action on a higher level!) People tend to minimize the time interval
invested into a particular action, unless this represents their exact wish or desire.
3.2. Some basic models
If the above framework is to be used to "measure" the well-being, we can soon come to a
somewhat dissapointing conclusion that well-being cannot be measured. There is a
possibility, however, of expressing the change or the difference in the level of well-being of a
person. The levels of well-being can be compared in time, between the state of a person
before and after making a choice. Nevertheless, the metric should always be carefully
developed when collecting the necessary data (by using questionnaires, for instance).
We can start our exploration with a relatively simple model. Let us assume we have a person
with a choice between two different courses of action, a and b. Courses of action can be
interpreted as using different elements of a user interface. Each action will take different
periods of time (ta and tb), but other resources of the person are equally "consumed" with both
actions (the necessary levels of knowledge, skills, experiences and other resources are the
same for both), and also the gain of well-being is the same for both actions (the person will
achieve a certain goal, and the goal is the same for both actions).
va ,b 
tb
ta
With va,b we denote the relationship between two states of well-being of the person.
According to the model, if va,b is higher than 1, the preferred option is a. Here we should once
more stress the important fact that other factors besides the time consumed for taking the
course of action are equal for both actions, and it is preferred by the user to spend as little a
time interval as possible performing that action.
If we consider the chance that a person will need to gain some additional knowledge for both
actions, we can further expand the above model:
va ,b 
t b  kb
ta  ka
We can see that the model already includes the possibility of two different levels of
knowledge for each action (a and b). We expressed the knowledge gaining effort with only
one resource of the person, which is time in the above model. If this additional knowledge
will give to the person the opportunity to spend less time in repeating the same course of
action several times, than we can say that the time for knowledge gaining is properly
"invested":
m
va ,b 
kb   tb , j
j 1
n
(2)
ka   ta ,i
i 1
Of course, when making a decision, it is hard to predict how many times a particular action
will be performed (m and n). Nevertheless, if the time investment for knowledge gaining is
relatively small and different for both options, the decision is relatively plain.
If a person chooses between more options, the preference order of options can be formulated,
although there are possible weaknesses of this approach (Sen, 1999). More complications are
also possible, such as the person considering taking one course of action, gaining experiences
with that option, and not even considering getting some knowledge of the other option. This
can be modelled in our framework, using the above notions about the importance of
information, the effort for well-being estimation and others.
If we consider two possible designs of a web page as a system for achieving a higher level of
well-being, a user will choose the web page according to the model, which can be written in a
similar way as (2), denoting the usability in higher amount of time spent to achieve the same
result.
The quality of visual appearance of a web page can be included into the model as follows:
m
va ,b  wt
kb   tb , j
j 1
n
ka   ta ,i
/ wg
gb
ga
i 1
where g are grades for appearance given to the user interfaces by the user, where a higher
grade means a higher evaluated visual design. For different factors different variables should
be used. They can either be continuous or discrete with an ordered set of values (ordinal
scale). When we deal with different resources or assets, different weights should be used for
each user. We used wt for denoting the importance of time and wg for the importance of
appearance in the above model.
There is another important property of the above model: the factors, which are obviously
presented as spending of resources and assets on one hand and achieving other goals (like
enjoying a nice design) on the other, can be favourable or non favourable to either possible
course of action (a or b). From this observation we can see the difference between
multiplication and division of factors within the model.
If we consider the time dimension, a comparison of two courses of action (or usage of a
particular device) can be represented by the general model:
m
j 1
x n
va ,b  w
m
x
b, j
x
i 1
a ,i
w
m
y
j 1
y n
b, j
y
i 1
z
j 1
z n
/w
a ,i
b, j
z
i 1
(3)
a ,i
Here x, y and z are factors, assets and desirability of a person, n and m are numbers of
repetitive spending or gaining of a particular asset or desirability, and w denotes weights
(importance of factors). We can see from the above that a higher value of the factor z is
favourable to the choice a.
The model (3) is somehow manageable, should we wish to examine the decision process in
practice, e.g. with questionnaires and with live experiments. A more complex form could be
drived directly from (1):
v A, B 
k pV p ( p A )  knVn (nA )  kwVw ( wA )
(4)
k pV p ( pB )  knVn (nB )  kwVw ( wB )
A graphical representation of the two choices and of the personal value of these choices in
time is represented in Fig. 2. Here, the expected well-being of choices a and b is compared
with the level of well-being at present state at the time p. Option a means a lower level of
well-being in the short run, but better prospects in the future. We should stress once more that
while deciding about different options, a person considers this to be a task, and before
examining their options they are making a choice between making the evaluation and not
bothering at all. Sometimes, the gain in the level of well-being, coming from making the right
decision can be lower than the effort spent on the proper evaluation.
The uncertainty of the future is represented graphically in Fig. 3. If a person chooses the
wrong option, the level of well-being does not change according to his prediction. A good
example of this would be, for instance, a user who chooses a more sophisticated user
interface and then gives up, because they feels incapable of using it.
v
va,p
vb,p
1
p
t
Fig. 2: Personal value of choices a and b in time.
v
va,p
1
p
t
Fig. 3: Uncertainty of the outcome from choosing an option.
If we compare our models with the capabilities framework, the model (3) represents
evaluation and comparison of two functionings a and b, which can be achieved by a person
when choosing one of the options. The functionings are firstly derived from characteristics of
different commodities (or ICT devices) and then converted according to the personal and
social conversion factors. These factors have different weights and they can either be
favourable or non favourable to an option. The achieved functionings are strongly related to
the state of a person, or their level of well-being.
4.
Conclusion
In this paper we have made a comparison of two frameworks, which try to help in explaining
the relationships between people and commodities. One framework is a more general
approach, while the other is ICT-oriented. We find both frameworks compatible and possible
theoretical cornerstones for empirical research about the ICT adoption process, the design of
user interfaces and other processes, which take place behind human - ICT relationships.
We believe that the influences of different factors on usage and adoption of ICTs are very
complex and difficult to research, but one can always entertain the idea of developing a
framework and models, which could give us a usable toolset to explore and understand the
reasoning behind decisions.
We choose the concept of well-being as the category that is the desired property behind
someone's project of a lifetime, the project of a day or even the project of a moment. When
considering different contexts this can be seen as utility, as defined by microeconomic
analysis (Varian, 1992), but to avoid possible misunderstandings, the well-being concept,
introduced by Sen (Sen, 1993), is wider and more clear. Usefulness, utility, usability,
efficiency and other categories in the quality of user interface context can be understood and
defined within our framework. We can illustrate this with an example. Let us say that a user
chooses a certain goal (they want to book a hotel room, for instance, because they expects
that this will somehow increase their level of well-being). They have two options, (reserving
the room with a phone call or through a web-site interface). Both options have utility (they
both have characteristics that represent a certain functionality required for the task). The
decision factors, together with their importance and favourability to either option and, of
course, user's assets and information are then important for their final decision. One can
easily construct an evaluation model with some typical categories, such as time for usage,
time for gathering necessary knowledge and skills, other sub-tasks (getting a phone
connection or turning the PC on), etc.
In this stage of our research our models (3) and (4) should be considered more as a possible
instruments when trying to deal with complex interactions between their elements. Future
research work on this toolset could incorporate the system dynamics approach and should be
tested empirically for relevance. Further exploration of the importance of emotions on human
behaviour is also interesting.
5.
References
Deaton, A. and Muellbauer, J., 1996, Economics and consumer behavior. Cambridge
University Press, ISBN 0-521-29676-5.
Gaines B. R., Shaw M. L. G. and Chen, L. L., 2003, Utility, Usability and Likeability:
Dimensions of the Net and Web. [http://ksi.cpsc.ucalgary.ca/KSI/]
Grasso, M., 2002, A dynamic operationalization of Sen’s capability approach. Working
Paper. Universita degli Studi di Milano.
Heres, J., Mante-Meijer, E., Turk, T. and Pierson, J.: Adoption of Information and
Communication Technology — A Proposed Framework. COST 269: Capabilities in
Action — What People Do. Final report. [http://www.cost269.org/]
Keates, S. and Clarkson P. J., 2001, Combining utility, usability and accessibility methods for
Universal Access. ACM CHI Workshop on Universal Design, Seattle, USA
Kragelj, B, 2003, Usabiliy of Web Site: What Does It Mean? Dnevi slovenske informatike,
16-18 april 2003. Zbornik posvetovanja. Slovenian Society Informatika, ISBN 961-616514-3.
Nielsen, J., 1993, Usability Engineering. Morgan Kaufmann Publishers, Inc, San Francisco,
CA.
Nielsen, J., 2000, Designing Web Usability. Indianapolis: New Riders Publishing.
Pribeanu, C., 2002, Towards a framework for the evaluation of web sites intended to support
on-line communities. COST 269, Montegrotto Report.
Sen, A. K, 1993, Capability and Well-being, in The Quality of Life (edited by Nussbaum, M.
and Sen, A. K.), pp. 31-53. Oxford: Clarendon Press.
Sen, A. K, 1999, Commodities and capabilities. Oxford: Oxford University Press.
Varian, H. R., 1992, Microeconomic Analysis. New York: W. W. Norton & Company, Inc.