Proceedings of Applied International Business Conference 2008
ASSESSING INFORMATION SYSTEMS SUCCESS THROUGH ORGANIZATIONAL
CONTEXT: A CONCEPTUAL DEVELOPMENT
Zakariya Belkhamza ψ and Syed Azizi Wafa Syed Khalid Wafa
Universiti Malaysia Sabah, Malaysia
Abstract
While the world of business and information systems is moving out increasingly in faster velocities,
organizational forms are changing and there is a need for a deep investigation on the linkage between
organization culture and information systems assessment, which lead to organizational business
effectiveness. Issues related to organizational culture appear frequently in discussions of information
systems success. The statement that the information system did not fit the organizational culture is
often part of the explanation of why particular information system encountered unanticipated resistance
and never met expectation. This paper proposes a theoretical development that attempts to investigate
this issue. The objective of this paper is to provide a new insight of this linkage and will look at the
success of the information system from the context of the organizational structure, in which it affects
the performance of the organization. This study explores the concept of organizational context
dimensions of discipline, stretch, trust and support as the shapers of organizational culture, and relates
them to the information systems research, focusing on the information systems success as captured by
systems performance, information effectiveness and service performance. The paper provides
theoretical justifications that aim to contribute to the integration of two streams of research on
information systems assessment that have emerged in the organizational and management literature.
Keywords: Managerial action; Behavioural context; Organizational context; Ambidexterity;
Information systems success.
JEL Classification Codes: O32; M14, M15.
1. Introduction
While the world of business and information systems is moving out increasingly in faster velocities,
organizational forms are changing and there is a need for a deep investigation on the linkage between
organization culture and information system, which lead to organizational business effectiveness.
Issues related to organizational culture appear frequently in discussions of information systems success.
The statement that the information system did not fit the organizational culture is often part of the
explanation of why particular information system encountered unanticipated resistance and never met
expectation (Alter and Shaw, 2003). This paper proposes a theoretical development that attempts to
investigate this issue. The objective of this paper is to provide a new insight of this linkage and to look
at the success of the information system within the context of the organizational structure, in which it
affects the performance of organization. This study explores the concept of organizational context
dimensions suggested by Ghoshal and Bartlett (1994) as the shapers of organizational culture, and
relates them to the information systems research, focusing on the information systems success,
particularly investigating the role of the strategic managerial action affecting this linkage, as illustrated
in Figure 1.
The main problem behind this paper is that information systems success is not approached from
organizational perspective, in position to the technical, the strategic or the emergent approaches, which
have been dominated the discipline so far. Checkland and Holwell (1998) argue that the whole process
of information systems implementation is in fact a process of organizational change. From this, it
follows that information systems implementation could be constructed to be a process of change where
a key criterion of an information system and organizational context is integrated, which shows the
merging of IT-based systems into organizations. However, information systems management literature
shows that a redefinition of the information systems function in organizations is well in process, but
ψ
Corresponding author. Zakariya Belkhamza. School of Business and Economics, Universiti Malaysia
Sabah, Teluk Likas, 88999 Kota Kinabalu, Sabah, Malaysia. Corresponding author Email:
[email protected]
Proceedings of Applied International Business Conference 2008
there is an observed lack in the integration of information systems discipline and the organizational
culture.
Managerial
Action
Organizational
Context
Organizational
Change
Information
Systems Success
Information systems Function
Figure 1: Theoretical concept of the study
As waves of new information systems applications find their ways into organizations, which are used
by increasing number of staff in increasing numbers of organizational tasks, the structure of the
technology are infused into social structure of the organization (Orlikwoski, 1992; DeSantics and
Poole, 1994). Similarly, information systems infusion must not be taken to mean the organizational
effect of information systems implementation just through the use of information systems applications.
Information systems infusion/diffusion goes beyond use and it is also concerned with IT related
planning, selecting, purchasing and evaluating, which involve the human factor. Such activities, in turn,
affect the routines, the practices, the beliefs and the values related to managing information systems
throughout the organizations. This concept of information systems has not been investigated
(Magalhaes, 2006). Thus, the main objective of this paper is to conceptualize and investigate the
relationship between the organizational context and information systems success in the organization.
Further detailed objectives are explained more as this paper moves on.
2. Information systems success
For academics and practitioners concerned with information systems, one central issue is the study of
development and implementation success. Previous studies suggest that information systems projects
have lower success rates than other technical projects (Barros et al., 2004; Poon and Wagner, 2001).
The number of unsuccessful information systems projects is over the number of successful ones.
However, success is not depending to just one issue. Complex relations of interdependence exist
between information systems and organization. As an example, reducing costs in an organization
cannot be derived solely from information systems implementation. Information systems success is
hard to assess because it represent a vague topic that does not easily lend itself to direct measurement
(DeLone and McLean, 1992).
The contribution of information systems-based assets to organizational performance provides a
benchmark from which the many processes of the information systems function, including business
information system, can be evaluated and refined. Without the benefit of these measures, information
systems assets may be undervalued by users and/or top executives resulting in curtailed budget
allocations and lower managerial profiles for top information systems executives. In other instances,
the absence of reliable performance metrics may cause users and/or top managers to overvalue
information systems assets. Users and strategic planners may therefore be unaware of innovations
adopted by competing organizations that are enhancing and/or changing their patterns of work and
competition. The lack of validated and complete performance criteria in either of the two instances can
result in misguided decisions regarding the acquisition, design, and delivery of information systems.
434
However, two problems occur with the current streams of research. Firstly, the dependent variable in
these studies has been elusive to define. Different researchers have addressed different aspects of
success, making comparisons difficult and the prospect of building a cumulative tradition for
information systems research similarly elusive. In fact, effectiveness of the information systems
function has proven practically impossible to define and measure. One important reason for this is that
the role of the information systems function in organizational performance and effectiveness can be
subtle and difficult to differentiate from other factors. Some organizations define information systems
effectiveness in a way that the true value of it is hidden. Some depend on mostly qualitative rather than
quantitative measures. Within the organizational context, many studies suggest that the efficacy
of information system deployment has a great value to the organization (Magalhaes, 2006). Evidence
also suggests that high information systems effectiveness is associated with high organizational
performance, which leads to a connection between assessment and productivity.
During the last two decades, the research stream of measuring the information system success has been
considered as one of the top ten issues of information systems management (Seddon et al., 1999;
Chang and King, 2005). It is evident that information systems assessment is not well established in the
current literature, and the few recent studies show that more research is needed ( Saunders and
Jones, 1992). Many studies have been conducted attempting to identify factors that contribute to
information systems effectiveness (DeLone and McLean, 1992, 2004). For example, information
systems success has been assessed as a function of cost-benefit (King and Schrems, 1978), information
value (Epstein and King, 1983), or organization performance (Turner, 1982), system acceptance
(Davis, 1989), critical success factor method (Butler and Fitzgerald, 1999)
The second problem with the current streams of research is the mismatch between information systems
and the culture of the organization (Alter and Shaw, 2003). Literatures suggest that the gap between
information systems success assessed from the organizational context still need investigation. The
proposed model tries to take these both problems into consideration.
In an attempt to overcome the confusion in the measurement of information systems success, a deep
investigation on the dependent variable is needed, as an information system has many stakeholders,
each with a different definition of assessing its success. From a developer’s perspective, a successful
information system may be one that is completed on time and under budget, with a complete set of
features that are consistent with specifications and that function correctly. From an innovator’s
perspective, a successful system is one that attracts a large, loyal, and growing community of users.
From a management perspective, a successful system may be one that reduces uncertainty of outcomes
and this lowers risks, and leverage scare resources. From the end user’s perspective, a successful
system may be one that improves the user’s job performance without inflicting undue annoyance
(Briggs et al., 2003). Since the focus here is on the organizations performance as an output of the
merging of information systems within the organization context, it is believed that information quality
is attributes of applications, not of information systems business unit. Therefore, they are not sufficient
to reflect the effectiveness of the entire information systems functions.
To solve this problem, it was found that guidelines from the organizational effectiveness literature was
appropriate for the study of information systems assessment, These guidelines were developed in
response to problems plaguing organizational effectiveness research as described by Steers (1975)
Cameron and Whetton (1983) and Cameron (1986) developed seven basic guidelines that are listed in
the left hand column of Table 1. Many information systems researchers have adopted these guidelines
to clarify conceptual developments in examining information systems functional effectiveness (Myers
et al., 1998; Seddon et al., 1999).
The reason behind following this guideline to develop an information systems assessment is because it
assesses the information systems success from organizational information systems user’ own
experience and perception of the performance for all of the aspects of the information systems function,
which is the main objective of this study. Members of the organization who are using the information
systems services and systems are the primary stakeholder for the information systems function (Segars,
and Hendrickson, 2000).
435
Proceedings of Applied International Business Conference 2008
Table 1: Implementation of Cameron and Whetton's (1983) guidelines
Guideline
Implementation
1. From whose perspective is effectiveness being Organizational users of information systems
services and systems.
assessed?
2. On what domain of activity is the assessment Products and services provided by the information
systems function.
focused?
The information systems function [84].
3. What level of analysis is being used?
Identify strengths and weaknesses; track overall
4. What is the purpose for judging effectiveness?
effectiveness.
5. What time frame is being employed?
6. What type of data is being used for judgments Periodically, ranging from quarterly to annually.
Subjective; perceptual data from individual.
of effectiveness?
7. What is the referent against which effectiveness Past performance measures.
is judged?
Following those guidelines, organization members’ perception of information systems activities derive
from their use of the information systems products and the services provided by the information
systems function. Therefore, following the work of Pitts et al. (1995) and DeLone and McLean (2003),
three constructs of information system success are suggested:
Systems performance
Systems refer to the set that encompass all information systems applications that the user regularly
uses. This construct assesses the quality aspects of systems such as reliability, response time, ease of
use, and the various impacts that systems have on the user' work (Chang and King, 2005).
Information effectiveness
Information is the set generated from any of the systems that the users make use of. This construct
assesses the quality of information in terms of the design, operation, use, and value provided by
information as well as the effects of the information on the user's job (Wang and Wang, 1996).
Service performance
The services provided by the information systems used in the organization include activities ranging
from systems development to help desk to consulting. This construct assesses the user's experience
with services provided by the information systems in terms of quality and flexibility (Fitzgerald et al.,
1993; Chang and King, 2005).
3. Organizational context
It should be noted that information systems success constructs suggested above are measured from the
information systems users’ perspective within the business-unit of the organization, these users’
perception is part of the behavioural context that form the corporate culture of the organization. Bartlett
and Ghoshal state that improved organizational performance depends primarily on the organizational
context, where managers are able to build in and fulfil their managerial roles and processes. They
suggest that an organization can create and embed in its context a work ethic that induce rational and
yet value-oriented actions on the part of its members (Bartlett and Ghoshal, 1994). As the outcome of
research into the practices of successful companies, a number of value-oriented characteristics of
managerial action have been identified. These characteristics are considered as key dimensions for
quality management which induce the creation of a favourable, supportive organizational context for
improved organizational performance, and thus information systems effectiveness. However, The
broad notion of organization context encompasses reflects a combination of the structural context,
culture, and climate of a business unit and it considered an objective, higher-level attribute of the unit
or organization as a whole (Gibson and Birkinshaw, 2004). This view manifests in the definition of
organizational context by Bartlett and Ghoshal (1994), as four behavioural attributes, which are created
and reinforced by a variety of micro- and macro-level actions taken by managers in the organization.
These attributes are: stretch, discipline, trust and support.
Stretch
Stretch is the attribute of an organization’s context that induces its members to voluntarily strive for
more rather than less ambitious objectives. Establishment of a shared ambition, the development of a
collective identity, and the ability to give personal meaning to the way in which individuals contribute
to the overall purpose of an organization contribute to the establishment of stretch.
436
Discipline
Discipline is the attribute of an organization’s context that induces its members to voluntarily strive for
meeting all expectations generated by their explicit and implicit commitments. Establishment of clear
standards of performance and behaviour, a system of open, candid, and rapid feedback, and consistency
in the application of sanctions contribute to the establishment of discipline.
Trust
Trust is the attribute of an organization’s context that induces its members to rely on the commitment
of each other. Fairness and Equity in organization’s decision process, involvement of individuals in
decisions and activities affecting them, and staffing positions with people who possess and are seen to
possess required capabilities contribute to the establishment of trust.
Support
Support is the attribute of an organization’s context that induces its members to lend assistance and
countenance to others. Mechanisms that allow actors to access the resources available to other actors,
freedom of initiative at lower levels, and senior functionaries giving priority to providing guidance and
help rather than to exercising authority contribute to the establishment of stretch.
The conceptualized causal model explains how the interaction of these four key dimensions will result
in an organizational context conducive to initiative and creativity, collaboration and learning, and,
therefore, to an improved organizational information systems performance. Therefore, it is proposed
that:
Proposition 1: The more the organizational context characterized by an interaction of stretch,
discipline, support, and trust is strong, the higher the level of information systems success as captured
by system performance, information effectiveness, and service performance.
4. Ambidexterity and organizational change
The structural context of these four behavioural attributes refers to the establishment of administrative
mechanisms that foster certain behaviours in members of the organization, but its emphasis is on
relatively tangible system and process such as managerial action. The argument is that the four
behaviour- framing attributes of discipline, stretch, support, and trust will create the organization
context in which an ambidexterity emerges.
The concept of ambidexterity in the organizational change and strategic management literature refers
broadly to the organization’s ability to do two different things at the same time - such as manufacturing
efficiency and flexibility (Adler et al., 1999), differentiation and low cost strategic positioning (Porter,
1996), or global integration and national responsiveness (Bartlett and Ghoshal, 1989). A more specific
definition put forward by Tushman and O’Reilly (1996) and Duncan (1976), namely that the
ambidextrous organization is one that achieves alignment in its current operations while also adapting
effectively to changing environmental demands (Tushman and Anderson, 1996). In other words,
ambidexterity refers to the configuration of the entire organization vis- à- vis its task environment, it is
aligned in order to respond efficiently to the demands of its existing customers, but it is adaptive in
order to meet emerging and future demands as they arise or stated slightly differently, ambidexterity is
the capacity that allows the organization to balance the conflicting demands for exploitation and
exploration (March, 1991).
The creation of a supportive organizational context is not about managers mandating specific
behaviours in organization members, but by creating an environment where they themselves take the
initiative to balance the capacities of two aspects of alignment and adaptability behaviours (Gibson and
Birkinshaw, 2004).
O’Reilly and Tushman (2004) identify a decentralized structure, a common culture and vision, and
supportive leaders and flexible managers as the key sources of ambidexterity. Similarly, Bartlett and
Ghoshal (1989) focus on building a shared vision, recruitment and selection, training, and career path
management of executives as ways of stimulating a company to be globally integrated and locally
responsive at the same time. As suggested by the conceptualization of ambidexterity, the two capacities
of both alignment and adaptability are developed through the creation of a particular type of
organization context at the business-unit level. Broadly defined, organization context is the systems,
437
Proceedings of Applied International Business Conference 2008
processes, and beliefs that shape individual-level behaviours in an organization (Ghoshal and Bartlett,
1994).
Ghoshal and Bartlett (1994) conceptualize these four attributes—discipline, stretch, support, and
trust—as interdependent, as an organization needs to foster discipline and stretch to encourage
members of the organization to push for ambitious goals, but it also needs support and trust to ensure
that this happens within a cooperative environment. Therefore, these attributes can be conceptualized
as a balance between a pair of hard elements (discipline and stretch) and a pair of soft elements
(support and trust). This would develop the capacity for ambidexterity, as they attribute towards
engendering individual-level behaviours that result in initiative, cooperation, and learning. However,
organization members take these actions of their own volition because it creates a supportive
environment that inspires an individual to do its best to deliver results. Thus, when a supportive
organization context is created, organization members engage in an ambidexterity represented by both
exploitation-oriented actions toward alignment, and exploration oriented actions toward adaptability,
which results in contextual ambidexterity, and subsequently enhances organizational performance.
Given this conceptualization of ambidexterity, this paper suggests that ambidexterity mediates the
relationship between the four attributes of organization context and subsequent information systems
success used in the organization. That is, the attributes of context influence information systems
success through the development of ambidexterity. When an organization has not developed the
simultaneous capacities for alignment and adaptability, the context characteristics, may or may not
influence information systems success. The reason for hypothesizing a mediating effect is that
ambidexterity is seen as a meta-capability that is developed gradually over time through the interaction
of the various features of an organization context. As both Ghoshal and Bartlett (1994) and Adler etal.
(1999) show, the development of this sort of capability takes many years. Stated slightly differently, it
would be wrong to suggest that an organization could simply institute the four attributes of
organization context and expect them to deliver superior performance. Rather, the four attributes shape
individual and collective behaviours that in turn shape business-unit capacity for ambidexterity, and it
is the ambidexterity that leads to superior performance of the information systems function within the
organization. Therefore, it is proposed that:
Proposition 2: Organizational Ambidexterity mediates the impact of the organization context as
captured by the interaction of stretch, discipline, support, and trust, on the information systems success
as captured by system performance, information effectiveness, and service performance.
5. Conclusion
This paper proposes a new insight of the linkage between the antecedents the information system
success from the context of the organizational structure, in which it affects the performance of the
organization. On reviewing the current literature, two problems arise. The first problem is that the
information systems success measurement has been elusive to define, and different researchers have
addressed different aspects of success assessment. Secondly, there is an observed gap in the literature
showing the strategic linkage relationship between information systems and the culture of the
organization. This paper suggests the concept of organizational context dimensions of discipline,
stretch, trust and support as the shapers of organizational culture, and relates them to the information
systems research, focusing on the information systems success, captured by systems performance,
information effectiveness and service performance. The concept of ambidexterity is also suggested to
mediate the relationship between organizational context and information systems context, occurring
when the organization achieves alignment in its current operations while also adapting effectively to
changing environmental demands, which results the organization to be ambidextrous, and subsequently
enhances organizational performance. Although this paper proposes an initial attempt to investigate the
conceptualised model, further studies should follow up to robust the theoretical model and enhance its
ground by empirical studies, in order to enhance its implications that would benefits the academics and
practitioners. It is believed that this study would contribute in providing a reliable performance metrics
to be used by information systems executives and top managers to evaluate the information systems
assets. The empirical validation of this model will also be beneficial for strategic planners and strategic
information executives to enhance and change their patterns of work and competition. The validation
would also provide complete performance criteria to guide decisions regarding the acquisition, design,
and delivery of information systems across the organization.
438
References
Adler, P., Goldoftas, B., and Levine, D. (1999) Flexibility versus efficiency? A case study of model
changeovers in the Toyota production system. Organization Science, 10, 43–68.
Alter, S., Shaw, N.C. (2003) Systems and Culture: connecting the dots. Working paper.
University
of
San
Francisco.
[Online,
Available
at
http://members.authorsguild.net/stevenalter/files/Systems_and_cul.pdf
accessed
on
12/01/2008]
Barros, M.O., Werner C.M.L. and Travassos, G.H. (2004) Supporting risks in software project
management, Journal of Systems and Software, 70, 1-2, 21-35.
Bartlett, C.A. and Ghoshal, S. (1989) Managing across borders: The transnational solution. Boston:
Harvard Business School Press.
Bartlett, C.A. and Ghoshal, S. (1994) Changing the role of top management: beyond strategy to
purpose. Harvard Business Review,(72, 6, 79-88.
Briggs, R.O., De Vreede, G.J., Nunamaker, J.F., and Sprague, R.H. (2003) Special issue: Information
systems success. Journal of Management Information Systems, 19, 4, 5-8.
Butler, T. and Fitzgerald, B. (1999) Unpacking the systems development process: An empirical
application of the CSF concept in a research context. The Journal of Strategic Information
Systems, 8,4, 351-371.
Cameron, K.S. (1986) A study of organizational effectiveness and its predictors. Management Science,
32,1, 87-112.
Cameron, K.S. and Whetton, D.A. (1983) Some conclusions about organizational effectiveness. In K,S,
Cameron and D,A, Whetton (eds,). Organizational Effectiveness: A Comparison of Multiple
Models. New York: Academic Press, 261-277.
Chang, J.C., King, W.R. (2005) Measuring the performance of information systems: A functional
scorecard. Journal of Management Information Systems,(22,1, 85-115.
Checkland, P. and Holwell, S. (1998) Information, system and information systems: Making sense of
the field. Chichester, UK, J, Willey.
Davis, F.D. (1989) Perceived usefulness, perceived ease of use, and user acceptance of information
technology. MIS Quarterly, 13, 3, 319-340.
DeLone, W.H., and McLean, E.R. (1992) Information systems success: The quest for the dependent
variable. Information Systems Research, 3,1, 60-95.
DeLone, W.H. and McLean. E.R. (2003) The DeLone and McLean model of information systems
success. A ten-year update. Journal of Management Information Systems, 19, 9-30.
DeSantics, G. and Poole, M.S. (1994). Capturing the complexity in advanced technology use: Adaptive
structuration theory. Organization Science, 4,4, 1-36.
Duncan, R.B. (1976) The ambidextrous organization: designing dual structures for innovation. In
Kilmann, R.H, Pondy, L.R, and Slevin, D. (eds), The Management of organization 1, 167188. New York: North-Holland.
Epstein, B.J. and King W.R. (1983) An experimental study of the value of information, Omega, 10,3,
249-258.
Fitzgerald, L., Johnson, R., Brignall, S., Silvestro, R. and Voss, C. (1993) Performance measurement in
service business. London: Chartered Institute of Management Accountants.
Ghoshal, S. and Bartlett, C.A. (1994) Linking organizational context and managerial action: The
dimensions of quality management. Strategic Management Journal, 15, 91-112.
Gibson, C.B. and Biriskinshaw, J. (2004) The antecedents, consequences, and mediating role of
organizational ambidexterity. Academy of Management Journal, 47,2, 209-226.
King, J.L. and Schrems, E.L. (1978) Cost-benefits analysis in IS development and operation.
Computing Surveys 10, 19-34.
Magalhaes, R. (2006) A context-based dynamic capability perspective of IS/IT organisational fit.
International Journal of Information systems and Change Management, 1,4, 396-420.
March, J.G. (1991) Exploration and exploitation in organizational learning. Organization Science, 2,
71–86.
Myers, B.L., Kappelman, L.A. and Prybutok, V.R. (1998) A comprehensive model for assessing the
quality and productivity of the information systems function: Toward a theory for information
systems assessment. In E.J. Garrity and G.L. Sanders (eds.). Information Systems Success
Measurement. Hershey, PA: Idea Group, 94-121.
O’Reilly, C.A. and Tushman, M.L. (2004) The ambidextrous organization. Harvard Business Review,
74-83.
Orlikowski, W.J. (1992) The duality of technology: rethinking the concept of technology in
organizations. Organization Science, 3,3, 389-427.
439
Proceedings of Applied International Business Conference 2008
Pitt, L.F., Watson, R.T. and Kavan, C.B. (1995) Service quality: A measure of information systems
effectiveness. MIS Quarterly, 19, 2, 173-185.
Poon, P. and Wagner, C. (2001) Critical success factors revisited: Success and failure cases of
information systems for senior executives. Decision Support Systems, 30,4, 393–418.
Porter, M. E. (1996) What is strategy? Harvard Business Review, 61–81.
Saunders, C.S. And Jones, J.W. (1992) Measuring the performance of the information systems
function. Journal of Management Information Systems, 8,4, 63-82.
Seddon, P.B., Staples, S., Patnayauni, R. and Bowtell, M. (1999) Dimensions of information systems
success. Communications of the AIS, 2, 2-39.
Segars, A.H. and Hendrickson, A.R. (2000) Value, knowledge, and the human equation: Evolution of
the information technology functions in modem organizations. Journal of Labor Research,
21,3, 431-445.
Steers, R.M. (1975) Problems in the measurement of organizational effectiveness. Administrative
Science Quarterly, 20,4, 546-558.
Turner, J.C. (1982) Towards a cognitive redefinition of the social group' In Tajfel, H. (eds.), Social
Identity and Intergroup Relations. Cambridge: Cambridge University Press, 15-40.
Tushman, M.L. and Anderson, P. (1997) Managing strategic innovation and change, Oxford
University Press, New York, NY.
Tushman, M. L. and O’Reilly, C.A. (1996) Ambidextrous organizations: managing evolutionary and
revolutionary change. California Management Review, 38,4, 8-30.
Wang,Y. and Wang, R.Y. (1996) Anchoring data quality dimensions in ontological foundations.
Communications of the ACM. 39, 11, 86-95.
440