Department of Management Science
and Information Systems
Semester 2 2002
INFOSYS 751
RESEARCH METHODS II
INSTRUCTOR: DR JIM SHEFFIELD
COURSE DESCRIPTION
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
Department of Management Science and Information Systems
INFOSYS 751 SC 2001 RESEARCH METHODS II
COURSE INFORMATION
1.
CONTACT INFORMATION
Instructor
Time
Place
2.
Dr Jim Sheffield, Room 1.32 Old Choral Hall, Phone x.7157
Email: [email protected]
Office Hours: Thursdays and Fridays 12:00 to 1:00pm and by
appointment or ‘drop by’.
Thursdays 10-1 A-L, Fridays 9-12 M-Z.
Students will form teams of size 6. These teams may cross streams
providing teams attend class together and the streams are of equal size.
Room 336 Executive Programmes Building 810 Short St
DESCRIPTION
It is the mark of an instructed mind to rest satisfied with the degree of precision which
the nature of the subject admits, and not to seek exactness when only an
approximation of truth is possible.. Aristotle.
There’s no point in being exact about something if you don’t even know what you are
talking about. John von Neumann.
Things should be made as simple as possible, but no simpler. Einstein.
This paper is the second of a two-part sequence on research methodology, and is designed to
be complementary to Research Methods I. Both papers prepare students to pursue research
and development in information systems and/or management science and operations
management. Both papers are compulsory for all PhD, Masters and Honours students in the
Department of MSIS.
Whereas Infosys 750 Research Methods 1 focused on the application of quantitative
techniques for data analysis, Infosys 751 Research Methods 2 focuses on the qualitative
methods that support learning and measure the quality, value and coherence of the outcome.
Research Methods 2 introduces qualitative methods for the exploration, description and
evaluation of concepts, information and knowledge. The classroom learning process involves
2
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
participation in a social world, observation of a material world, and experience of a personal
world – for all of which the exactness afforded by numbers is illusory.
3
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
3.
AIM
This course is designed to assist MCom students to complete their theses and BCom
(Honours) students to enrich their dissertations. Theory guidelines are kept relatively
simple and parsimonious so that time is available during the course for practice. Students
apply their ‘book learning’ by participating in collaborative action research. Students are
expected to be willing and able to enunciate what it feels like to ‘eat their own cooking.’
Students may also experience, and may need to resolve, some minor issues relating to
individual motivation and teamwork.
4.
LEARNING OBJECTIVES
1. THE IDEA
Why will pursuing
this idea add value?
THE MAXIMUM OF 15 POINTS
IS ALLOCATED AS FOLLOWS:
1
6
To reduce
equivocality the..
ONE POINT FOR EACH BOX
AND DIRECTED LINE
2. THE OBJECTIVE
What is our (my? your?)
objective?
2
To reduce
uncertainty the..
5
PLUS AN EXTRA
POINT EACH
FOR
BOXES 4 AND 5
3
3. THE ACTION PLAN
How will we meet each
sub-objective?
Start. Planning.
6. THE PAYOFF
Why did pursuing
this idea add
value?
..key reference
is Habermas
5. THE RESULTS
What is the evidence
that we have met our
(my? your?) objective?
..key reference
is McGrath
4
4. THE PLAN IN ACTION
How strong is the evidence that we
have met each sub-objective?
Links between Intention & Outcome.
Implementation. Finish.
THE BASIC 15 POINT METRIC FOR THESIS QUALITY
Upon completing this course students, individually and as part of a group, will be able:
1. To employ a consistent framework (viz, the Systems V model) to analyze the chain of
evidence in any completed research in the Faculty of Business and Economics.
2. To identify the strategic choices that reduce equivocality about three worlds (viz, Social,
Material and Personal – see Habermas 1984; 1987) an/or world views (viz Interpretivist,
Positivist and Critical - see Cavana, Delahaye and Sekaran 2001).
3. To identify the strategic choices that reduce uncertainty about research methods (McGrath
1982), especially methods such as field studies, proof of concept, and action research that
are commonly used in the Department of MSIS.
4. To apply the Systems V model to evaluate the quality and coherence of business research
to meet one of three objectives (exploration, description or evaluation) in each of three
4
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
broad application areas of research in the Department of MSIS (concept engineering,
information engineering and knowledge engineering).
5. To undertake Interpretive Research in Concept Engineering. The nature of the
engagement necessary to understand why human subjects in a particular context may
choose to use (or not use) artifacts or tools such as those provided by Information
Technology. The self-organizing, and emergent behavior by which we humans enact a
social consensus on ‘soft’ systems. The complex web of language, meaning, social
practices, rules and resources that link beliefs and actions, intentions and outcomes in a
particular context. The image of the window frame.
6. To undertake Positivist Research in Information Engineering. The process engineering or
software engineering research techniques and technologies that deliver proof of concept
or proof by existence. The key roles of (formal) language, specification and (assumed)
social context in the iterative development and demonstration of how an emergent
software object or art/i/f/act behaves. The image of the mirror.
7. To undertake Critical Research in Knowledge Engineering. The action research processes
to build personal commitment to a social consensus for informed, emancipatory change.
Inquiry into Why, What and How individuals and groups seek empowerment, and relief
from systems crises, via Information Technology. Does technology resolve or create
systems crises? Are there winners and losers? Who decides? Who cares? How do we tell?
Why do we care? The image of the lantern.
8. To develop integrated sets of quality indicators for research in the Department of MSIS.
Teams will make presentations to the class of the results of benchmarking (i.e., comparing
and contrasting) multiple, conceptually related theses against the quality metric they have
developed. Informed class-wide debate on ‘best research practices’ is expected from all
students. This dialogue should lead to an understanding of why some theses are of high
quality and should be used as a reference model for other researchers.
5.
ASSESSMENT
In-Course Assessment
50%
Weeks 1-4: Interpretivist Research: Exploring Concept Engineering
15%
Most but not all of the evaluation is based on the oral presentation and report in week 4
Weeks 5-8: Positivist Research: Describing Information Engineering
15%
Most but not all of the evaluation is based on the oral presentation and report in week 8
Weeks 9-12: Critical Research: Evaluating Knowledge Engineering
15%
Most but not all of the evaluation is based on the oral presentation and report in week 12
Occasional quizzes, exercises and/or team mini-tutorials (5@1%)
5%
Weeks for exercises and/or team mini-tutorials only to be announced in advance
The grade points associated with the in-course assessments will reward active exploration of
the opportunities presented and full participation in class meetings. The in-course activities
target the same skills as those assessed in the Final Exam. The in-course assessments are
5
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
expected to be highly correlated with, although higher than, the mark earned in the final
exam.
Final Exam
50%
A copy of a Final Exam for a recent year is included at the end of this Course Description.
6.
LEARNING AND TEACHING STRATEGIES
Compared with the quantitative methods covered in Research Methods1, the qualitative
methods studied in Research Methods 2 are easy to state in procedural terms but deceptively
difficult to understand and to perform in practice. To the uninitiated, the concepts appear both
vague and overly abstract. It appears difficult to make them concrete, to operationalize and
internalize them. A balance is sought between the theory of qualitative research methods and
actual hands-on practice in simple yet relevant research activities.
Because qualitative methods require personal experience to develop ‘know how’, Research
Methods 2 employs an experiential and skill-building approach. Students work, individually
and in teams, on a variety of exercises. These ‘hands-on’ exercises provide opportunities for
participation, personal experience, surprise, dialogue, observation and self-knowledge. A key
element is the personal ‘frame-breaking’ experience that exposes hidden beliefs and
structured contradictions.
7.
LEARNING RESOURCES
Required Texts
1. Cavana, Delahaye and Sekaran. Applied Business Research: Qualitative and Quantitative
Methods. Third Edition. Wiley. 2001. This text should be brought to every class,
including the first, as well as the Final Exam.
2. Yin, Robert K. Case Study Research: Design and methods. Second Edition. Sage. 1994.
(Expensive! Look for second hand copies.)
Theses
Copy 2 of Management Science and Information Systems Department theses are to be
checked out of the library and brought to class on a regular basis. See the list of theses at the
end of this document.
Readings on Research Methodology (i.e., conceptual frameworks for discussing,
denoting and expressing research participation, observations and experiences):
Required and/or Highly Recommended Readings (Extracts are supplied in class)
6
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
1. Checkland, Peter and Sue Holwell. Information, Systems and Information Systems.
Wiley. 1998.
2. Cohen, Louis and Lawrence Manion. Research Methods in Education. Fourth Edition.
Routledge. 1994.
3. Collins, Jeff and Howard Selina. Introducing Heidegger. Icon Books Ltd. 1999.
4. Crotty, Michael. The Foundations of Social Research: Meaning and Perspective in the
Research Process. Allen & Unwin. 1998.
5. Garvin, David A. Learning in Action. A guide to putting the learning organization to
work. Harvard Business School Press. 2000.
6. Gellatly, Angus and Oscar Zarate. Introducing Mind & Brain. Icon Books Ltd. 1999.
7. McGrath, Joseph E. Martin, Joanne. Kulka, Richard A. Judgement Calls in Research.
Sage Publications. 1982. This is the second book in a series entitled Studying
Organizations: Innovations in Methodology.
8. McLachlan, Ron. Management initiatives and just-in-time manufacturing. Journal of
Operations management, Vol. 15, pages 271-292. 1997.
9. Nunamaker Jr, Jay F., Minder Chen, Titus D.M. Purdin. System Development in
Information Systems Research. Journal of Management Information Systems, Winter
1990-91, Vol. 7, No. 3, pp. 89-106.
10. Pinsonneault, Alain and Kraemer, Kenneth L. “Survey Research Methodology in
Management Information Systems: An Assessment.” Journal of Management
Information Systems, Vol. 10, No. 2, pp. 75-105. Fall 1993.
11. Rosenhead, Johnathan and John Mingers, Editors. Rational Analysis for a
Problematic World Revisited: Problem Structuring Methods for Complexity,
Uncertainty and Conflict. Second Edition. Wiley 2001.
12. Shank, Gary D. Qualitative Research: A Personal Skills Approach. Merrill Prentice
Hall. 2002.
13. Taylor, James R. and Elizabeth J. Van Every. The Emergent Organization:
Communication as Its Site and Surface. Lawrence Erlbaum Associates. 2000.
14. Trask, R.L. and Bill Mayblin. Introducing Linguistics. Icon Books Ltd. 2000.
15. Winograd, Terry and Fernando Flores. Understanding Computers and Cognition. A
new foundation for design. Addison-Wesley. 1987.
16. Weick, Karl E. The Social Psychology of Organizing. Second Edition. Random
House. 1979.
8.
CONTENT OUTLINE AND SCHEDULE
The course is divided into three complementary sections:
Weeks 1-4: Interpretive Research: Exploring Concept Engineering
Weeks 5-8: Positivist Research: Describing Information Engineering
Weeks 9-12: Critical Research: Evaluating Knowledge Engineering
See the initial pages of each section of the course book for assessment and schedule details.
7
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
AN EXAMPLE OF A RECENT FINAL
EXAMINATION
Note That The Course Has Changed By 15%-25% In The Interim.
There Is Now Somewhat More Emphasis On Qualitative Research
(The first module is now entitled Interpretive Research: Exploring Concept Engineering)
About The Same Emphasis On Software Engineering
(The second module is now entitled Positivist Research: Describing Info. Engineering)
And Somewhat Less Emphasis On The Cavana, Delahaye and Sekaran Text
(The third module is now entitled Critical Research: Evaluating Knowledge Engineering as it
focuses on Conflict, Values and the Imperative for Change)
___________________________________________________________________________
THE UNIVERSITY OF AUCKLAND
EXAMINATION FOR MCom BCom(Hons) ETC 2000
MANAGEMENT SCIENCE AND INFORMATION SYSTEMS
Research Methods II
(Time allowed: THREE hours)
NOTE:
1. This examination counts for 50% of your final assessment and is marked out of 100.
2. You must answer ALL QUESTIONS. The points associated with each question
range from 6% to 30%.
3. All materials supplied in the course plus all materials that you prepared during the
course may be consulted.
8
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
-2-
1.
Case Study: Problems at Roadway Hospitality
John Campbell, the CEO of Roadway Hospitality was wondering how to
differentiate among the three different types of facilities offered under the Roadway
Hospitality flagship so as to attract the right type of clients to each of the facilities.
The Roadway Deluxe was meant for business travelers, the Roadway Express was
meant for those looking for the least expensive accommodation, and the Roadway
Royal was meant to provide high quality services for big spenders. Campbell felt
that revenues could be quadrupled if only clients understood the distinction among
the three types of facilities offered.
Keen on developing a viable strategy to eliminate the brand confusion and make
clear the distinction, John Campbell conducted a customer survey of those who had
used each type of facility and found the following. The customers were blissfully
unaware of the differences among the three types of facilities. Many complained
about how old the buildings were and how poorly the facilities were maintained.
The quality of services was also rated as poor. Furthermore, when rumour seems to
have spread that one of the ideas of Campbell was a name change to differentiate
the facilities, franchise owners became angry and the mixed messages they gave to
the customers had not helped clients to understand the differences.
Campbell thought that he first needed to understand how the different
classifications would be important to the several classes of clients, and then he
could develop a marketing strategy that would enhance revenues. Simultaneously,
he recognized that unless the franchise owners fully cooperated with him in all his
plans, mere face lifting and improvement of customer service would not bring in
the added revenues he hoped for.
(a)
Identify the main problem
(4 points)
(b)
Develop a theoretical framework
(4 points)
(c)
Develop at least four hypotheses
(4 points)
9
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
-3-
2.
Why is the unit of analysis an integral part of the research design?
3.
(a)
4.
(6 points)
What three conditions should be met to establish that variable X
causes variable Y?
(5 points)
(b)
Define external validity and internal validity
(3 points)
(c)
What research method or methods would you choose:
(a)
(i) To achieve high external validity?
(2 points)
(ii) To achieve high internal validity?
(2 points)
(iii) To achieve high external validity and high internal validity?
(2 points)
The following variables were observed at the Ellerslie Race Course.
Identify the type of scale that can be used to measure each of these variables:
(i) The number on the back of each jockey
(1.5 points)
(ii) The placing (e.g., first, second, third, etc) of each horse in the race
(1.5 points)
(iii) How far each horse was ‘off the pace’ (i.e., how many minutes and
seconds it finished behind the winner)
(1.5 points)
(iv) How long each horse took to complete the race
(b)
5.
‘28 C is twice as hot as 14 C.’ Discuss.
(1.5 points)
(4 points)
“A valid instrument is always reliable, but a reliable instrument may not always be
valid.” Discuss.
(8 points)
10
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
-4-
6.
This question is based on Mclachlin, Ron. Management Initiatives And Just-In-Time
Manufacturing. Journal Of Operations Management, Vol. 15, Pages 271-292. 1997.
The McLachlin paper describes Yin’s ‘pattern matching’ technique and uses it to
logically test (rather than statistically test) twenty propositions.
Briefly outline the ‘logic’ that lies at the heart of this test with the aid of one or
more worked examples.
(10 points)
7.
Discuss the behaviour that accompanied various experiential exercises (such as
Global Warming, String, Bookworm, Dictionary, and Population) in the light of the
guidelines and explanations in Winograd and Flores.
(10 points)
8.
Provide a detailed and critical assessment of the research reported in papers
(a) and (b) below. For the purpose of this assessment these two papers should
be treated as a single piece of research. The assessment should be based on the
principles enunciated in paper (c).
a) Taurima, Wayne, and Michael Cash. Researching by Networking: Developing
a Draft Protocol on Researching Maori Business Enterprises. Proceedings of
the NZSMS Educators Conference, pages 111-118. 1998.
b) Taurima, Wayne, and Michael Cash. “Aroha as Strategy”?: Suggestions
towards a post-initial theory of Maori business development. PECNZSM, Vol
1, Number 1, 167-173. 1999.
c) Klein, Heinz K., and Michael D Myers. A Set of Principles for Conducting
and Evaluating Interpretive Field Studies in Information Systems. MIS
Quarterly, Vol. 23, No 1, pages 67-94. March 1999.
(30 points)
______________________________________________________________
11
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
THE UNIVERSITY OF AUCKLAND
___________________________________
SECOND SEMESTER, 2001
Campus: City
___________________________________
INFORMATION SYSTEMS
Research Methods II
(Time allowed: THREE hours)
NOTE:
1. This examination will be evaluated in accordance with the Guidelines for
Evaluating all Assignments (see Course book 1, Item #9)
2. All materials supplied in the course plus all materials that you prepared during the
course may be consulted.
3. This examination counts for 50% of your final assessment and is marked out of
100 points.
4. Each question is worth 25 points.
5. YOU MUST ANSWER ANY FOUR OF THE SIX QUESTIONS.
12
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
2.
1.
This question requires you to critique certain aspects of Theoretical Bases for
Action Research through the theoretical lens provided by Cohen and Manion;
Cavana, Delahaye and Sekaran pages 8-11; Winograd and Flores.
In the Introduction, and in the answer to Research Question 1, Theoretical Bases
for Action Research suggests that the objectives of a workshop process for the
strategic evaluation of a comprehensive urban plan should match the Habermas
criteria for Rational Communication.
2.
(a) Explain why you agree or disagree with this suggestion.
(12.5 points)
(b) Explain why this is an important issue.
(12.5 points)
This question requires you to critique certain aspects of Theoretical Bases for
Action Research through the theoretical lens provided by Cohen and Manion;
Cavana, Delahaye and Sekaran pages 8-11; Winograd and Flores.
In the answer to Research Question 2, Theoretical Bases for Action Research
suggests that the critical, interpretive and positivist modes of enquiry are required
to evaluate a comprehensive urban plan.
(a) What is the value of the insights provided by each mode?
(12.5 points)
(b) What is the validity of the evaluation outcome produced by each mode of
enquiry?
(12.5 points)
3.
This question requires you to critique certain aspects of Theoretical Bases for
Action Research through the theoretical lens provided by Checkland and Holwell,
and the Pop exercise.
In answering Research Question 3, Theoretical Bases for Action Research
suggests that a workshop process for the strategic evaluation of a comprehensive
urban plan may be supported by facilitated, electronically-supported dialogue.
(a) Explain how facilitated, electronically-supported dialogue impacted workshop
processes.
(12.5 points)
(b) Explain how facilitated, electronically-supported dialogue impacted workshop
outcomes.
(12.5 points)
13
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
3.
4.
5.
Examine the behaviour associated with the experiential exercises Global
Warming, Pop, String and Bookworm through the theoretical lens provided by:
(a) An Introduction to Cognitive Engineering
(12.5 points)
(b) Dawson
(12.5 points)
This question is composed of three parts each of which contains sub-parts. Your
answers may be very brief. No explanation is required.
(a)
Name the research strategy or strategies McGrath associates with the
following.
(i) A study in a setting that is existentially real for the participants
(1 point)
(ii) A study emphasizing precision of measurement of behaviour in a generic
setting
(1 point)
(iii) A study where universal human behaviour systems are studied in an
unobtrusive manner
(1 point)
(iv) Studies that do not involve human behaviour
(2 points)
(b) Case Study 1: The Packing Department
A visitor to a factory observes that the workers in the packing department
have to interact with one another to get their jobs done. The more they
interact, the more they seem to tend to stay after hours and go to the local pub
together for a drink. However, the women packers, even though they interact
with the others as much as the men, do not stay late and visit the pub after
work hours.
Produce a diagram that:
(i) Names the variables
(3 points)
(ii) Identifies the type of each variable
(3 points)
(iii) Illustrates the relationships among the variables
(1 point)
(c) Case Study 2: The Machining Department
A foreman believes that the low efficiency of the machine tool operators is
directly linked to the high level of fumes emitted in the workshop. He would
14
INFOSYS 751 SC 2002
RESEARCH METHODS II
COURSE DESCRIPTION
___________________________________________________________________________
4.
like to demonstrate this to his supervisor through a research study so that
action may be taken to fix the problem.
(i) Would this be a causal or a correlational study?
(ii)
Name the variables in the study.
(1 point)
(2 points)
(iii) There are three flows of causality that are consistent with the above
statement of the foreman’s belief. Draw a simple diagram to illustrate
each.
(6 points)
(iv) Is this an exploratory, descriptive or evaluative (explanatory or
hypothesis-testing) study?
(1 point)
(v)
Is this a field study, lab experiment or field experiment?
(vi) What is the unit of analysis?
6.
(1 point)
(2 points)
This question is composed of three parts some of which contains sub-parts. For all
but the first part and sub-part, brief explanations are required.
(a) Operational definitions
(i) What is the ‘C’, ‘D’ and ‘E’ of an operational definition?
(ii) What is the purpose of an operational definition?
(iii) What should not be included in an operational definition?
(b)
(1.5 points)
(2 points)
(1.5 points)
Scales
(i) ‘28 C is twice as hot as 14 C.’ Discuss.
(3 points)
(ii) On page 16 of Theoretical Bases for Action Research the performance of
two options against five criteria is measured on five-point scales. Options
and criteria are complex qualitative concepts that lack well-defined
dimensions. The scale markers are ++, +, 0 (i.e., zero), -, and --. Does the
presence of the zero mean that this is a ratio scale? If not, what kind of
scale should it be? Why?
(9 points)
(c) Goodness of measures. “A valid instrument is always reliable, but a reliable
instrument may not always be valid.” Discuss.
(8 points)
_______________________________
15