Tech., Inst., Cognition and Learning, Vol. 6, pp. 177–192
Reprints available directly from the publisher
Photocopying permitted by license only
© 2009 Old City Publishing, Inc.
Published by license under the OCP Science imprint,
a member of the Old City Publishing Group
Using Adaptive Simulations to Develop
Cognitive Situational Models of
Human Decision-Making
Matt Watkins1 and Amlan Mukherjee2,*
Department of Computer Science, Michigan Technological University,
1400 Townsend Dr., Houghton, MI 49931. Email: [email protected]
2
Department of Civil and Environmental Engineering, Michigan Technological University,
1400 Townsend Dr., Houghton, MI 49931.
1
Situational models allow decision-makers to organize situational information
using domain specific knowledge, thus helping them prioritize trade-offs that
lead to effective decisions. This is specifically relevant to domains, where
there can be more than one “correct” decision and the effectiveness of decisions are not immediately obvious. Construction project management is such
a domain. We study situational models of construction managers using a situational simulation of the construction domain to create scenarios that require
managers to make trade-offs and see the impact of their decisions unfold in
time. Previous work has shown that expert performance is a necessary, but not
a sufficient condition for situational awareness. The relationship between situational awareness, decision-making and expertise in complex dynamic environments, has not been formally constructed and measured in a way that
allows researchers to identify knowledge organization patterns of expertise
The notion that effective situational models lead to effective decisions, and
that experts are effective decision-makers is critical to this paper. We use this
notion and the construct of situational models to specifically investigate the
relationship between situational awareness and expert performance. The broad
goal of this research is to improve construction management education by
furthering our understanding of differences in expert and novice cognition. In
this paper we formally define situational models and propose a method to
quantify and measure them.
Keywords: Mental models, decision-making, situational simulations, construction management, cognition and learning
*Corresponding author: [email protected]
177
178
Watkins and Mukherjee
1 Introduction
The construction management domain can be studied as a complex system,
which has multiple interacting components (schedule, cost, resource distribution
and availability, etc.) with multiple feedback loops. Sterman [1992] asserts that
attributes of construction projects are complex, consisting of multiple inter-dependent parts, involving multiple feedback processes and non-linear relationships
involving schedules, costs and productivity of resources like labor and equipment. In addition, construction projects are often impacted by external setbacks
such as bad weather, labor strikes, and delayed material deliveries. Construction
managers aim to complete projects on time and within budget, while holding in
tension all the different requirements and constraints that are specific to the project as well as constraints such as safety that generally apply to the domain. This
paper is part of an on-going investigation into the nature of expertise in construction managers.
Experience is a critical component of expert decision-making among construction managers. Time allows them to inductively construct and organize
knowledge about the domain that often cannot be easily formalized. Given the
nature of the domain, where there are few “correct” decisions, and many competing “effective” decisions, expert knowledge is difficult to formalize. Analysis, of
the impacts of decisions taken in critical scenarios, often has to wait till the project is completed. Such post-performance analysis helps managers to constantly
update their knowledge of the domain and sometimes re-organize their “mental
models.” Over time and with experience managers develop “mental models” that
support expert decisions.
Previous research investigated expertise by exploring mental models
[Mukherjee et al. 2005] of construction managers. It has shown that there exists
a high correlation between levels of structuredness of thought and knowledge
organization between expert and novice construction managers. This indicates
that over periods of time, experience fine-tunes the mental models of construction managers. A qualitative analysis shows that experts tend to apprehend the
future impacts of their plans while deciding, as compared to novices.
This research is based on the notion that an expert decision is informed by
situational detail and domain knowledge that is organized by mental models of
expertise. It is difficult to formally capture a knowledge organization schema
such as a mental model hence, we use situation specific knowledge organization
models that are more feasible to analyze and capture. We call these models situational models. Aggregating situational models over multiple subjects in similar
circumstances is a first step towards developing general mental models of exper-
Using Adaptive Simulations to Develop Cognitive
179
tise. The next section defines and discusses situational models and their relationship to situational awareness in greater detail.
2 Situational Models
We define situational models as transient internal organizations of information
that decision-makers in dynamic complex environments use to comprehend a
scenario and formulate effective decisions. A situational model is based on situational awareness, and mental models of the knowledge domain. Situational
awareness provides decision-makers with scenario specific information that is
necessary for effective decision-making. Mental models of the domain provide
them with a schema to organize the situational information and create a situational model. The close relationship of situational models to situational awareness merits a discussion of situational awareness.
Uhlarik [2002] surveys the relevant work in situational awareness, (henceforth
referred to as SA) and classifies the different approaches based on how SA is
characterized: as a product or a process, and the method used in doing so. Based
on their survey we briefly discuss SA and justify the premise for defining situational models (henceforth referred to as SM).
Endsley [1995] defines SA as the perception of elements in the environment
within a volume of time and space, the comprehension of their meaning, and the
projection of their status in the near future. Her approach is based on the information processing paradigm and uses constructs of short-term memory and attention. SA is defined as a “state of knowledge” that consists of information
perception, comprehension and the ability to project future status of the environment. Endsley asserts that SA is a final product and is different from the process
that is used to achieve it. In addition, SA captures only those aspects of the environment that are dynamic and constantly changing. For example, in defining SA
for air traffic controllers, the knowledge of current aircraft positions is considered
but the number of airports in the local area is not included. Endsley’s approach
[Uhlarik and Comerford 2002] has been criticized because it strongly asserts that
SA is a product that can be achieved through a finite number of steps including
only dynamic information.
Adams et al. [1995] conceptualizes SA both as a product and a process. Their
description of SA consists of three elements: information about the environment,
the schema or internal knowledge organization of the decision-maker and exploration of the environment by him/her in order to update their knowledge of the
environment. This action/perception cycle provides a description of SA, as a
180
Watkins and Mukherjee
product that is dynamically being updated by a feedback loop. It also emphasizes
the cognitive inter-dependence between memory, perception and action. The criticism of this approach is that it does not provide a way to measure SA. In addition, it is based on psychological constructs such as memory and perception that
are not well understood [Uhlarik and Comerford 2002].
Some researchers at the interface of decision-making and SA have asserted
that SA is equivalent to expertise [Crane 1992]. Other researchers, as reviewed in
[Uhlarik and Comerford 2002], have likened situation assessment to SA. Situation assessment is considered to be performed based on clusters of knowledge
that allow subjects to categorize events. However, there are opposing opinions
among researchers [Adams et al. 1995] who assert that it is quite possible to have
good SA and not be an expert decision-maker. This observation is very pertinent
to our study. Previous research shows that when construction managers are provided the same information, i.e., the cognitive aspects of SA are removed and SA
is provided as a product, experts and novice construction managers differ in their
responses. Experts tend to consider future impacts of their decisions and focus
more on identifying general constraint violations in the domain that could create
problems, while novices tend to focus on particulars of the problems at hand. This
is in line with the differences in expert/novice cognition identified by Bransford
et al. [1999].
2.1 Measuring SA
Various research efforts have been conducted to measure SA by quantifying
observed variables. SAGAT [Endsley 2000] is probably the most popular system
for measuring situation awareness. It uses a set of questions, embedded within
interactive experimental and simulated scenarios, to query human subjects about
their perception of the state of the environment. The situations though dynamic in
nature, are frozen while the subject is queried. The queries are made at all three
levels of SA identified by Endsley’s [1995] model. A subject’s SA is measured by
the percentage of correctness of their responses.
Kirlik and Strauss [2006a], build on Endsley’s method of measuring SA by
quantifying correspondence between a human subject’s perception of an environment and its true state. The significant difference between previous efforts at
measuring SA and their approach is that it is conceptually founded in theories of
judgment under uncertainty, and free of psychological constructs such as attention, memory and schemata. They define SA as the degree of correspondence
between a set of judgments made by a human subject and the statistical distribution of real variables in the environment. Data is gathered by questioning the
subject in order to determine the perceived state of the environment. The degree
Using Adaptive Simulations to Develop Cognitive
181
of correspondence of the human subject’s responses is conceptually decomposed
into seven components, each of which can be measured objectively using correlation statistics. The components are organized into an equation that provides a
measure of the subject’s “skill score”, taking into account uncertainty in the task
environment, regression biases, and the human subject’s knowledge and consistency in information acquisition and processing. This statistically rigorous technique provides the extra dimension of quantifying numerically and defining the
ranges of the situation and judgment, thus providing a rich understanding of
SA – as could not be provided by previous work.
Strauss and Kirlik [2006b] tested their proposed approach to measuring SA
using an interactive simulation of submarine stealth missions. Human subjects
were asked to make judgments regarding the state of enemy submarines in the
simulation at several points within the trial. This experiment demonstrated that
the decomposition of SA into the seven proposed components revealed information about the design of the display in the simulation, which traditional methods
of measuring SA would have missed.
Strauss and Kirlik’s research captures the most significant and recent advances
in understanding and measuring SA. Our approach is similar to theirs because we
use a situational simulation intervention called ICDMA (discussed in a following
section) to measure our construct of SM. We also agree with the importance of
separating the SM from psychological constructs of memory, attention and schemata because they are difficult to measure. However, there are significant diff­
erences between our motivation and goals – thus necessitating the definition of
the SM construct. Their goal is to understand interface mediated SA – with the
focus being primarily on measuring perception at the human-operator interface.
Abstractly speaking, they try to “separate the signal and noise in the performance
of uncertain tasks” – specifically for human interactions across well-defined
operational interfaces. Knowledge for them is defined as “correlation between the
model of the situation and model of the operator”. They make no claims to measuring the cognitive processes and knowledge organization that contribute to the
development of SA. Our research focuses on measuring the differences in cognitive knowledge organization between experts and novices and we measure the
construct of SM to statistically infer them. Our research domain is in construction
management, which requires decision-making in a complex dynamic domain
under uncertainty. Decisions involve trade-offs and experts often differ in their
decisions. Our quest is to measure a human-subject’s (both expert and novice)
situational awareness and document decisions in such situations so that we can
statistically infer mental models of expertise – thus helping us to better understand the relationship between decision-making and expertise. Given the lack of
182
Watkins and Mukherjee
an “operational interface” and the uniqueness of situations we need to develop the
construct of SM, that is based on SA, to achieve our goals. In short, we are interested in the nature of the signal and by measuring different characteristics of a
signal we intend to infer the cognitive organization that produces the signal. In
the following paragraphs we describe the conceptual underpinnings of the SM
framework, the ICDMA situational simulation and specifically our approach to
formally measuring SM.
3 The SM Framework
We use the existing body of research as a foundation for the developing the
theoretical framework of SM. The underpinnings lie in the following:
••
••
••
Endsley’s [1995] formulation of SA
The dynamic feedback cycle proposed by Adams et al. [1995]
Domain knowledge organization patterns or mental models of decision making
We connect these theories and develop the SM construct, which can be formalized, measured and assessed. This is significant given that it will allow us to
investigate the relationship between decision-making and expertise. It will also
allow us to develop general measures for the effectiveness of decisions that are
made under similar circumstances.
Endsley and Adams et al.’s research on SA has been criticized because both
the descriptions of SA are based on psychological constructs of schemata and
exploration that are not very well understood and are difficult to measure. In the
proposed framework we have constructed the definition of SM using a combination of Endsley’s “state of knowledge” (product) definition and Adams et al.’s
“action perception cycle” (process) definition. We try to avoid this criticism by
defining SM as a transient state of knowledge of a situation, rather than a cognitive state of a decision-maker. Decisions reflect changes made to this transient
state of knowledge. As described later in this paper, this allows us to formally
measure SM without losing the strength of the psychological constructs that previous research has provided.
Figure 1, represents the proposed framework. It consists of two primary cyc­
les: the action/perception cycle [Adams et al. 1995] and the experience cycle. The
action/perception cycle is responsible for feedbacks that are immediate to
the situation. It accounts for the dynamic nature of decision-making and fulfills
Using Adaptive Simulations to Develop Cognitive
183
two functions: 1) it allows decision-makers to continuously explore and update
their SA as the situation unfolds in time and 2) it allows decision-makers consider
the immediate and long term impacts of their decisions within the context of the
project. The experience cycle allows decision-makers to learn from their experiences across different project management experiences. Experience of the success or failure of decisions in diverse projects allows them to modify their mental
models of the domain. Their mental model of the domain is a map or schema of
variations and patterns in relationships between different project variables. This
feedback in the long term allows decision-makers to transition from naivety to
expertise.
Figure 1
The Situational Model (SM) framework.
The SM is a transient construct that is informed by SA and domain mental
models, which are shaped and formed by the action/perception and experience
cycles, respectively. Mental models and SA are both difficult to quantify and can
be measured using external indicators. Quantifying and documenting the SM
construct helps in instantiating a human subject’s domain knowledge organization stored in the mental model, by unifying it with the specifics of a situation.
184
Watkins and Mukherjee
Situations can be formalized through indicators and quantitative descriptors. A
decision which is driven by such an unification of general domain knowledge
with specific situation particulars also reflects the organization of knowledge in
the mental model. Hence, the construct of the SM will be relatively easy to quantify and measure. SM data collected from multiple human subjects will be investigated using data-mining methods to develop models to infer cognitive knowledge
organization and information processing that motivates decision-making. Indirectly, the proposed method will also measure and classify patterns of knowledge
organization among construction managers. A description of the proposed representation and data mining method is discussed in detail in the next sections.
4 Situational Simulations: ICDMA
In order to study SM, a simulation called ICDMA (Interactive Construction
Decision Making Aid) has been developed. The ICDMA is an advanced version
of the Virtual Coach, which was developed at the University of Washington by
one of the co-authors. The Virtual Coach continues to be under development. It
has been used to educate construction managers. Currently ICDMA provides the
simulation back end to the Virtual Coach. A detailed explanation of the Virtual
Coach and how the ICDMA functions can be found in previously completed
research [Rojas and Mukherjee 2006; Rojas and Mukherjee 2005b; Rojas and
Mukherjee 2003]. We provide a brief description in this section.
The ICDMA simulates construction management projects and the humansubjects in it are construction managers. Construction managers are decisionmakers, whose primary goal is to complete a construction project on time and
under budget. This is often a significant challenge. The simulation presents a
construction manager with a situation, and allows the manager to respond. It
proceeds according to the decisions, which are made by the human subject. Consequences from the decisions result in new scenarios that require the subjects to
respond. This process continues to completion of the simulated construction project. The goal of the manager/human subject is to complete the project on time and
under budget. Figure 2, illustrates the interface of ICDMA, providing the subject
with information on planned and actual performance with respect to budget and
schedule (Gantt Chart) and future predictions with respect to project completion
time and final cost.
Throughout the simulation, human subjects are presented with events that
force the simulated project to deviate from its original plan. These events are
often external in nature and often construction managers have no control over
Using Adaptive Simulations to Develop Cognitive
(a)
185
(b)
Figure 2
The ICDMA interface: (a) Schedule and Commodity curves (b) Activity specific information.
them. Examples are: a failed material delivery and bad weather. They cause delays
in planned construction activities, which may in turn have consequences cascading through out the project, as a delay in one activity may delay other activities
that are related to it by time and resource constraints. The manager’s reaction to
these crisis scenarios, and the specifics of the scenarios help in formally capturing
the SM of the manager. Data regarding the decisions made and the consequences
of each decision is collected through the course of the simulation. Analysis methods have been described in the next section.
4.1 Framework and Analysis of SM in Situational Simulations
In this section we describe a framework and that is used to describe the data
capture from ICDMA and a method that is used to analyze the data and develop
SM. The ICDMA is based on a representation of construction management
domain [Rojas and Mukherjee 2005b; Rojas and Mukherjee 2006] using interval
temporal logic within a framework of software agents that can reason about the
domain. Hence, situations as well as the resource, and temporal constraints and
relationships between activities and events in the simulation can be asserted using
formal logic statements. The assertions are made using multi-state variables that
reflect the state of the simulated system.
For example, consider the narrative: In the event of a labor strike that lasts for
the time interval t, productivity for all activities is reduced to 0 due to a 0% avail-
186
Watkins and Mukherjee
ability of labor. Formally, we represent productivity by the variable prod and
labor by the variable labor. In the event of a labor strike the variables are set to
the null state for the interval of the strike – t. This is represented across all activity
contexts as: {labor(null,t), prod(null,t)}. Assuming there was 100% productivity
for a time interval of t’ before the strike, the situation before the strike started is
represented by: {labor(100%,t’), prod(100%,t’)} where the predicate Meets(t’,t)
asserts that the time interval is t’ immediately before t. This event represents a
violation in a resource constraint i.e. a necessary resource, labor, is missing. This
can be logically asserted by the following:
∀t · Act _ Labor _ Strike(t.start ) → ∃ t’· labor (100%, t’) ∧ prod (100%, t’)
∧ Event ( Labor _ Strike, t ) ∧ ImmBefore(t’, t ) (1)
A detailed description of the representation and reasoning can be found in
[Rojas and Mukherjee 2005b].
The data that will be analyzed using the proposed framework is collected from
construction managers whose level of experience ranges from novice to expert.
The novices are students in civil engineering programs, learning to become practicing construction managers. Understanding expert SM will help educate novice
construction managers better.
Each subject manages a construction project using ICDMA. At each state of
the simulation, a situation vector is captured. A situation vector at any time t
consists of the state of all variables in the simulation, and is expressed as:
SVt = {v1 (t ), v2 (t ), v3 (t ), …, vn (t ) }
(2)
where n is the number of variables defined in the simulation of the construction project.
The subject has control over a subset of the variables within the simulation.
Examples of these variables include the amount of each material type purchased
and the amount of each type of labor hired. The set of variables over which the
subject has control can then be defined as:
CV = {cv1, cv2 , cv3 , …, cvk }
(3)
Where cvi is a member of the set of all variables in the simulation, and k is the
number of variables over which the subject has control. The subject controls these
variables through input fields in the simulation, and inputs a new set of values at
Using Adaptive Simulations to Develop Cognitive
187
each step. A decision, is defined as the change in variables over which the subject
has control, and the decision vector at time t is defined as:
Dt = {cv1 (t ) − cv1 (t −1), cv2 (t ) − cv2 (t −1), cv3 (t )
− cv3 (t −1), …, cvk (t ) − cvk (t −1) }
(4)
Each decision of the subject is captured using this decision vector.
Within the simulation, there are two types of events: aleatory and epistemic.
Aleatory events are the set of events over which the subject has no control, such
as the weather. Epistemic events are events which are under the control of the
subject, such as violations of constraints between activities. The set of epistemic
events E is denoted by:
E = {e1, e2 , e3 , …, em }
(5)
The occurrence of both aleatory events and epistemic events within each simulation is also captured.
Because the purpose of the simulation is to minimize the cost of the project,
the subject’s decision will reflect a change in state, which minimizes the perceived risk. The subject’s perceived risk is computed as a product of the perceived
probability of a situation and its perceived impact. SA provides the user with
both: perceived probability and perceived impact. SM is represented by the organization patterns present in this data. Let the perceived probability of each
epistemic event at each time point t be denoted as a function of the simulation
variables as:
P(ei , t ) = f (v1 (t ), v2 (t ), v3 (t ), …, vn (t ))
(6)
The structure of the function f dictates the method of analysis of the collected
data. For example, if P(ei, t) is computed by taking a linear combination of the
variables, then, the function f becomes:
P(ei , t ) = Ai,1 ∗ v1 (t ) + Ai,2 ∗ v2 (t )
+ Ai,3 ∗ v3 (t ) … + Ai, n ∗ vn (t ) + Ai,( n+1)
(7)
A matrix M of the coefficients are used to characterize the functional form of
f and reflecting the specific analysis method use. The matrix M can be written
as:
188
Watkins and Mukherjee
A1,1
A2,1
M = A3,1
…
Am,1
A1,2
A2,2
A3,2
A1,3
A2,3
A3,3
… A1, n
… A2, n
… A3, n .
Am,2
Am,3 … Am, n
A different M will be generated for each subject studied. Based on the data
collected, M will be updated such that it reflects the perceived probability of each
event for each subject.
It is the purpose of this study to search for statistical correlations between the
M generated by a subject and the number of years of experience the subject has
in construction management. There are different mathematical techniques available for this task, including graphical analysis. A graph G can be derived from the
data to establish the statistical correlation, dependence and conditional independence that characterize the decisions that drive the simulation. Each node in the
graph represents a variable in the simulation. The graph connects the variables vj
within the context of the events ei with a link weighted by Ai,j, if Ai,j exceeds some
threshold of significance. This graph can then be analyzed using to reveal conditional dependence of one variable on another. Patterns of conditional dependence
are analyzed in order to determine if a pattern exists which correlates to the level
of expertise of the subject.
In a preliminary experiment data was collected from five simulation runs of
ICDMA. The purpose of this experiment was to illustrate how data can be captured and modeled using the proposed approach, and also to provide a concrete
picture of a formally quantified SM. In order to control the context of a decision,
a single context was chosen from which data was collected. Within the context
the human participant (a student) monitored a crew working on a welding activity
that is part of constructing a steel framed office building. The details of the simulated construction project can be found in [Anderson et al. 2007]. The data collected in this situation tracked how often the subject assigned more than planned
labor (by increasing the crew size), more than planned hours (making labor work
overtime) and the amount of material to be installed for the particular crew. Specifically, the variables monitored were the Labor, Hours and Material quantities.
The data showed that there were 89 decision vectors (across all the five simulation runs) where the crews worked as planned and no change was registered in
any of the variables. There were 8 decision vectors when material and hours were
Using Adaptive Simulations to Develop Cognitive
189
increased only, and there were 10 decision vectors where all three variables were
increased.
Based on this data, the SM in Figure 3 was generated. The MIM software
package [Edwards 2000] was used to analyze the data using discrete graphical
modeling techniques. It shows that hours worked is conditionally independent
from crew size given materials installed. The model shows that an increase in
materials installed is accompanied by increases in crew size and hours worked.
This relationship is reasonable and reflects the efforts of the simulation participants to improve performance by installing materials faster with a larger crew
size. There is no significant relationship in the model between crew size and
hours, as an increase in crew size does not always correspond to an increase in
overtime.
Figure 3
The best fit graphical model for the data collected, based on the MIM software package.
This preliminary results from this experiment shows that the proposed model
can be used to represent and analyze SM. We collected preliminary data and
showed a simple analysis, using discrete graphical modeling methods that can be
used to develop SM using ICDMA. There are limitations to this analysis method.
The temporal dynamics of the decisions made have not been considered. In addi-
190
Watkins and Mukherjee
tion, the graphical modeling method used assumes that the relationships between
each of the variables are hierarchical. That means, in a three variable model, the
lack of an individual relationship between any two variables excludes the possibility of any higher order relationships involving those two variables. This may
not be true for complex domains. In a non-hierarchical domain the representation
scheme will still be the same, though an alternative analysis method will have to
be chosen. In addition, a rigorous human subject data set was not used and the
users were not classified by expertise. The set of users were all students. These
issues will be considered in future research.
5 Instructional Context
The interactivity of situational simulations allows them to be platforms that
can be used to promote learning based on contextually rich information. They can
be classified as learning environments that are based on theories in situated cognition [Winn 2002]. Such environments expose learners to clinical exercises that
help them explore future consequences of present decisions and the sensitivity of
their contexts to such decisions, and over time develop better decision-making
skills. The Virtual Puget Sound [Windschitl and Winn 2000] and the Surgical
Simulator [Oppenheimer and Weghorst 1999] efforts are comparable learning
environments.
The instructional context of situational simulations such as the ICDMA is
explained in detail in [Rojas and Mukherjee 2005a]. The Virtual Coach simulation (predecessor of the ICDMA) has been used on senior level construction
management students. The goal of the simulation was to help students identify
the inter-relationships between construction costs and constraints driving activity schedules, and recognize the impacts of unexpected external events on construction planning and decision-making. In order to test the efficacy of the
simulation as a learning environment the students were given pre-tests and posttests. Results illustrated that the students showed a statistically significant
improvement in their post-test performance as compared to their pre-test performance, after the simulation was used as an educational intervention. The details
of this simulation and the relevant testing can be found in [Rojas and Mukherjee
2006].
Systems like the Virtual Coach helped in improving construction management education by providing students with a simulation environment within
which they can explore different decision scenarios that are difficult to observe
in real time. The ICDMA is an advanced version of the Virtual Coach and in
Using Adaptive Simulations to Develop Cognitive
191
addition to providing the same educational and instructional advantages as the
Virtual Coach, the ICDMA can measure organization of knowledge among the
subjects participating in the simulation through the development of SM. Understanding expert SM will help educators identify aspects of decision making that
need to be emphasized in order to improve construction management edu­
cation.
In this paper, we have introduced a framework and illustrated a simple application by which the situational awareness of the students can be captured while they
are participating in the simulation and simple association models can be captured
from their responses to the simulation model. The association models are used to
capture the situation model of the participant. Future applications of this model
can be used to capture SM of students at different times in their learning cycle, to
establish the dynamics of change in SM. Future research will investigate if the
shift in the SM of novices can indeed be considered to be learning, and a step
towards expertise.
6 Conclusion
In this paper we proposed a framework that can be used to measure situational
models of cognition using situational simulations such as the ICDMA. We further
presented an initial analysis of data collected using the ICDMA situational simulation, and analyzed using graphical modeling methods to establish simple SM.
Future work will address data mining methods that can be used to analyze SM
collected from multiple human subjects (both expert and novice) to identify
knowledge organization patterns among construction managers. The eventual
goal is to understand how changes in SM over time reflect changes in knowledge
organization and the eventual shift from naïveté to expertise. This paper is a first
step in that direction.
Acknowledgments
This work was supported by the NSF grant SES 0624118 to Amlan Mukherjee. However, these contents do not necessarily represent the policy of the NSF
and there should be no assumption of endorsement by the Federal Government.
We would also like to thank Dr. Nilufer Onder, Associate Professor in Computer
Science at Michigan Tech. for her reviews and thoughtful comments on the
paper.
192
Watkins and Mukherjee
ReferenceS
Adams, M. J., Tenney, Y. J., and Pew, R. W. (1995). Situation awareness and cognitive management
of complex systems. Human Factors, 37(1), 85–104.
Anderson, G. R., Onder, N., and Mukherjee, A. (2007). Expecting the unexpected: representing, reasoning about, and assessing construction project contingencies. In the Proceedings of the Winter
Simulation Conference, S. G. Henderson, B. Biller, M.-H. Hsieh, J. Shortle, J. D. Tew, and R. R.
Barton, eds. WSC, 2041–2050.
Bransford, J. D., Brown, A. L., and Cocking, R. R. (1999). How People Learn. National Academy
Press, Washington D.C.
Crane, P. M. (1992). Theories of expertise as models for understanding situation awareness. 13th.
Annual Symposium on Psychology in the Department of Defense. 148–152.
Edwards, D. (2000). Introduction to Graphical Modelling. Springer-Verlag, 2nd edition.
Endsley, M. R. (1995). Toward a theory of situation awareness in dynamic systems. Journal of Human
Factors and Ergonomics, Vol. 37(1), 32–64.
Endsley, M. R. (2000). Direct measurement of situation awareness: Validity and use of SAGAT. Situation Awareness Analysis and Measurement, M. R. Endsley and D. J. Garland, eds., Lawrence
Earlbaum Associates, Mahwah, NJ.
Kirlik, A. and Strauss, R. (2006a). Situation awareness as judgment I: Statistical modeling and quantitative measurement. International Journal of Industrial Ergonomics, 36, 463–474.
Kirlik, A. and Strauss, R. (2006b). Situation awareness as judgment II: Experimental demonstration.
International Journal of Industrial Ergonomics, 36, 475–484.
Mukherjee, A., Rojas, E., and Winn, W. D. (2005). Exploring mental models of construction managers. Construction Research Congress, April 5–7, San Diego, CA, I. D. Tommelein, ed. ASCE.
Oppenheimer, P. and Weghorst, S. (1999). Immersive surgical robotic interfaces. In the Proceedings
of Medicine Meets Virtual Reality. 242–248.
Rojas, E. and Mukherjee, A. (2003). Modeling the construction management process to support situational simulations. Journal of Computing in Civil Engineering, ASCE, 17(4), 273–280.
Rojas, E. and Mukherjee, A. (2005a). A general purpose situational simulation environment for construction education. Journal of Construction Engineering and Management, ASCE, 131(3),
319–329.
Rojas, E. and Mukherjee, A. (2005b). Interval temporal logic in general purpose situational simulations. Journal of Computing in Civil Engineering, ASCE, 19(1), 83–93.
Rojas, E. and Mukherjee, A. (2006). A multi-agent framework for general-purpose situational simulations in the construction management domain. Journal of Computing in Civil Engineering,
ASCE, 20(6), 1–12.
Sterman, J. D. (1992). Systems dynamics modeling for project management. Systems Dynamics
Group. Sloan Business School, MIT.
Uhlarik, J. and Comerford, D. A. (2002). A review of situational awareness literature relevant to pilot
surveillance functions. Report to the Office of Aerospace Medicine, Federal Aviation Administration, US Dept. of Transportation, DOT/FAA/AM(02/3), 1–25.
Windschitl, M. and Winn, W. D. (2000). A virtual environment designed to help students understand
science. International Conference of the Learning Sciences, B. Fishman and S. O’Connor-Divelbiss, eds. Mahwah, NJ: Lawrence Erlbaum Associates, Inc., 290–296.
Winn, W. D. (2002). Learning in artificial environments: Embodiment, embeddedness and dynamic
adaptation. Tech., Inst., Cognition and Learning, 1.