Running head: DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
The Differential Effects of Cognitive Load on Time and Accuracy of Response
Ari Klevecz, Daniel Slyngstad & Shayla Velthuis
Claremont Graduate University
1
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
2
The Differential Effects of Cognitive Load on Time and Accuracy of Response
Data presentation has become an increasingly popular subject due to the demand that new
technology has placed on data visualization, ease of presentation, and graphical images. Not only
do people enjoy seeing data visually, but there is now a dramatically increased capability for
creating visual data representations and sending them via email to increase speed and depth of
understanding of large amounts of data for increasingly large audiences. However, knowledge of
contingencies in data presentation, these new ways of graphically representing data can not only
be taxing in terms of the time it takes to understand the information presented, but also how
accurately people interpret the information, and how they are affectively influenced by the
information presented in a visual format. Thus the questions remain: What does the “ideal” graph
look like, and is it consistent with principles articulated by data presentation scholars (Wainer;
1984, Tufte, 2001)? How much information is too little/too much?
Previous work done on this topic of graphical representation focuses on how not to
present data (Wainer, 1984; Wainer, 1997). In the data presentation rules by Wainer (1997), he
says that bad graphs minimize the data density, and thus have a very low data to ink ratio. In this
case, the ideal graph would have a high data to ink ratio and provide as much information
possible with the least amount of ink. Many people have done work on the understanding of
graphical information from the “too much” information or high ink-to-data ratio stand point
(Wainer, 1997; Zawitz, 2000). Research in cognitive psychology supports the idea that too much
information or ink presented might increase a person’s (extraneous) cognitive load (Sweller et
al., 1998), which is a measure of strain on a person’s working memory; applied to the current
context, it would mean inability to interpret information correctly if too many items that require
mental attention are presented at one time (Kalyuga, 2011). However, it should be noted that too
little ink on the graph might make the graph difficult to interpret as well and that in a way, too
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
3
little information creates high (intrinsic) cognitive load as well (Sweller et al., 1998), in that
people assessing visual representations of data would need to exert more executive control over
working memory in order to decipher information being presented in the graph and how pieces
of information fit together. It is this relationship that has yet to be thoroughly explained in the
graphical data research.
Additionally, the current endeavor is consistent with research in multimedia presentations
that seeks to account for inherent limitations of human cognition, including the dual-channel
assumption (words and pictures are processed differently, and thus both must be considered
when making a multimedia presentation) and the limited capacity assumption (Mayer & Moreno,
2003). Further, Mayer and Moreno’s (2003) description of Type 2 cognitive load, or
overloading processing demands in multiple processing channels closely matches the current
study’s attempt to induce high cognitive load. Therefore, graphs (depicting both visual and
verbal information) with both inadequate and overabundant amounts of information are likely to
produce high cognitive load, meaning that there would be a curvilinear relationship between the
amount of cognitive load and time, accuracy, and affect, in the sense that too little information is
as detrimental to a person’s understanding of information as too much information. In this
research we aim to being to explore the effect of components of a graph on (a) time spent on the
attempt to understand the graph, (b) actual correct responses to the relevant information in the
graph, and (c) a measure of the respondents affect, with respect to the amount of relevant
information that is provided in each condition.
There are several applicable principles of “good data presentation” that the researchers
used to create an “ideal” graph. Tufte (1997) recommends that a graph should serve a single
purpose which should be one of the following: data description, tabulation, exploration, or
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
4
decoration. That is, a graph’s purpose and information should be clear, and the viewer should
understand relatively quickly what is being displayed by the graph. This can be done by focusing
on the substance of the graph, rather than focusing on extra designs, methodology, or technology
(Tufte, 1983; Tufte, 1997). By incorporating extra information or making the graph more
technologically interesting (such as visual cues or motion) the actual meaning of the graph could
be misinterpreted. It is this research that guided the way in which we attempted to create our
“ideal” graph. The researchers attempted to include relevant information that would be helpful
for the understanding and quickness of response when answering questions, in addition to either
too little or too much relevant information, specific to the graphs with varying cognitive load
(from either too little or too much information).
Klass (2002) explains that the relationship between how quickly and effectively the
information is digested by the reader and the graphical content is understood, is increased when
the data in a table or graph is formulated well. An efficient graphical display will allow a reader
to quickly discern the purpose and importance of the data and to draw a variety of interesting
conclusions from a large amount of information. How quickly a reader can digest the
information presented, discern the critical relationships among the data, and draw meaningful
conclusions depends on how well the chart is formatted. We further hypothesize that the most
“ideal” graph will have (1) the lowest response time to the related questions for the graph, (2) the
most correct responses to the questions about the graph, and (3) will have the most positive
affect and least negative affect response from the participants in their attempt to interpret the
graph.
Our first hypothesis (1) is that an ideal graph will have the lowest response time to a set
of survey questions related to the graph. The hypothesis can be elaborated by describing that the
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
5
researchers expect to find a curvilinear relationship between cognitive load, regardless of
whether it is due to too little relevant information (intrinsic cognitive load) or too much relevant
information (high extraneous cognitive load). Stated in other words, graphical displays with
extremely high data-to-ink ratios will create a longer response time, and the graphs with very low
data-to-ink ratios will create the similarly lengthened response time to a set of questions. The
“ideal” graph will have the lowest response time, due to an optimal data-to-ink ratio that
facilitates rapid understanding of the most important and relevant aspects of the presentation.
Our second hypothesis (2) that an ideal graph will yield the most correct responses to the
relevant information presented in the graph might be better understood within the context of the
first hypothesis. The ideal graph should have both the fastest response time for the most correct
amount of answers for the relevant information presented in the graph. The reason hypothesis 2
is grounded in hypothesis 1 is that the point of an ideal graph would be to interpret all the
relevant information quickly, thus we are not interested in 100% accuracy of interpretation if the
participant takes an extensive amount of time to answer the questions as this is not the point of a
graphical presentation of data.
Our third hypothesis (3) is that the most ideal graph will lead to less negative affect, and
more positive affect for the participant. That is, that an easily interpreted graph will be more
enjoyable to interpret and will lead to higher perceived engagement and challenge, while the
graphs that produce higher cognitive load will be (in this case) more frustrating, confusing, or
boring for the participants to interpret and answer questions about. This hypothesis is less
grounded in research on data presentation and represents an attempt at exploratory inquiry by the
researchers. It is plausible that those who present data are interested in how their viewers react
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
6
to a presentation, such that it would be beneficial to find an optimal balance between the
capability to understand a graphical presentation and the affect that it invokes.
Method
Participants
Participants were recruited using Amazon’s Mechanical Turk (mTurk) and asked to
complete a survey on Qualtrics survey design software. In total, 245 respondents completed the
survey. Due to difficulties with data collection speed, compensation was increased twice.
Therefore, 29 participants were compensated $0.25, 53 participants were compensated $0.40,
and 150 participants were compensated $0.60 upon completion of the survey.
Design and Measures
The researchers implemented a between subjects design in which participants were
assigned to one of four distinct conditions (Appendix A). Each conditions included a graph with
a set of ten questions (Appendix B), and each condition (from 1-4) included more relevant
information provided on the graph to answer the questions presented. Conditions 1-4 can be
summarized as “high load, low information”, “medium load, medium information”, “low load,
optimal information”, and “high load, high information”, respectively. Condition 3 represents
the optimal condition, constructed according to principles of good data presentation (Tufte, 1983;
Wainer, 1984; Jacoby, 1997; Klass, 2002). The conditions contained an identical set of
questions, and questions were randomized within each condition to account for ordering effects
(Sudman, Bradburn & Schwartz, 2010). Each graph was also standardized to the extent that each
condition that included more relevant information only added information to the existing graph
(e.g. the formatting in Condition 4 was consistent with Condition 3, except for the information
that was added to Condition 3 to produce the graph presented in Condition 4). Questions
referring to the graph all contained 6 response options to minimize the likelihood that
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
7
participants could get a correct answer by chance, and response options were constrained (as
opposed to open ended) to avoid the possibility of respondents providing impossible or irrelevant
responses, a tradeoff that is commonly described in survey methods literature (Dillman, 2007).
Additionally, respondents were timed according to how long it took them to complete the entire
condition. A question designed to examine whether those taking the survey were paying
attention was embedded within each condition, instructing participants to select a particular
answer.
Consistent with research supporting the idea that surveys follow norms of conversation
(Dillman, 2007; Sudman, Bradburn & Schwartz, 2010), participants were prompted with a
fictional scenario involving a malfunctioning time machine that presented data from the future
(in this case, the Olympics) in the form of graphs. Participants were then asked to help interpret
the information. This technique (“tickling”) was meant to increase participant engagement in the
task. Data used to create the graphs for each condition was gathered from publicly available
information about the race for gold and total cumulative medals in the 2012 London Olympic
Games. Participants were told that the data was being gathered from the 2048 Olympics, and the
names of the countries were kept anonymous to avoid the potentially biasing effects of public
knowledge on survey results (Crano & Brewer, 2002).
After the respondents were finished answering the 10 questions about the graph of
cumulative gold versus total medals, they were then directed to answer questions about affective
outcomes, including frustration, confusion, engagement, boredom, and challenge. The questions
were presented as sliding scales on Qualtrics, ranging from 0-100 for each measure (Appendix
C). Like earlier questions, they were presented in random order. Given that the research is
primarily exploratory in nature, validated scales for each of these items were not used. Since the
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
8
survey was hosted on mTurk twice before, a question was added at the end of the survey (so as
not to jeopardize comparability between data “batches”) inquiring whether participants had ever
taken the survey before.
Results
Descriptive Analyses
Two cases were removed before analyses began, one for failure to provide informed
consent, and another for failing the attention check question embedded within their condition.
Three additional respondents were removed due to failure to complete the survey. Descriptive
analyses were conducted for the participant’s country of origin, amount of time it took
participants to complete their assigned condition, the total number of correct answers given (out
of 10 possible), and for the assigned condition (to assess cell sample size), in order to assess
violations of statistical assumptions. Of the remaining cases, 79.7 percent were from the United
States (n=192), 18.1 percent were from India (N=43), and 2.0 percent (0.4 percent each) were
from Canada (n=1), Egypt (n=1), Jamaica (n=1), Qatar (n=1), and Sweden (n=1). To be
consistent with previous research on data presentation, and with the reading norms that were
consulted during the construction of each condition, only those participants from the United
States were included in univariate analyses (Dillman, 2007).
Results of descriptive analyses for number of correct answers reveals no major departures
from normality (skewness = -0.40, kurtosis = -0.72). No univariate outliers were detected for
this variable. For the time to complete condition variable, however, evidence for departure from
normality (skewness = 1.60, kurtosis = 3.85) prompted the removal of three univariate outliers
and a logarithmic transformation. The transformed variable more closely approximated
normality (skewness = -0.25, kurtosis = 0.83). After removal of outliers, a listwise deletion
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
9
procedure was applied, such that data 179 participants (N=179) were included in analyses to test
research hypotheses. Thirty-nine (n=39) respondents were given Condition 1, forty-seven
(n=47) respondents were assigned to Condition 2, forty-seven (n=47) participants were assigned
to Condition 3, and forty-six (n=46) respondents were assigned to Condition 4.
Test of Hypothesis 1: Time
A univariate ANOVA was conducted using condition as a fixed factor with the log
transformed time variable as the dependent variable, testing the hypothesis that there is a
quadratic component in the relationship between time and condition, such that as load increases
(either with too little or too much relevant information presented), the time it takes to complete
the questions also increases. The descriptive statistics used for the analysis of the time variable
are represented in Table 1. Overall, the researchers found a significant main effect of condition,
F(3, 176)=2.85, p<.05. Additionally, a significant quadratic component was discovered,
suggesting a curvilinear relationship between assigned condition and time to completion, F(1,
176)=4.98, p<.05.
Pairwise comparisons were conducted to assess simple effects between each condition.
Because this research is mainly an informed exploratory endeavor, Fisher’s Least Significant
Difference (LSD) was used to assess statistical significance of mean differences between the
average time taken to complete the task for each condition. A significant difference was
observed between Condition 1 (M=2.50, SD=0.24) and Condition 3 (M=2.40, SD=0.20), p<.05,
and between Condition 3 and Condition 4 (M=2.53, SD=0.24), p<.01, suggesting that conditions
characterized by higher cognitive load take longer time to complete, regardless of whether the
condition has high or low levels of relevant information. Figure 1 depicts the curvilinear
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
10
relationship between condition and time, showing that those in Condition 3 took the least time.
Overall the the results provide strong support for hypothesis 1.
Figure 1. Plot of means for completion time for each condition. Significant differences were found between the
“High Load, Low Info” and “Low Load, Optimal Info” conditions, and between the “Low Load, Optimal Info” and
“High Load, High Info” conditions. Note that the y-axis begins at “2.40” and not “0”.
Test of Hypothesis 2: Total Correct Answers
A second univariate ANOVA was conducted using condition as a fixed factor with the
number of correct answers as the dependent variable, testing the hypothesis that there is a
quadratic component in the relationship between condition and number of correct answers given
by respondents, such that the number of correct answers given would be lower, on average, for
conditions with higher cognitive load, regardless of whether increased load came from the lack
of excess of relevant information given. The descriptive results for this hypothesis are depicted
in Table 2. The researchers found a significant main effect of condition, F(3, 176)=20.01,
p<.001, however no statistically significant quadratic component was detected for this
relationship.
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
11
Pairwise comparisons were also conducted to test simple effects for this hypothesis.
Since no significant quadratic component was discovered, the researchers were less interested in
collecting nuanced exploratory data for comparisons between each condition, and thus Tukey’s
Honestly Significant Difference (HSD) was chosen as the method of comparison to adjust for
alpha inflation. Significant differences were found between the means of Condition 1 (M=4.85,
SD=2.07) and Condition 3 (M=6.83, SD=2.57), p<.001, and between Condition 1 and Condition
4 (M=8.00, SD=1.80), p<.001, as well as between Condition 2 (M=5.34, SD=1.96) and
Condition 3, p=.001, and between Condition 2 and Condition 4, p<.001. Conditions 3 and 4
were also found to be significantly different from each other, p<.01. The results generally
suggest that as the amount of relevant information increases, the number of correct answers
increase regardless of the amount of cognitive load (from low or high amounts of relative
information). Overall, the researchers found partial support for hypothesis 2. Figure 2 depicts
the relationship between condition and number of correct answers. The “ideal” graph presented
Figure 2. Plot depicting the means of total correct answers for each condition. The graph suggests a relatively linear
relationship between the two variables, such that number of correct answers increases with the amount of
information presented. Significant differences were found between the “High Load, Low Info” and “Low Load,
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
12
Optimal Info” conditions, and between the “Low Load, Optimal Info” and “High Load, High Info” conditions. Note
that the y-axis begins at “4” and not “0”.
in Condition 3 did not show the highest average amount of correct answers, but it was not
statistically significantly different from Condition 4 in this regard.
Test of Hypothesis 3: Affective Outcomes
A multivariate analysis of variance (MANOVA) was conducted using condition as the
fixed factor and the affective outcomes of frustration, confusion, engagement, boredom, and
challenge as dependent variables. The analysis revealed a significant effect of condition, Pillai’s
Trace=0.23, F(15, 526)=2.98, p<.001. Five univariate ANOVAs (with trend analyses) were also
conducted to more accurately describe the effect of condition, and only frustration and confusion
showed a statistically significant main effect of condition, F(3, 185)=3.24, p<.05, and F(3,
185)=10.55, p<.001. Trend analyses revealed no evidence for a curvilinear relationship was
discovered for any of the five measures. Refer to Tables 3 and 4 for descriptive statistics for the
frustration and confusion outcomes in each condition, respectively.
Pairwise comparisons were conducted to assess simple effects for the frustration and
confusion outcomes. Since the omnibus F again did not detect significant quadratic components,
the researchers were not as concerned with detecting nuances in the data in an exploratory
fashion, therefore the more conservative Tukey’s HSD was used as the method of pairwise
comparison to correct for potential alpha inflation. For frustration (Figure 3), significant
differences were discovered between Condition 2 (M=50.22, SD=33.62) and Condition 3
(M=32.39, SD=29.57), implying that participants were significantly more frustrated when asked
about graphs that lack an optimal amount of information to answer questions. For confusion
(Figure 4), significant differences were found between Condition 1(M=57.09, SD=32.01) and
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
13
Conditions 3 (M=26.67, SD=27.74) and 4 (M=35.73, SD=30.40), and between Condition 2
(M=54.18, SD=35.21) and Conditions 3 and 4. Results suggest that participants are more likely
to be both confused and frustrated in conditions in which a less than optimal level of information
was presented to them, and provide partial support for hypothesis 3.
Figure 3. Mean frustration scores for participants in each condition. The figure suggests that participants are
significantly more frustrated when less that optimal information is presented to them.
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Figure 4. Mean confusion scores for participants in each condition. The figure suggests that participants were
significantly more confused when a less than optimal amount of relevant information was presented to them.
14
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
15
Discussion
Results from the present study provide strong evidence for the curvilinear relationship of
our first hypothesis concerning the average time it took to complete each condition, and mixed
evidence for our second and third hypothesis concerning accuracy of response and affect
measures. Each hypothesis sought to demonstrate that cognitive load has a curvilinear
relationship with performance, where high cognitive load can be induced by either too little
information or too much information, and that the “optimal” condition constructed according to
data presentation principles will produce the least cognitive load, the shortest time of completion
and the best performance. Out of the four conditions that we used, the third condition was
hypothesized to have the best balance between information and load, and thus people in this
condition were thought to demonstrate the lowest times for completion, better accuracy, and
more positive affect measures.
Statistical evidence demonstrated for the curvilinear relationship for the first hypothesis,
where time to complete was significantly higher in the first, second, and fourth condition as
compared to our third condition. From this finding, we extrapolate that people in the third
condition were able to more quickly gather relevant information from each graph than people in
the other three conditions. According to CTL theory (Sweller et al., 1998; Mayer & Moreno,
2003), this would be explained by people in the first two conditions having high cognitive load,
being forced to use a relatively larger amount of effort to find relevant information with certain
desirable elements absent. Conversely, those that were in the fourth condition would be
overloaded by extraneous factors of the graph that distract their attention from relevant
information, and thus also increases their time to identify the correct response.
The second hypothesis concerning a curvilinear relationship with accuracy was not
validated, and instead there was a positive linear relationship between accuracy and amount of
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
16
information provided. Thus, those that were in the fourth condition, who were given the
supposed overabundance of information, were the actually the most accurate, with the third,
second, and first condition following in descending order. Though this does not coincide with
our overarching hypothesis of too much information and too little information similarly
decreasing performance, this finding does at least validate that the information added to each
graph was relevant and useful to the task. The fact remains that the fourth condition was similar
to the two low-information conditions in taking significantly longer than the third condition (the
first hypothesis), and so the combination of high information content and greater time devoted
may be indicative of high germane cognitive load (Sweller et al., 1998). Germane cognitive load
is when working memory is taxed in such a way that high effort increases learning, and this is
likely what occurred when diligent respondents used more effort (measured in time) as a means
to be more accurate. And so, even though our graphs are inherently a tool for learning, the aim
of the present study was to capture the least taxing to highest performance template for graphical
display, and the results from the fourth condition may demonstrate superior accuracy with an
unnecessarily high degree of effort. This effect may be an interesting avenue for further research
into the relationship between pedagogy and graphical display, as desired use and target audience
may have interesting interactions for optimal information present in data representation.
The third hypothesis explored whether measures of affect would vary with the different
degrees of cognitive load that people in each condition experienced. Similar to findings related
to the second hypothesis, there was no curvilinear relationship found, but there was a significant
difference between the lower information conditions and the higher information conditions in
terms of confusion and frustration. People tended to be significantly more frustrated by the
second condition than the third condition, and also more confused by the first and second
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
17
condition as compared to the third and fourth. Thus, there seems to be a trend of more
information leading to less confusion and frustration in terms of our graphical displays. The fact
that people took the most time in the first, second, and fourth condition, but people were the most
accurate in the fourth condition, seems to show that in the first two conditions the amount of time
was more indicative of failed attempts to find relevant information, whereas the fourth condition
was more indicative of effectively sifting through an abundance of relevant information. Thus,
relating these measures to time, our participants seemed to demonstrate that frustration and
confusion is strongest when great effort leads to little validation of a correct response.
Conversely, even though people in the fourth condition were called to put more effort into
finding information, their perceived accuracy seemed to mitigate any frustration or confusion.
Further research is necessary to validate these additionally theories of affect measures related to
graphical displays, as this study was more geared to control comparisons between conditions in
terms of time and accuracy.
Conclusion, Limitations, and Future Directions
One of the greatest limitations to our study is that it is very difficult (or impossible) to
design graphs with quantifiable intervals of cognitive load. The current study presented
conditions that were at best ordinal in terms of the degree of cognitive load. The four conditions
were devised using various tenets of data presentation, and were informed guesses at
manipulating interactions between cognitive load and content. Thus, there is much room to
improve and test our hypotheses, especially in terms of manipulating the content of each graph,
and which components are the most meaningful for improvement. Our study did demonstrate one
component that seemed to make a significant difference between the second and third condition.
Examination of the graphs of the second and third condition reveals there is actually only one
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
18
functional difference between the two, which is that there are dots corresponding for each data
point for each line extending along the x-axis—and this seemingly small improvement
generated a significant effect. Further research that aims at breaking down components of
effective graphs would elucidate best practices in graphical displays, as well as possibly
generating larger and more precise effects of cognitive load than we were able to uncover.
Another limitation that may have affected participant’s accuracy and time was that
participants had the capability to correct responses that they realized were incorrect after
answering a later question. This is a problem for two reasons, firstly, this was a potential
confound in the sense that some people may have corrected their responses, and others may not
have, and we have no way of grouping people on this parameter to see if it had an effect.
Secondly, any corrected responses that were originally incorrectly marked because of issues with
cognitive load, and so, it is possible that some of the measures of accuracy (and in turn time
spent) are inflated. A design where people are either not allowed to correct their responses once
given, or where they are given each question in isolation with the graph would be recommended
for future research.
The ability to efficiently process graphs is arguably one of the most important aspects of
a graphical representation. It is quite a familiar experience when one finds graphs to be
intrinsically difficult to piece apart, or the opposite, where relevant information presented in a
graph seemed to be highly salient. Using tenets of cognitive psychology, such as theories of
cognitive load, we can reveal components of graphical representations that are the most
conducive for information processing, and those that are unnecessarily mentally taxing. Our
study was successful at demonstrating a curvilinear effect between information presented and
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
19
time, which opens the doors for further research on finding the effective balance between ink and
data (Wainer, 1984).
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
20
References
Crano, W. D. & Brewer, M. B. (2002). Principles and methods of social research. New York,
NY: Psychology Press.
Dillman, D. A. (2007). Mail and internet surveys: The tailored design method. Hoboken, NJ:
John Wiley & Sons, Inc.
Henry, G.T. (1995) Graphing data: Techniques for display and analysis. Applied Social
Research Methods Series, Vol. 36. Thousand Oaks, CA: Sage Publications.
Jacoby, W. G. (1997). Statistical graphics for univariate and bivariate data. Thousand Oaks,
CA: Sage Publications, Inc.
Kalyuga, S. (2011). Cognitive load theory: How many types of load does it really need?.
Educational Psychology Review, 23(1), 1-19. doi:10.1007/s10648-010-9150-7
Klass, G. (2002) Creating Good Charts. Downloaded 24 August 2010 from
http://lilt.ilstu.edu/gmklass/pos138/datadisplay/sections/goodcharts.htm
Mayer, R. E. & Moreno, R. Nine ways to reduce cognitive load in multimedia learning.
Educational Psychologist, 38(1). 43-52.
Nicol, A. A. M., & Pexman, P. M. (2010). Displaying Your Findings: A Practical Guide for
Creating Figures, Posters, and Presentations. Washington, DC: American Psychological
Association.
Robbins, N.B. (2005). Creating more effective graphs. Hoboken, NJ: Wiley
Sudman, S., Bradburn, N. M. & Schwartz, N. (1996). Thinking about answers: The application
of cognitive processes to survey methodology. San Francisco, CA: Jossey-Bass, Inc.
Sweller, J., van Merriënboer, J. J. G., & Paas, F. (1998). Cognitive architecture and instructional
design. Educational Psychology Review, 10, 251-296.
Tufte, E.R. (1983). The visual display of quantitative information. Cheshire, CT: Graphics Press.
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Tufte, E.R. (1990). Envisioning information. Cheshire, CT: Graphics Press.
Tufte, E.R. (1997). Graphical explanations. Cheshire, CT: Graphics Press.
Wainer, H. (1984). How to display data badly. American Statistician, 38, 137-147.
Wainer, H. (1997). Visual revelations. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
Zawitz, M. (2000). Data presentation: A guide to good graphics.
http://www.scs.gmu.edu/~wss/methods/zawitzg.html
21
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Tables
22
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Appendix A: Graphs Given to Participants in each Condition
Graph presented to participants in the “High Load, Low Info” condition (Condition 1).
Graph presented to participants in the “Medium Load, Medium Info” condition (Condition 2).
23
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Graph presented to participants in the “Low Load, Optimal Info” condition (Condition 3).
Graph presented to participants in the “High Load, High Info” condition (Condition 4).
24
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
25
Appendix B: Standardized Questions in each Condition
Note: Questions were presented in random order with the font style and size shown below, and
are therefore not numbered.
How many times did the leader of cumulative total medals (versus gold medals) change during the 2048
games?






Once
Twice
Three times
Four times
Five times
Six times
About how many gold medals did Country 1 have at the end of the 9th day at the 2048 Olympics?






35
32
30
28
25
20
On what day were Country 1 and Country 2 tied for gold medals during the 2048 Olympics?






6
7
8
9
10
11
What was Country 1’s total medal count after day 16?






88
104
92
61
38
47
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
26
How many times did Country 1 and Country 2 tie for total medals won during the 2048 Olympics?






Once
Twice
Three times
Four times
Five times
Six times
How many times did Country 1 and Country 2 tie for gold medals won during the 2048 Olympics?






Once
Twice
Three times
Four times
Five times
Six times
What was Country 2’s gold medal count after day 15?






32
38
40
44
47
50
How wide is the difference between Country 1 and Country 2 in terms of cumulative total medals won
during the 2048 Olympic Games?






6 medals
16 medals
27 medals
32 medals
47 medals
49 medals
On how many days was Country 2 ahead of Country 1 with regards to gold medals won?






3 days
4 days
5 days
6 days
7 days
8 days
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
27
On how many days was Country 1 ahead of Country 2 with regards to total medals won?






3 days
4 days
5 days
6 days
7 days
8 days
The following question was presented with questions and was also incorporated into the random
presentation. It is meant to monitor data quality (in terms of how closely participants were
paying attention.
Please select "67" for this question.






37
47
57
67
77
87
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
Appendix C: Affective Measures
28
DIFFERENTIAL EFFECTS OF COGNITIVE LOAD
29
Appendix D: Instructions (“Tickling”)
Note: Directions were presented using the font style and size shown below.
Welcome to the survey! Please read the following instructions carefully. As it happens, the authors
have invented a time machine. This time machine is extremely advanced, and is able to look into the
future and choose important information about future events to send back to us. Unfortunately, this
time machine is malfunctioning, and it has begun to send back information outside the control of its
creators. We need your help to understand the information. It looks like this time it wants to tell us
about the 2048 Olympic Games. It appears that they will be held on a lunar colony, and this will be the
first time they have ever been hosted off-world. For some reason, the information is being depicted in a
graph. Please analyze the graph carefully and answer the questions provided.