Review of I 3 Applets and Tutorials
Randall Pruim
December 2007
Notes about the reviewer
This review was done on a Mac (OS 10.4.11) using the Firefox browser. Occassionally, pages were also
viewed in the Safari browser.
I have taught undergraduate statistics at the 100-, 200- and 300- levels at liberal arts college, as well
as course for graduate students in public health at a major state university. I have have experience with a
variety of texts, including ISCAT. For 200- and 300-level courses I usually use the statistical package R. For
lower level courses I have used a variety of tools.
I have also used, modified, and written quite a number of mathlets (some java, some not) over the last
decade.
1
Binomial Distribution
1.1
Pre-Test
1. I would prefer the first question ask which is not a requirement. Also, your choices are all simply
negations of the the true conditions. It might be better to have the conditions be more natural.
Alternatively, this could be done as a sequence of true/false items.
2. Graph matching
(a) n and p are not really properties of a binomial distribution (and even less of the graphs). Why
not just use the word parameter here?
Possible wording. Below are probability histograms of different binomial distributions. Select the
correct values of n and p for each.
(b) “Match to the correct graph” is a poor description of what is going on here. You aren’t picking
a “correct graph”, you are picking a correct set of parameters.
3. Shape question:
(a) Items A and D are both “true” (if one interprets A to mean that the probability is fixed). I don’t
like multiple choice responses like this.
(b) These choises are only half expressed using comparison language: number is high, distribution is
closer.
4. Time of course item.
This is worded awkwardly. How about When was your last statistics course? with choices never, less
than 1 year ago, 1–3 years ago, more than 3 years ago.
1
1.2
Applet
Here’s a screen shot on my machine (Firebox above, Safari below).
The basics of the applet are OK, but there are a number of details that could be improved.
1. There is not always enough room allowed for the text (see the labels of the y-axis in my screen shots).
2
2. The applet is smashed up against the lower right edge of the browswer window. Some space would be
good. (Note how the browser scroll bar and the applet scroll bar are right on top of each other in the
next screen shot.)
3. The tabular display at the bottom is very compressed, so it is pretty hard to get a feel for the data by
scrolling. I can only ever see two values at a time. There is not even enough room for the whole scroll
bar.
Thanks to Firebug, I was able to look at a longer version:
That gives a better sense of what is going on. Perhaps the default can be somewhere in between. Or
perhaps the data view can be opened optionally in a separate window?
4. I was surprised to find the mean and standard deviation at the bottom, and it is unclear (from
the display) what they are the mean and standard deviation of : one sample? (which ?) sampling
distribution? (what statistic?)
The layout makes it look like they are the mean and standard deviation of the theoretical probabilities.
Perhaps this information can simply be moved to a different location on the applet.
5. Probability would be a better label than Distribution. Neither is a good label for the mean and median
at the bottom.
OK, I’ve finally figured out what Data means. So you have columns for the theoretical and empricial
probabilities (based on whatever samples were drawn in the applet.)
I realize space is precious, but these are not great labels.
6. Can you color coordinate so that numbers in the table are the same color as the corresponding objects
in the graph?
3
7. The red vertical bar is pretty ugly. I realize this is a sclaing issue when the number of replicates is
small, but it is still ugly. Perhaps the scaling should be dynamic so that the tops of all the bars are
always visible.
8. Does the applet show me anywhere how many replicates there have been?
9. M is a pretty strange choice for proportion of success, and it took me a while to figure that out. (The
hover help is nice, but I didn’t find it right away.)
Added later: the hover help boxes get to be bother sometimes. Even in your flash video (which I have
watched but without sound because I am proctoring an exam) shows this. Perhaps they should turn
back off after a certain delay.
10. When you do multiple runs at once, only one of them gets added to the table on lower left. I realize
that you might not want to store and display 1000’s of runs, but I think it would be better to pick how
many you can store (s) and always keep the last s runs around.
If storage is not an issue, keep them all. The user can scroll around and get a sense that there were
actually 10,000 runs (or however many).
11. Do you ever explain what the little box-without-whiskers plot is? (For that matter, do you ever make
use of it?) It appears to identify the mean and standard deviation. The display looks much like a
boxplot, which would show quartiles. When I hover on it, I get “X distribution” as a response.
An example Applet
Here is a screen shot of my favorite statists applet (from the Rice Virtual Statistics Lab).
4
Notice
• Color coordination
• Clear labeling
• Use of space to side of graph for summary statistics
• Separate graphs rather than overlayed graphs
Perhaps you cannot do all of these in this situation (they don’t have to leave room for the tabular dislay of
the data), but some of them might improve both the look-and-feel and the usability of your applet.
1.3
Lesson
1. The lesson seems to be primarily targeted at getting students to understand the variety of shapes a
binomial distriubtion can have.
2. The questions are quite open-ended but could be made more focussed by addressing specifically the
symmetry and spread in the questions.
3. Width and height questions are a bit subtle since your applet rescales the axes as you go along. This
means that as you move the slider to increase n, the image gets narrower but the distribution is getting
“wider”. The height of the image never changes. This may be too subtle for students. Perhaps you
should add a lead in question that serves to make sure they are aware that the scale is changing.
4. The tabular display isn’t ever used. (Hide it, perhaps?)
2
Randomization
2.1
Pre-Test
I like the tripping video. It makes the experiment easier to understand/visualize. There should be a link to
it from the lesson.
2.2
Tutorial
1. description of two groups: indpendent of other information, it sounds like you are observing whether
folks use an elevating or lowering strategy rather than assigning them to the groups. This is easily
fixed with a small change in the audio text. Instead of saying “men who used the lowering strategy”,
how about “men assigned to the lowering strategy.”
You actually do this later in the tutorial. It would be good to do it from the start.
2. Why are there dots at both .33 and −.33 in the dot plot?
2.3
Lesson
1. Tutorial says that animation is turned off with multiple replications. Lesson tells student to turn it
off manually. If you aren’t able to animate multiple replications, then disabling animation should be
automatic and the lesson should be adjusted. Else the tutorial should be adjusted to match the actual
behavior of the applet.
2. Describe the basic shape of the differences → Describe the basic shape of the dotplot (?)
3. Describe the basic shape of the differences → Describe the basic shape of the distribution (?)
5
4. The lesson is very pedantic and prescriptive, but seems to get at the point that using randomization
to assign groups tends to balance things for various covariates (categorical or quantiative, known or
unknown).
2.4
Applet
1. On my screen the image quality is poor. The text is fuzzy and a bit hard to read. It looks to me like
you are not actually running the applet but are running flash with a “pre-recorded” applet session.
That probably explains the fuzziness. Can it be improved?
2. It looks like you have simply taken the Rossman/Chance applet and added a layer of control over top.
Is the intent simply to provide an introduction to that applet?
You could include a link out to the unrestricted version of the applet.
3. It is a bit strange that the “applet” only allows you to do just the right thing and in the correct order.
I wanted to explore some other features, but it doesn’t let me. I wanted to get to 200 replications by
clicking randomize 3 times without changing from 50 to 150 replications; no go. This has advantages
and disadvantages, I suppose.
4. The actual applet
I don’t know if you have any resources to modify the underlying applet, or if you are in contact with
Rossman/Chance about this, but here are some comments on the applet itself.
(a) Why have question marks after the middle radio button labels?
(b) Label the card display. Something like “Last Randomization”, or “Run n”.
6
3
Confidence Intervals
3.1
Lesson
Status Quo
Using statistical software it is easy to determine
that this distribution is approximately normal,
has a mean of 39.8 inches.
item # 1
. . . change z with sigma to z with s; . . . Is this
percent approximately 90%?
Continue sampling until you have 1000 confidence intervals.
Press the Sort button. What types of X values
lead to intervals that fail to capture µ? Are all
of the intervals the same length?
Comment/Suggestion
bad grammar
a tabluar display of what settings need to be adjusted would make this easier to read.
I think it would be better to have your lesson
include all three methods (z with sigma, z with
s, then t) and have students compare the results
that they get using the three methods.
about or at least 1000
The use of some headers in the text layout might
make the overall structure of the lesson easier to
see.
I like this sequence of questions, but it could be
improved.
• It might be a bit unclear to students if they
should do this with the z with s intervals,
the t intervals or both.
• X should be x? (are we focussing on the
simulated data displayed or on X as a random variable?)
• You are all set up to talk about how both x
and s affect whether a confidence interval
covers the paramter value, but you don’t
go there.
3.2
Applet
I’m not sure if you are planning to revise this applet (or even if you have access to do so), but here are a
few comments.
1. The applet could be used to explore robustness of these procedures if there were an option to select
other distribution shapes.
2. Textboxes are not large enough to contain the fraction and proportion when the number of CI’s get
large; the proportion gets cut off in these cases.
7
4
4.1
Friendly Observer
Pre-Test
Status Quo
Define “statistical significance” in the context of
determining if there is a difference between the
population averages for two independent groups.
• I’ve never heard of this term
Comment/Suggestion
The radio buttons don’t fit the item description.
Make the free form answer part of the response
for the last option and rewrite the question to
ask about their familiarity with the term.
Same comment for the next item.
• I’ve heard of this term, but I dont really
know what it means.
• I’have a pretty good understanding of this
term.
4.2
Review
Status Quo
no navigation
We start by assuming . . .
4.3
Comment/Suggestion
This is really more of an introduction than a review, isn’t it? A review should look back at something already learned. This looks forward to the
tutorial to come.
it would be nice to be able to back up, pause,
etc.
When doing Fisher’s exact test we don’t assume
indpendence. The big idea is to compare our
results with what would have been expected if
independence held. The audio could be clarified
on this point.
You will need to decide about how/when/if to
use subjunctive throughout.
Tutorial
Status Quo
You may recall [researcher data] . . .
people icons cover up applet text
Comment/Suggestion
recall from where?
raise them, or have them go away after a little
while
audio seems less polished on this one
8
4.4
Lesson
Status Quo
Use the relative frequency distribution in question 3 to calculate the weighted average of the
number of winners randomly assigned to group.
Use this weighted mean calculator if needed or
use your TI to compute the weighted mean.
Keep in mind that these results assume the observer’s incentive has no effect.
4.5
Comment/Suggestion
If I recall correctly, you are following ISCAT language here, but this is not a weighted average. It
is just the average (calculated from the relative
frequency table). I’d prefer that you emphasize
that this is just a shortcut for actually adding all
1000 values and gives the same result, not something new.
replace with simulations to emphasize the difference from the actual results of the study
(where no such assumption is made).
Applet
Something seemed broken here on my first two (re)loads:
On the third try it “worked,” but my plot doesn’t appear when I add 95 replications:
9
The plot shows up when I click ’show tallies’ (but the tallies do not). As the audio notes, there are some
spacing issues in this applet.
5
Summary Comments
5.1
Why I 3 ?
I didn’t see anywhere on the pages I looked at where you explain this name. In fact, it is hard to tell whether
it is an ‘I’ or an ‘L’.
5.2
Prerequisites
It is a bit unclear to me when you envision these lessons being used. That is, it is not always clear to me
what you expect the students to know coming in, what they are learning through the lesson, etc. Do you
have a clear idea? Perhaps this could be articulated on a page for instructors.
5.3
Layout, Look-and-Feel
1. The pretest requires scrolling to view, but there is empty space at the left margin. This is awkward.
2. Sometimes you have lessons on the left and applets on the right. Sometimes this is reversed. Is there
a reason for the inconsistency?
3. When the lesson is on the left, the scroll bar ends up in the middle of the browser.
• Advantage: It’s not next to the browser scroll bar.
• Disadvantage: It is an unusual location for a scroll bar.
10
4. Alternative layout approach: Anchor the applet, and use the browser scroll bar to move everything
except the anchored applet.
This reduces the number of scroll bars, gives users a familiar interface, and allows you to keep the
applet viewable as they scroll. I think it would also allow for keyboard scrolling while the applet is
focused. As is, if you click of the lesson frame (say, to adjust something in the applet) the keyboard
arrows no longer scroll the lesson text.
Hmm. This may have issues with left-right-scrolling.
5. Other alternatives to consider:
(a) Applet and lesson in separate windows.
Let the user arrange as they choose.
(b) Applet above, lesson below.
The current layout requires a very wide window to see everything. A vertical arrangement may
require too much space vertically, but in general, I hate scrolling horizontally (because you have to
do it for every line). So if things can’t fit on the screen all at once, I would rather scroll vertically
than horizontally.
The horizontal scrolling would be less of an issue if you avoided fixed width layouts.
5.4
Audio
1. The audio is nice, but there are situations where it would also be nice to be able to work through
things without audio (say in a public lab space without headsets). In particular, things like the short
directions for the lesson in the randomization unit should also appear (or be obtainable) as text.
2. The audio has numerous hitches where the speaker stumbles or sounds distracted by running the applet.
Any chance the audio can be rerecorded (the video could be used as is if the audio is done with care).
3. There are places where the audio text is sloppy and could be improved.
Here are some examples from the binomial tutorial, but there are examples throughout the audio.
• vague phrases like “that number” and “the axis” (use horizontal or vertical)
• sloppy phrases like “0 is .125” when you mean the probability of getting 0 successes is 0.125 and
“the height of this graph” when you mean the height of one bar in the graph.
Again, any chance it can be rerecorded? It might be worth writing the text in advance so that it is
more precise and flows better.
5.5
Navigation
Some portions of the materials have navigation controls. Others do not. Having the ability to back up and
rehear a bit of the audio can be especially nice.
5.6
Giving instructors more control
I don’t know your plans for the eventual use and dissemination of these applets, but if you design the applets
to take a number of parameters and provide just a bit of instruction, others (instructors or their IT people)
could post your applets on other sites with different defaults and behaviors.
Examples of flexibility that could be make available via the applet tag:
• The data display in the binomial applet could be optional.
11
• The default values of the sliders could be specified.
• Some of the sliders could be optionally fixed at specified values.
For an example of an applet that does this sort of thing, take a look at http://www.calvin.edu/
~rpruim/courses/m162/S02/mathlets/SequencesOfFunctions.shtml. The applet tag for the applet used
there is given below. (Some line breaks have been added for readability.)
<applet archive="jcm1.0-config.jar"
code="SequencesOfFunctions"
width=600 height=470>
<param name="UseFunctionInput" value="yes">
<param name="UseMouseZoom" value="yes">
<param name="UseLoadButton" value="no">
<param name="UseLimitsPanel" value="yes">
<param name="UseGrid" value="yes">
<param name="UseZoomButtons" value="yes">
<param name="UseRestoreButton" value="yes">
<param name="FunctionCount" value="3">
<param name="TwoInputColumns" value="no">
<param name="Example1"
value="1/(1-x) = 1 + x + x^2 + ...;
-2 2 -2 2, 0 100 0; 1/(1-x) ; sum(i,0,n, x^i)">
<param name="Example2"
value="1/(1-x^2) = 1 + x^2 + x^4 + ...;
-2 2 -2 2, 0 100 0; 1/(1-x^2) ; sum(i,0,n, x^(2*i))">
<param name="Example3"
value="(1+x^2)/(1-x^2) = 1 + 2x^2 + 2x^4 + ...;
-2 2 -2 2, 0 100 0; (1+x^2)/(1-x^2) ;
1 + sum(i,1,n, 2*x^(2*i))">
<param name="Example4"
value="arctan(x) = x - x^3/3 + x^5/5 - x^7/7 + ... ;
-2 2 -2 2, 0 100 0; arctan(x);
sum(i,0,n, ( (-1)^(i) * x^((2*i)+1) ) / (2*i+1) )">
<param name="Example5"
value="cos(x) = 1 - x^2/2! + x^4/4! - x^6/6! + ... ;
-2 2 -2 2, 0 100 0; cos(x);
sum(i,0,n,(-1)^i * x^(2*i)/(2*i)!) ">
<param name="Example6"
value="sin(x) = x - x^3/3! + x^5/5! - x^7/7! + ... ;
-2 2 -2 2, 0 100 0; sin(x);
sum(i,0,n,(-1)^i * x^(2*i+1)/(2*i+1)!) ">
<param name="Example7"
value="e^(-x^2) = 1 - x^2 + x^4/2 - x^6/3! + x^8/4! ... ;
-4 4 -2 2, 0 100 0; e^(-x^2);
sum(i,0,n, (-1)^i*(x)^(2*i)/i!) ">
<param name="Example8"
value="2 power series for 1/(2+x); -3 3 -2 4, 0 100 0;
1/(2+x); sum(i,0,n, (-x-1)^i);
sum(i,0,n,(1/2)*(-x/2)^i) ">
<param name="GraphColor1" value="black">
<param name="GraphColor2" value="red">
<param name="GraphColor3" value="blue">
</applet>
12
For more examples, check out David Eck’s Java Components for Mathematics.
5.7
Overall
The applets and lessons are OK, but not great. There is nothing particularly novel to them (especially after
realizing that 3 of the applets already existed), and they require polish. Once polished, they should provide
useful tools for instruction in introductory statistics courses.
Since three of the four applets existed, and the other is simply a visualization tool for the binomial
distribution, the primary contribution of this project is the lessons. The primary benefit of these is that
students can work through them on their own (provided they have a space where listening to the audio is
not disruptive).
There is no particular reason to marry any of these units to the ISCAT book. These units clearly work
well with that text, since three of the applets come right from the ISCAT site. But there is no reason to
rely on that text for anything here. It will be more widely useable if it remains unlinked, and the authors
should take care to avoid unnecessary dependencies.
13