Technology and Learning
Alfred Bork
Donald Bren School of Information and Computer Science
University of California
Irvine, CA 92697-3425
[email protected]
Xiwen Zhang
California State University, San Bernardino
San Bernardino, CA
Carole Bagley
Distinguished Service Professor
University of St Thomas
St. Paul, MN
[email protected]
The potential of technology, particularly the computer, to aid learning has been
frequently proclaimed to be great, by some of us and by many others. So why has
technology not played a major role in assisting learning? Somehow the potential of
technology to greatly improve learning has never been realized. This chapter
investigates why this is so. Then it introduces a form of computer-based learning
1| Technology and Learning
technology that has the potential to lead to an order of magnitude improvement in
learning globally.
We begin with some history of learning development in the United States, and then
discuss the work using computers in learning environments at the University of
California, Irvine.
A major curriculum effort before computers
A major effort in curriculum development in the United States occurred after the USSR
Sputnik was launched. The success of this Russian mission, before the United States
had launched an earth-orbiting satellite, convinced many people in the United States
that they had a major need to improve learning, to be competitive.
The funding for this post-sputnik curriculum development was large, with most courses
receiving millions. Book-based courses were developed at all levels, from elementary to
university. The concentration was on science and mathematics. All of this development
involved pre computer technology, given its time. Toward the end of this effort several
unsuccessful attempts were made to obtain funding for developing related computerbased learning units.
This post-Sputnik movement died quickly, all do to one course that had major political
problems. Few of the courses survived, in spite of this enormously expensive effort. A
major problem was that teachers were not prepared for the new learning approaches,
and the vast sums spent for training and retraining teachers, far more than on
development, did not do the job. Training of teachers has frequently been the Achilles
heel of improving learning. ADD REF
Another problem was that some of the material, such as the interesting elementary
science courses developed at Berkeley and MIT, demanded too much individualized
attention, usually not possible in the classroom environment. In some cases, as with the
Harvard Project Physics course, the publishers would not publish all the material that
had been developed. Other reasons were also proposed for the failure of this expensive
effort.
Learning and Technology
Technology, particularly the computer, provided new possibilities for learning. Some of
us have been personally involved with technology and learning since 1958. From that
time until now many directions using computers have been tried.
Major projects such as Plato and Ticcit in the United States had large funding from the
government and the National Science Foundation. In the early 70’s, PLATO was one of
the first Computer assisted instruction systems, originally built by the University of
2| Technology and Learning
Illinois and later taken over by Control Data Corporation. PLATO pioneered concepts
such as online forums and message boards, learning management, online testing,
email, chat rooms, instant messaging, remote screen sharing, and online games.
In the mid-70’s, the MITRE Corporation developed and implemented the TICCIT system
as a Computer-Assisted Instruction (CAI) system for community colleges. MITRE
subcontracted with the CAI Laboratory at the University of Texas at Austin and also with
the Instructional Research and Development Department of Brigham Young University
to refine the user interface and create the massive amounts of courseware needed for
a complete college-level English and algebra course. The courses were self-paced.
Many smaller projects also received substantial support. States and school districts
have spent very large sums on acquiring computer technology, both hardware and
software, and on training teachers to use this technology. Minnesota Educational
Computing Consortium was a state-funded company founded in 1973 and was a
national leader in providing educational computing hardware, software and courseware
for schools AND provided training for teachers in how to use technology. The state of
Minnesota spun off the company as a private corporation in the late 1980s I
Work in Irvine
We review briefly in this section work at the University of California, Irvine. With NSF
support beginning in 1968 we developed some rewarding early projects. One concerned
the phases of the moon. Students constructed a geometrical model based on observed
data, and then used it to make predictions in new situations. ADD REF
Introductory University physics course
Our first extended project using computers to assist learning was one quarter of a
beginning calculus-based university physics course, about 1970. This course was
based on the Keller plan, the Personalized System of Instruction, which required that, at
each point of the course, students should not move forward until the material already
covered was fully mastered. This reasonable strategy has not been used recently.
The course we developed was based on a set of online exams with the questions
coming from problem generators. Each exam covered about one weeks work, but there
was no fixed pace; students took an exam when they wanted to. The students might
need to take an exam several times, until they did perfectly on it. We never gave the
same problem twice, because all problems came from the problem generators. No
multiple choice was used.
Help was available to students in trouble, in several ways. The most important help was
available directly online as the student was taking the exam. This assistance was
based on incorrect student responses to the problems, often showing the nature of the
3| Technology and Learning
problems the students were having. So the computer system moved smoothly from
exams to learning assistance.
This course ran on a time-sharing machine, a Sigma seven. The emphasis on mastery
continued to guide our later projects at Irvine, even though we did not use the Keller
plan. It received some interesting national publicity, but timesharing meant that it could
not be made widely available.
Scientific Reasoning Series
With the coming of personal computers, we moved in new directions. The Scientific
Reasoning Series, about twenty hours of adaptive learning units, was the first effort. It
was marketed by IBM through their K-12 division. Funding for development came from
NSF and FIPSE. The aim of the units was to help young students to think like scientists.
The typical audience was students of about eleven years old. The pace through the
material varied from student to student. The ten programs were highly interactive and
conversational.
One important direction in the Scientific Reasoning Series was to allow students to
discover their own knowledge. In one of the ten modules, for example, students
discover the laws of genetics. In another student discovered the laws of simple electrical
circuits. In a third program in the Series students invented the concept of heat. We
followed this strategy of student discovery in later work.
In spite of the success of these units we could not find support to update these modules
and to develop further modules in a similar direction. As we will mention, this is a
frequent problem with the use of technology in learning. We would like to bring the
graphics up to modern standards, and to add speech input, but could not find funding
for these activities in spite of vigorous efforts over twenty years.
MORE??
Understanding Spoken Japanese
A later project, Understanding Spoken Japanese, was developed with funding and
support from Fujitsu and Nippon Television Network. Each module was based on a
video sequence, made in Japan, but for reasons that will be discussed later we only
used a few seconds of video at any one time. Many features of this project were similar
to those in our earlier work, including individual pacing for each student. We were at
work on the second set of ten modules when troubles with the Japanese economic
system ended the project.
In a typical sequence a bit of video is played, and then the student is asked what the
people in the video were discussing. If the student knows, we proceed to another
4| Technology and Learning
question. If not, we backtrack in student learning as much as is needed. We may get
down to helping the student in recognizing individual Japanese words. Thus we never
‘teach’ something to someone who already knows it! This has been important in our
later work.
Another similar strategy was also used. The student had access to a small piece of
video, with controls similar to those on a video player allowing the student to move back
and forward. Students were asked to find locations where the people in the video were
talking about XXX. Again help was given when necessary.
The tactics used in these programs could be used in other language learning situations,
such as in ESL activities. For example, they would be very useful in Japan now, since
they have decided that all students should learn English in primary school.
Technology and improving learning
Similar situations using technology for learning existed outside the United States. What
were (and are) the results of spending many billions of dollars, globally, in using
technology in learning?
Technology so far has not helped learning
The results of using technology, particularly computers, in learning have so far not been
impressive. A variety of studies and opinions have questioned the use of technology to
improve learning. Although it has been many years since computers have been used in
learning environments, there is little improvement in learning, with or without
technology. For example, a number of recent articles have commented that science
education is no better today than it was fifty years ago. I think one could argue that
learning in all subjects is worse now than fifty years ago, given the constant high stakes
testing of students with inferior methods such as multiple choice. This is in spite of the
hundreds of billions of dollars that have been spent ‘improving’ learning in fifty years,
including the vast sums spent on computers.
The market for computer-based software in learning has declined recently. This may
indicate that the school administrations and governing boards doubt that these
programs are worth the money they cost. The recent domination in the United States of
the No Child Left Behind program may also be a factor, unfortunately.
We now consider some of the reasons why technology has not led to improvements in
learning, globally.
5| Technology and Learning
Grabbing onto each new technology
Technology has a constant stream of new approaches and products, both hardware and
software. These products receive considerable marketing publicity. More often than not
teachers and developers appear to believe that each new technology is the key to
improving learning. Among the many examples of this are Basic, time sharing, the
personal computer, the mouse, computer games, email, artificial intelligence, objects,
the internet, higher speed internet connections, open software, Linux, blogs, faster
processors, wikis, instant messaging, the semantic web, and many others.
As one example of such grabbing, consider faster internet access. Many people state
that broadband access for students and teachers will lead to better learning. But no
convincing argument for this additional cost is offered. Another example is learning
objects, used in ways that have little connection with the use of objects in computer
languages. As far as we know, no full courses have ever been developed with objects
not those of the developer.
Failure to continue successful developments
Funders often look for something new. So they do not follow up on successful
approaches. We have had that problem with followups to the Scientific Reasoning
Series, for example. In a few cases, such as with Plato, the strategies are picked up by
commercial sources, but this is still rare.
Poor evaluations
Many education innovations have no or inadequate evaluations. Sometimes only
student opinion polls are presented as evidence, a useless approach because it tells us
little about the learning effects of the innovations. Even typical university-based
evaluations by faculty in education and related areas are mostly useless. The usual
education experiment involves about a hundred students with one treatment, and a
similar number with another one, usually in only one or two locations. The model is a
drug-testing model, but with far fewer people.
Given the wide variety of students, with different learning approaches and capabilities,
in a county or in the world, these numbers (hundreds of students) are entirely
inadequate. At best these evaluations purport to show very small improvements in
learning, not the major improvements we should be attaining with computers. Further
this type of evaluation often shows only the Hawthorne effect of something different.
Numbers in good learning evaluations should be in the thousands, in many different
environments; an example of an evaluation with adequate numbers was that of Writing
to Read, conducted by Educational Testing Service.
6| Technology and Learning
Failure to relate technology to learning
One failure of each of these ‘new’ approaches is that they are driven by the technology
rather than by learning. They do not start with the problems of learning, but only with
recent developments in technology. That is a backwards approach, if we are to see
major improvements in learning. We need to begin with the learning problems that we
face, not with new developments ub technology.
What is missing today in learning?
Often in workshops in many parts of the world I begin the session by asking the
participants about what is wrong with learning today. In these brainstorming discussions
I do not mention computers. The results everywhere are surprisingly similar.
One factor that always comes up is that we do not adapt the learning process to the
wide variety of students, with many different backgrounds, learning styles, and interests,
trying to learn. Educational psychologists mostly agree that students differ greatly from
a learning point of view. Very few teachers or professors can adapt learning to each
student in the typical large classes we have today in either schools or universities,
because there are so many students in the room. In the United States and elsewhere
this problem is complicated by the emphasis on multiple choice tests. This suggests the
need for both better learning methods and better learning environments.
A related serious problem raised in these discussions is that many students do not
learn, or learn only partially, in current learning situations. Our grading systems show
this, as all students do not make an A. Oddly, many in universities see the giving of
many low grades as showing that the learning system is of high quality. This is a
peculiar way to measure the quality of learning. A related problem is that this emphasis
on grades leads to widespread cheating.
Benjamin Bloom showed twenty five years ago, as reported in his two sigma paper, that
almost all students can learn to the mastery level, given the right learning environment.
But this important work is unfortunately unknown to most teachers and university
professors. In his experiments the most successful learning strategy was tutoring, to be
discussed further later in this chapter.
Another factor mentioned frequently in my workshops is that many students do not
enjoy learning. Students often say that they hate schools! So they are not interested in
learning. This failure to enjoy learning has devastating effects in the future on lifelong
learning.
Finally, we mention that we do no have an adequate theory of learning today, in the
sense of the powerful predictive theories in areas like physics. We do not seem to be
7| Technology and Learning
close to such a theory. Education schools tell students about many theories, but none of
them are adequate.
How can we overcome these learning problems with technology?
The problem in most attempts at improving learning is not with the technology, but with
how it has often been used, as suggested above. We now move toward a view of how
the reasonable use of computers in learning could lead to an order of magnitude
improvement. This may not be the only possibility, but it has good prospects.
First we consider adaptive learning, and then tutorial learning as a way to achieve
adaptive learning.
Adaptive learning
As mentioned a critical factor missing in most learning today is the capability to adapt
very frequently to each student. At any point in learning, each student is a unique
individual, in many ways, from the standpoint of learning. Several factors are important
in this adaptation to student needs and desires.
Each student should move at a unique pace
Given all the variations between student backgrounds, interests, and abilities, it is highly
desirable to have each student move at a unique pace in the learning units. We can in
the computer learning environment allow this, because computers have branching
capabilities, but it is close to impossible in conventional learning environments without
the appropriate programs, except with special teachers.
We expect many students in an adaptive learning environment to learn faster than at
present, for reasons that we will soon mention. This could be a major change in learning
for many students. It will also lead to more economical learning, and it will allow
individuals to become more productive members of society at an earlier age.
Adaption should be very frequent
The learning activity should adapt to each student on a moment-by-moment basis. So
changes based on occasional exams are inadequate. Students should feel that the
adaptive program is responding to them as individuals.
Each student should be successful in learning
A major advantage of the use of adaptive variable pace is that the students can
continue to learn in a given area until they have learned the material. This ia a critical
difference from most learning today. We know from Bloom’s research that almost all
8| Technology and Learning
learners can succeed, achieve mastery. The decisions in the program as to when
mastery has been achieved are all made by the design groups.
When something is successful learned the learner should move on
Often in classes a student has learned something, but the class continues working on
the topic, boring for the student. This will not occur in a fully adaptive learning
environment. This and the next item lead to faster learning for some students. We
expect this to be a major gain in time, but this cannot be verified empirically until we
have a full range of computer-based adaptive learning units.
No one should be taught something they already know
Another waste of student time in conventional learning systems today is that we teach
things that are already know to the student. As an example, consider students at an
early stage in mathematics. Such a student may or may not know how to count objects.
If they cannot, they need to learn this. If they can, learning can proceed to another topic.
But this is very difficult to achieve in the learning environments now available.
Adaptive tutorial learning for tomorrow
How can we attain adaptive learning for everyone in the world? The key factor is a
learning method over 2500 years old, tutoring. This is also the approach used in the
experiments of Bloom, described earlier. We cannot afford an excellent human tutor for
every student, but the computer can now be a tutor, making tutorial learning possible for
everyone. No tactics from artificial intelligence are required for the system proposed
here. But we need learning modules that follow this approach, in all levels of learning.
Socratic tutoring
The model for this proposed new approach to learning is the tutorial process used by
Socrates. He did not supply information to the small group of students with him, as
described by Plato, but only asked questions. Student answers prompted new
questions. Very difficult areas could be learined with a skilled tutor like Socrates.
Very frequent questions
The questioning in the tutorial learning environment is very frequent. In experiments
done many years ago at the University of California, Irvine, we found that to hold
student attention, and to react frequently to student needs, the time between questions
typically should not exceed twenty seconds. This means that students are always
active, interacting with the computer. The situation is like a conversation or a dialog.
Students are not usually given long passages to read. And they do not watch large
9| Technology and Learning
amounts of video, as noted in our discussion of the Understanding Spoken Japanese
work. Such activities would violate the twenty second rule.
Storing and using information about the student
Designers may decide at various points in the program to save information about
student performance. This information can be used later in the program, again as
decided by the designers. No attempt is made to build a model for each learner; we do
not now know enough about learning to create such a model. Such stored information is
also important in evaluation, and in research on better understanding the learning
process. These ideas will be discussed further.
The next question
A critical role of the designers is to analyze student responses, looking for various likely
correct answers for our intended student population. No use of artificial intelligence is
needed in this analysis. Designers also need to look for answers for which we can
immediately give some assistance to the student. This and other student messages
should be friendly help, not viewed as critical by the students. Based on this analysis,
and on previous student inputs, the designers decide what question to ask next.
Peer learning
Our studies have shown that it is best to have three or four people working together in
adaptive tutorial learning, to gain the important advantages of peer learning and social
interaction. We video taped groups of various sizes in this research. Details of these
activities are available.
Within a group, students may progress at a different rate. So occasionally the computer
will ask students to work alone, to see directly their progress. Based on this information
and on stored information about students, the computer will occasionally rearrange the
groups, so that each student will be in the best learning environment with others in the
group at the same level.
Working in groups of this size has important social advantages. There are very frequent
student interactions within the group, focused on the learning activity. As groups
change, students will work with many different students. Social interaction is superior to
that typically found in classes.
Invisible tests
Neither students nor teachers like tests. In the tutorial environment learning and testing
are part of the same process, so students are not conscious of taking tests. The
questions from the computer generate both learning and testing. This means also that
10 | T e c h n o l o g y a n d L e a r n i n g
cheating in the usual ways is impossible, eliminating another common problem in
learning today.
Enjoying learning
Lack of testing is one factor in enjoying learning, as students do not like tests. Success
in learning is another factor of the same kind; students enjoy success. Friendly
language is another factor, a requirement of the design process. Being sure that
students like to learn is another task of the designers. This can be verified in the
evaluation stage.
Discovery learning
One great advantage of adaptive tutorial learning is that students can discover much of
their own knowledge. This leads to better understanding of what is learned and better
retention. In this discovery situation, students behave like scientists.
There are several examples of such discovery in the Scientific Reasoning Series,
developed at the University of California, Irvine, about twenty years ago and marketed
by IBM. We have already mentioned these.
The development of computer programs that allow discovery of knowledge is yet
another function of the designers.
Creativity
Discovery of knowledge is one factor leading to more creative individuals. The
designers should also seek other ways to promote creativity, important for the future of
our global society..
Production of adaptive tutorial learning units
A complete system is available for developing adaptive tutorial modules. Full details on
this process are available.
Design
The most important stage is design, done by groups of excellent teachers and
researchers in the area under development. Many of the features of the work of the
designers have already been mentioned.
In the first stage of design, the overall form of the material is specified in outline form,
with a brief description of each module to be developed. These descriptions are the
basis of the next step, detailed design.
11 | T e c h n o l o g y a n d L e a r n i n g
The role of designers in this stage has already often been mentioned. These tasks are
mentioned briefly here.

Formulation of the questions asked by the computer and other computer
message

Specification of the analysis of student responses to questions

Decision of what message or question is to come next

Decision of what student information is to stored, and how this information is to
be used later in the program

Decision of when a student has achieved mastery in a given area

Keeping the student happy

If teachers or parents are available, the designers must plan for information to be
sent to them.

Designers, perhaps in special sessions, must design the strategy for the
formation of new learning groups, to keep peer learning groups at the same level
It should be noted that no use of artificial intelligence occurs. All decisions are made by
the designers.
Designs are recorded in a form called a script, either on paper or stored in the
computer.
Implementation
In this process we move from the script to a running program ready for evaluation, to be
discussed next.

If the script is stored in the computer, much of the code can be written by the
computer

Another possibility is that the script can be interpreted at run time

Some human programming will generally be needed

Media of all types is prepared by media specialists

Beta testing finds errors in the program and these are corrected
12 | T e c h n o l o g y a n d L e a r n i n g
Full evaluation and improvement before use
The final stage in producing adaptive tutorial learning units is evaluation. As designers
are people, they may not take all factors into account. So a critical stage in development
is evaluation of the units. This is followed by improvements based on the information
gained from evaluation.
Evaluation involves thousands of students. As they use the units, information is
recorded by the program. This extensive information may show factors overlooked by
the designers.

There may be reasonable answers to questions not thought of by the designers.

There may be unanticipated responses that should lead to immediate assistance

There may be sections where the time between questions is too long

There may be sections where students do not reach mastery in a reasonable
time.

We need information about how long it takes students to learn in this
environment, as compared to students in convention classroom environments.

Students may not enjoy learning
These and other problems suggest improvements. We recommend two cycles of
evaluation and improvement, with the second focusing primarily on the effectiveness of
the learning units.
Financial considerations
For such a new learning system to be widely adaptable, learning cost for a student hour
of learning must be reasonable. In looking at costs, it is important to consider all factors;
many considerations of cost miss some factor. Again, more details are available.
Not all the factors in costs for the new units are now known. Additional experience with
adaptive tutorial units is needed. The new system proposed will probably cost less when
large numbers of students are involved.
Adaptive tutorial units and global learning
The new learning strategy proposed here has great promise for lifelong global learning
in the future. Two steps are proposed to realize this promise.
13 | T e c h n o l o g y a n d L e a r n i n g

An initial extensive experiment is needed to demonstrate the effectiveness of
adaptive tutorial learning units. Several areas should be explored in this
experiment. We favor an experiment with very young children, at the beginning of
‘school’ learning, including reading and writing, mathematics, and science. Full
proposals for this activity are available. Another possibility, perhaps a
simultaneous experiment, would be an experiment covering several courses at
the introductory college level.
As this is an experiment, and so might fail to meet expectations, we would not
proceed further unless the experiment was successful.

If the experiment suggests further activity, we need a full plan as to how we
should proceed after the experiment. Proposed details are available.

Further units need to be developed to cover the range of learning activities
from birth to old age.

Units need to be moved to many languages and cultures if this system is
to support global learning.

The learning units must work in schools, and in areas where no schools,
or very poor schools, exist.

New inexpensive computers will be needed, particularly for the very poor
parts of the world. We must consider areas where no electricity is
available.

A new distribution system must be developed to reach everyone on earth.
Satellites seem to be a good possibility.

Financing should be sought for these developments.

New learning organizations for managing the new system will be needed
for the future.
These and other details are discussed fully in material available from Alfred Bork
Conclusion
So far technology has not led to the major improvements in learning that were expected.
But the use of adaptive tutorial learning with computers may lead globally to orders of
magnitude improvements in learning at all levels.
14 | T e c h n o l o g y a n d L e a r n i n g
References
Bloom, Benjamin S., The 2 Sigma Problem: The Search for Methods as Effective as
One-to-One Tutoring, Educational Research, July 1984.
Bork, Alfred, Bertrand Ibrahim, Alastair Milne, Rika Yoshii, The Irvine-Geneva Course
Development System, in R. Aiken (Ed.), Education and Society, Information Processing
2, Elsevier, Holland, 1992.
Bork, Alfred, Interview, EDUCOM Review, July/August 1999.
Bork, Alfred, Tutorial Learning for the New Century, Journal of Science Education and
Technology, March 2001, Volume 10, Number 1.
Bork, Alfred, Four Fictional Views of the Future of Learning, The Internet and Higher
Education, 3 (2000)
Bork, Alfred, What is Needed for Effective Learning on the Internet, Educational
Technology and Society, 4(3) 2001
Bork, Alfred, Sigrun Gunnarsdottir, Tutorial Distance Learning – Rebuilding our
Educational System, Kluwer Academic Publishers, New York, 2001. Chinese translation
available from Tsinghua University Press, Beijing, China, 2004
Bork, Alfred, Distance Learning, Today and Tomorrow. In Online Learning, editor Greg
Kearsley, Educational Technology Publications, Englewood Cliffs, NJ, 2005.
Bork, Alfred, Adaptive Lifelong Global Learning - Vision and Plan, 2006 Draft available,
Rischard, J.F., High Noon - Twenty Global Problems, Twenty Years to Solve Them,
Basic Books NY, 2002
15 | T e c h n o l o g y a n d L e a r n i n g