CC467
Biologically inspired robotics
Introduction to Lego Mindstorms
Owen Holland
Computer Science
University of Essex
Why Lego is different
In most robotics work, you are given a task, and a
robot that must perform it, and you program the
robot.
Easy.
Why Lego is different
In the old days, there were hardly any robots.
If you wanted a task to be done by a robot, you first
had to build your robot, and then you would be able
to program it. Not easy…
- mechanical design and manufacture
- electronics design and manufacture
- system integration
- and then you could program the robot, often
in assembly language
Why Lego is different
Now, with Lego, most of the hard bits disappear:
you just stick Lego bits together to make the robot
you want
- NO mechanical design and manufacture
- NO electronics design and manufacture
- NO system integration
- and you can program the robot in almost
any language you want: C, Java, Lisp, Ada,
Prolog, Logo, and even assembler.
- easy.
Isn’t Lego just a toy?
Let’s look at the history of Lego robotics.
It goes back a long way…
At MIT in the 1980s, the Mechanical Engineering
Department started a course called 2.70:
“Introduction to Design”.
Students used scrap parts to build joystickcontrolled machines that competed against each
other at the end of the course.
The birth of 6.270
In 1987, an Electrical Engineering and Computer
Science (Course 6) student wanted a similar course
in his department, and invented the 6.270
competition:
Students programmed simulated autonomous
robots that competed against one another at the end
of the course.
LEGO/Logo
At about the same time, a group of people at MIT
Media Lab were sponsored by Lego to develop a
system to enable children to build robots using
Lego, and to control them using Logo, a simple
programming language fashionable at the time.
One of these people, Fred Martin, began helping
with 6.270, and developed some Lego-based kit to
help.
Back to Fred Martin
One of the outputs of Fred’s Lego sponsored project
was the MIT Programmable Brick
Note the 6 sensor inputs
and 4 motor outputs
And finally…
An in-house Lego design team, obviously inspired
by the MIT work, developed the Lego RCX brick.
Note the 3 sensor
inputs and the 3
motor outputs
Lego Mindstorms
Released in 1998 in various kits.
Little changed since.
Aimed at children.
All kits contain:
- an RCX brick
- 2-way IR communication between RCX and PC
- Lego and Lego Technic parts
- sensors and motors
Lego sensors
Lego light sensor
Lego touch sensor
Lego temperature sensor
Lego rotation sensor
The Lego light sensor
This measures the intensity of the light coming in
through the lens in the front of the sensor.
It’s quite directional.
The light coming in through the lens can come from
two sources:
(1) The environment – we call it ‘ambient light’.
(2) Light coming from the red LED on the front of
the sensor that has been reflected off objects in
the environment. The amount reflected depends
on the colour of the object.
Lego Mindstorms: the Good
- Interfaces with all Lego and Technics parts
- Practically bomb-proof
- 4 types of sensors available (touch, light, rotation,
temperature)
- Good battery life
- Not very expensive for whole kits
- Good support for schools
Lego Mindstorms: the Bad
- Motor control is awful
- Limited sensor inputs and motor outputs
- Individual parts are very expensive
Lego Mindstorms: the Ugly
- Dumbed down programming environments
Visual
programming
(RCX code)
Is there any hope?
The RCX brick contains a very capable and fully
featured microcontroller, the Hitachi H8/3292,
running at the maximum clock speed of 16MHz.
It has 16kB of on-chip ROM (Read Only Memory)
containing routines for communication with the IR
channel, sensor and motor interfacing, LCD
drivers, some test programs, and so on. It imposes
some limitations, but nothing too serious, which is
just as well because nothing can be done about it.
Is there any hope?
The RCX brick also contains 32kB of external
RAM (Random Access Memory). This holds
information as long as the battery is connected.
It is used for two things:
(1) The ‘firmware’ that forms a kind of operating
system. It is uploaded to the RAM via the IR
link. The firmware supplied by Lego has some
limitations (no floating point, only 32 variables).
(2) The user programs, compiled on the PC and
uploaded via the IR link. Only 6kB is available
for these!
Yes, there is hope.
Here is what happens:
- you produce a program on your PC
- the program is transformed into low-level
instructions (bytecodes)
- the bytecodes are transmitted via the IR link to
the RCX RAM
- the firmware interprets the bytecode and
converts it into machine code using the ROM
- the processor executes the machine code
Yes, there is hope.
If we only knew what the bytecodes actually caused
the microcontroller to do, we might be able write
a program to generate more interesting
sequences of bytecodes than we can using the
Lego software.
In 1998, Kekoa Proudfoot was a student at
Stanford University…
Respect to Kekoa Proudfoot
He reverse engineered the RCX, discovering and
cataloguing all the bytecodes for the interpreter,
and much else besides.
The RCX
board –
top view
Bottom view
Enter Dave Baum
Following on from Kekoa Proudfoot’s work, Dave
Baum was able to write NQC – Not Quite C.
This allows you to write a program for the RCX in
a language very similar to C, and to upload it to the
RCX, where you can run it as normal.
NQC is text-based. Now other people have written
pretty front ends so it’s just like normal
programming in a typical IDE – you’ll be using
John Binder’s Bricx Command Center.
More hope…
NQC uses the original Lego firmware. If we used
Kekoa Proudfoot’s discoveries, could we devise new
firmware that would enable us to do even more?
YES.
In 1999 Marcus Noga began the LegOS project,
designed to replace the Lego firmware with a new
version allowing more or less direct control of the
Hitachi processor. Smaller firmware, bigger
programs (in C), and much faster execution – but
for experts only.
And now…
The newest firmware replacement is part of the
LeJos system. It permits a Java Virtual machine
(JVM) to run on the RCX. You can now program
your robots in Java (in fact a subset of Java).
Why is this nice? Because it gives you access to lots
of the neat stuff associated with Java. The LeJos
package comes with classes for handling all the
Lego hardware – you don’t have to build
everything from scratch, which is just as well
because Java is really too high level for robotics.
Hardware developments
From the early days, enthusiasts began designing
and building their own sensors. Some of these are
excellent. Some are on sale. Some third-party
sensors are even supported by Lego.
Here’s a list of
home-brewed
sensors from just
one site…
Hardware developments
From the early days, enthusiasts began designing
and building their own sensors. Some of these are
excellent. Some are on sale. Some third-party
sensors are even supported by Lego.
If you wanted to build a complex Lego robot now,
you could probably find designs for the sensors you
need on the Web. Some will be accompanied by the
Java classes for dealing with them.
The range of hardware and software now available
for Lego has transformed it from a toy into a
serious educational and investigative tool.
So who uses it?
Dozens of universities in many countries, including
top engineering colleges in the USA, Switzerland,
Denmark, Germany, and Sweden.
What do they use it for?
Teaching programming
Teaching control
Teaching robotics
Individual projects
Student competitions etc. etc.
How can I find out more?
Books:
Core Lego Mindstorms Programming by Brian
Bagnold
Good intro to Mindstorms and to LeJos
Web:
Too many sites to list. Just Google any mixture of
mindstorms, lego, robot, sensors, java, nqc, rcx.
Lots of good stuff, and most sites have links to lots
of others.
A simple robot
Some of the simplest robots are called Braitenberg
machines.
In two weeks you’re going to build and program the
one known as Vehicle 2.
We have a problem
In the space of 2 hours, working in pairs, and
starting from scratch, you have to build, program,
and run such a vehicle.
It’s not realistic.
So we’ll give you some strong guidance, in the form
of the instructions to build a suitable vehicle…
…and in the form of a suitable program
tailored to the vehicle
Back to basics
A suitable vehicle needs the following:
- a left motor (LM)
- a right motor (RM)
- a left sensor (LS)
- a right sensor (LR)
Back to basics
A suitable program needs the following:
- a way of driving each motor as a function of
the reading on the sensor on the opposite side
Back to basics
And the environment needs to ensure that the
change in sensory inputs produced by the
movement of the vehicle leads to some interesting
and obviously useful behaviour…
…with a simple performance measure so that
we can have a competition
Constraints
All vehicles will have to be run in exactly the same
environment
The only suitable sensors are light sensors
Consider the termite
How termites work
Termites (and many other trail-following insects)
follow trails by sensing concentrations of special
chemicals (pheromones) using their antennae.
How termites work
They use a technique called osmotropotaxis.
Roughly speaking, their left legs go at a speed
controlled by the strength of the sensory input to
their right antenna, and their right legs go at a
speed controlled by the strength of the input to
their left antenna…
In other words, they operate just like little
Braitenberg vehicles, and this enables them to
follow trails.
Could we make visible trails on the ground,
and sense them with light sensors?
Yes…there are some trails on the floor of the
arena…
But surely…
…if the trail is either black or white, won’t the
sensors only ever give maximum or minimum
readings?
No, because the sensor takes in reflected light from
an area, not a point.
If the area is wholly inside the black region, you’ll
get the minimum reading (with some noise)
And if it’s wholly inside the white region, you’ll get
the maximum reading (with some noise)
But surely…
…if the trail is either black or white, won’t the
sensors only ever give maximum or minimum
readings?
No, because the sensor takes in reflected light from
an area, not a point.
If the area contains both black and white regions –
if the sensor is somewhere on the borderline – then
the reading will be somewhere in between the
minimum and maximum (again, with some noise)
A basic strategy
If we space the sensors so that they are separated
by just less than the width of the trail, then when
both sensors read the minimum, we know we must
be on the trail…
…and so moving forwards is a good idea
A basic strategy
If the sensor on the left reads the maximum, and
the sensor on the right reads the minimum, we
know we’re moving off the trail to the left…
…so turning right is a good idea
A basic strategy
If the sensor on the right reads the maximum, and
the sensor on the left reads the minimum, we know
we’re moving off the trail to the right…
…so turning left is a good idea
A basic strategy
And if both sensors read the maximum, you know
you’ve completely lost it
Of course, you could devise some cunning way to
get back on the trail (termites do)…
But…
What if one or both sensors give a reading
somewhere in between?
The quick and dirty solution: keep on doing
whatever you were doing – it won’t last
The elegant solution: you can produce one if you
want, but it won’t make any difference…
The performance measure
We can break this into two stages:
(1) How far down the trail you get
(2) If you make it to the end, how long it took
First, catch your robot
You can make whatever you like, but since time is
so short I recommend that you use a version of the
Trusty robot described in Jonathan Knudsen’s
“The Unofficial Guide to Lego Mindstorms Robots”
Instruction sheets and pictures will be in the lab.
And finally…the program
You’ll be programming your robot in NQC (Not
Quite C). You’ll use the BricxCC (Bricx Control
Centre) as your development environment.
When you first fire up BricxCC (by double clicking
on the BricxCC icon) it looks for the RCX brick. It
will find it, because you will have switched it on
(using the red On-Off button) and placed it with its
infra-red windows facing the IR tower and a few
centimetres away from it.
Click on RCX2, then OK
You’ll get this. You should also get another window
If you don’t, press F9 once or twice until you do
File, New gets you a programming window
Copy and paste the text file ‘trusty’ from your email
And save it locally as trusty (a .nqc file)
Click on Compile to compile, and on Download to put it into the RCX.
If your program won’t compile…
You will get helpful error messages below the
program
If you can’t sort out the errors by yourself, call
Renzo, the GTA
The trusty.nqc program
It’s straight from the Knudsen book – a photocopy
of the relevant pages will be in the lab.
Since I can’t teach you NQC in 2.4 minutes, we’ll
skip through some basics, and then go through the
program.
NQC/RCX basics: Sensors
Sensor connections 1, 2, and 3 are known as
SENSOR_1, SENSOR_2, and SENSOR_3
You have to tell each sensor connection what sort of
sensor is plugged into it. For the light sensor, it’s
just a matter of:
SetSensor (SENSOR_2, SENSOR_LIGHT)
NQC/RCX basics: Motors
Motor connections A, B, and C are known as
OUT_A, OUT_B, and OUT_C
You set the direction of a motor by
OnFwd (OUT_A) or OnRev(OUT_B)
You set the power of a motor by
SetPower (OUT_A, x) where x runs from 0 to 7
You turn a motor off by
Off(OUT_B)
NQC/RCX basics: Tasks
NQC can run several tasks at the same time.
(Unfortunately you have to be careful to ensure that
tasks do not conflict, especially over resources.)
The format of a task is:
task taskname( ) {task instructions etc go here}
Every program must contain a task called main:
task ( ) main {stuff}
NQC/RCX basics: Subroutines
There are several ways of avoiding having to write
out long strings of repetitive chunks of code. One is
to use subroutines.
The format of a subroutine is:
sub subroutinename ( ) {code goes here}
And you call it in the main body of your code by
subroutinename ( );
(Semicolons are used as separators for expressions
etc.)
NQC/RCX basics: Control structures
while (true) {stuff}
will execute stuff forever.
if (condition) do_something_simple will
do_something_simple if the condition is true
if (condition) do_something_simple
else do_something_else will do_something_else if
the condition is not true
NQC/RCX basics: Variables
All variables are integers. You have to declare them,
otherwise the compiler will moan.
int thing;
at the beginning of your program will let you use
the integer variable thing anywhere.
If you declare it within some curly brackets (as part
of an expression or subroutine etc.) it won’t work if
you try to use it outside the brackets (and the
compiler will complain).
NQC/RCX basics: Constants
You can give constants meaningful names instead of
numbers by defining them at the start of your
program:
#define MONKEY_SCORE 25
means you can write MONKEY_SCORE in your
code instead of just 25, and so you will be able to
distinguish it from all the other constants that
might be 25 – for example, SLAMMER_RECORD
trusty.nqc
int state;
#define BOTH_ON 3
#define LEFT_ON 1
#define RIGHT_ON 2
#define BOTH_OFF 0
#define INDETERMINATE 255
#define DARK2 35
#define LIGHT2 40
#define DARK3 40
#define LIGHT3 45
#define POWER 4
trusty.nqc
int state;
THIS IS THE DECLARATION OF THE VARIABLE WE
WILL USE TO REPRESENT ALL THE DIFFERENT
MEANINGFUL COMBINATIONS OF SENSOR STATES
trusty.nqc
#define BOTH_ON 3
#define LEFT_ON 1
#define RIGHT_ON 2
#define BOTH_OFF 0
#define INDETERMINATE 255
THESE ARE THE VALUES THE VARIABLE state CAN
TAKE.
BOTH_ON MEANS BOTH SENSORS ON THE TRACK,
ETC.
INDETERMINATE MEANS AT LEAST ONE SENSOR IS
ON THE BORDERLINE
trusty.nqc
#define DARK2 35
#define LIGHT2 40
#define DARK3 40
#define LIGHT3 45
THESE ARE THE SENSOR INPUT VALUES FOR
DECIDING WHETHER A SENSOR IS ON OR OFF THE
LINE, OR ON THE BORDERLINE
Sensor tip
If the sensor port is correctly configured, you can
read the value of a sensor directly from the LCD
display on the RCX.
Press VIEW. You will see a little ^ pointing to input
1. The number on the LCD will be the current input
of the sensor connected to input 1.
Press VIEW again. The ^ will move round one
place to input 2.
Keep on pressing and it will go 3, A, B, C, and then
out of VIEW mode.
trusty.nqc
#define POWER 4
THIS SETS THE POWER LEVEL THAT WILL BE USED
FOR MOTORS THAT ARE ON.
trusty.nqc
task main() {
initialize();
while (true) {
if (state == BOTH_ON)
OnFwd(OUT_A + OUT_C);
else if (state == LEFT_ON) {
Off(OUT_A);
OnFwd(OUT_C);
}
else if (state == RIGHT_ON) {
Off(OUT_C);
OnFwd(OUT_A);
}
}
}
trusty.nqc
task main() {
initialize();
while (true) {
CODE HERE
}
}
THIS STARTS THE MAIN TASK, INITIALIZES
EVERYTHING USING THE SUBROUTINE INITIALIZE,
AND EXECUTES THE CODE IN THE MIDDLE
trusty.nqc
if (state = = BOTH_ON)
OnFwd(OUT_A + OUT_C);
else if (state = = LEFT_ON) {
Off(OUT_A);
OnFwd(OUT_C);
}
else if (state = = RIGHT_ON) {
Off(OUT_C);
OnFwd(OUT_A);
THIS GOES FORWARD FOR BOTH_ON, TURNS LEFT
FOR LEFT_ON, AND TURNS RIGHT FOR RIGHT_ON
trusty.nqc
sub initialize() {
SetSensor (SENSOR_2, SENSOR_LIGHT);
SetSensor (SENSOR_3, SENSOR_LIGHT);
SetPower (OUT_A + OUT_C, POWER);
OnFwd(OUT_A + OUT_C);
start watcher;
}
THIS IS THE SUBROUTINE initialize. IT SETS
THE SENSOR INPUTS CORRECTLY, STARTS THE
ROBOT GOING FORWARD, AND STARTS THE TASK
watcher.
trusty.nqc
task watcher() {
while (true) {
if (SENSOR_2 < DARK2) {
if (SENSOR_3 < DARK3) state = BOTH_ON;
else if (SENSOR_3 > LIGHT3) state = LEFT_ON;
else state = INDETERMINATE;
}
else if (SENSOR_2 > LIGHT2) {
if (SENSOR_3 < DARK3) state = RIGHT_ON;
else if (SENSOR_3 > LIGHT3) state = BOTH_OFF;
else state = INDETERMINATE;
}
else state = INDETERMINATE;
}
}
trusty.nqc
task watcher() {
while (true) {
ONCE STARTED, WATCHER KEEPS ON RUNNING FOR
EVER
trusty.nqc
if (SENSOR_2 < DARK2) {
if (SENSOR_3 < DARK3) state = BOTH_ON;
else if (SENSOR_3 > LIGHT3) state = LEFT_ON;
else state = INDETERMINATE;
IF BOTH SENSORS ARE BELOW THEIR INDIVIDUAL
DARK THRESHOLD, state IS SET TO BOTH_ON.
IF THE SENSOR ON THE LEFT IS DARK, AND THE
SENSOR ON THE RIGHT IS ABOVE ITS LIGHT
THRESHOLD, state IS SET TO LEFT_ON.
IF THE SENSOR ON THE LEFT IS DARK, AND THE
SENSOR ON THE RIGHT IS BETWEEN ITS LIGHT AND
DARK THRESHOLD, state IS SET TO INDETERMINATE
trusty.nqc
else if (SENSOR_2 > LIGHT2) {
if (SENSOR_3 < DARK3) state = RIGHT_ON;
else if (SENSOR_3 > LIGHT3) state = BOTH_OFF;
else state = INDETERMINATE;
THIS SORTS OUT RIGHT_ON AND BOTH_OFF, AND ANOTHER
KIND OF INDETERMINATE
trusty.nqc
else state = INDETERMINATE;
AND THIS CATCHES THE CASE WHERE BOTH
SENSORS ARE BETWEEN THRESHOLDS
THAT’S ALL YOU NEED – OFF YOU GO…