Introduction to Smart Systems
Introduction to Mindstorms
Educational Robotics Equipment
1
Mindstorms Introduction – Overview
Mindstorms NXT is an educational robotics system which enables a vast
range of experimental embedded systems and robotics work.
The physical devices are quite easy to construct.
If Java is used; the programming can also be quite easy.
The processor is powerful enough to support the most ambitious control logic.
leJOS NXJ is a firmware replacement for Mindstorms NXT. It includes a Java
virtual machine, so allows Mindstorms robots to be programmed using Java.
(see later)
Downloads and info available at: http://lejos.sourceforge.net/
Java Class / method information for Mindstorms sensors:
http://lejos.sourceforge.net/p_technologies/nxt/nxj/api/lejos/nxt/packagesummary.html
LeJOS tutorial available at:
http://lejos.sourceforge.net/nxt/nxj/tutorial/index.htm
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
2
The Mindstorms computer unit - the “NXT Brick”
The NXT brick is the computer unit that runs a program to control the ‘robot’.
The NXT brick is designed to actually be part of the ‘robot’ and can have other
structural components attached to it.
Input sensors and output actuators are attached to the NXT via ‘RJ12’ cables:
• Up to four input sensors can be attached inc: Touch / Light / Sound / Distance
• Up to three output actuators can be attached inc: Motors / Lights
Direct I/O is also provided on the NXT:
• A 100 x 64 pixel monochrome LCD display.
• Four input buttons (can be programmed to have specific functions).
• A speaker can play sound files (typically one word or short sound clip).
• A USB 2.0 port for connection to the programming host (a PC).
• Bluetooth wireless connectivity:
• Allows loading programs to the NXT wirelessly (instead of USB),
• Can be used to control ‘robots’ remotely (through phone / PDA etc.),
• Can be used to provide NXT-NXT communication.
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Simplified block diagram of the Mindstorms platform
User application program – Java
High-level I/O logic
e.g. set motor speed and direction
LeJOS NXJ (JVM, O/S firmware)
MicroProcessor
I/O control interface (I2C / TWI)
Sensors
(inputs)
MicroController Low-level I/O control
e.g. setting specific port bits, timing
events, handling interrupts
Output
devices
4
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
NXT Brick - Technical specification
Main microprocessor (runs the application program):
32-bit AT91SAM7S256
256 KB flash memory (for the program),
64 KB RAM (for data, variables etc.)
Peripheral control processor (controls the I/O devices):
8-bit ATmega48 microcontroller @ 4 MHz
(4 KB flash memory,
512 Bytes RAM)
This is not accessible to the programmer – (i.e. you).
It is used as a helper for the main processor, (it simplifies the control
operations and reduces the complexity of the interface electronics needed).
For further technical information see:
http://mindstorms.lego.com/Overview/The_NXT.aspx
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Main processor - AT91SAM7S256
32-bit ARM7TDMI RISC processor.
256K bytes of embedded high-speed Flash (for program storage)
64K bytes of SRAM (for data, variables).
Peripherals include:
USB 2.0 Full Speed Device Port,
USARTs (serial programmable interfaces),
Serial Peripheral Interface Bus (SPI) 4-wire synchronous serial data link,
Two-Wire Interface (TWI) – serial interface,
8-channel 10-bit ADC.
The maximum clock frequency is 55MHz.
Typical core supply is 1.8V (low voltage, good for battery-powered deployment).
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Secondary processor - ATmega48
ATmega48
Architecture
Works as a peripheral controller
to handle the I/O interfacing on
the Mindstorms NXT smart brick.
4K memory,
8-bit processor,
8 ADC channels.
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
8
Mindstorms NXT Brick – Output Ports
(Hardware implementation using Atmel ATmega48)
9
Mindstorms NXT Brick – Input Ports
(Hardware implementation using Atmel ATmega48)
NXT hardware development manual URL:
http://www.legolab.daimi.au.dk/DigitalControl.dir/LEGO%20MINDSTORMS%20NXT%20Hardware%20Developer%20Ki
10
t/LEGO%20MINDSTORMS%20NXT%20Hardware%20Developer%20Kit.pdf
Mindstorms NXT Brick – Sensor Ports (Input)
The NXT has four sensor ports.
Input sensors can be connected to any ports, in any combination.
Need to identify which type of sensor is connected to each port.
This is done in the constructor, when creating a sensor object, e.g.
static UltrasonicSensor us = new UltrasonicSensor(SensorPort.S1);
static SoundSensor sound = new SoundSensor(SensorPort.S2);
static TouchSensor touch1 = new TouchSensor(SensorPort.S3);
static TouchSensor touch2 = new TouchSensor(SensorPort.S4);
LCD
Buttons
1
2
3
Sensor (input) ports
4
11
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Mindstorms NXT Brick – Motor Ports (Output)
The NXT has three output ports.
Motors can be connected to these ports.
Need to identify which motor is connected to each port.
This is done by having three separate motor objects (A, B and C), e.g. :
Buttons
Motor.A.stop();
Motor.B.rotate(10);
Motor.C.setSpeed(100);
(see later for details of these methods)
LCD
Motor (output) ports
USB
C
B
A
12
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Mindstorms NXT Brick – Battery
The NXT has a rechargeable battery pack. This must be charged up with a
mains adapter (usually these will be provided already charged).
The rechargeable battery clips onto the bottom of the NTX.
(Hint: Think about access to the battery, especially the charger port, when
building robots – it is time-consuming if you have to dismantle to gain access).
Battery
Pack
Battery
Pack
Press release catch
Charger
port
NXT (face down)
Power on
Battery
charging
Embedded Systems Programming II
13
Richard Anthony, Computer Science, The University of Greenwich
LeJOS NXJ overview
leJOS NXJ is a tiny Java Virtual Machine ported to the LEGO Mindstorms
NXT brick.
leJOS NXJ provides:
• Object oriented language (Java)
• Preemptive threads (tasks)
• Multi-dimensional Arrays,
• Recursion
• Synchronization
• Exceptions
• Java types including float, long, and String
• Most of the java.lang, java.util and java.io classes
• A Well-documented Robotics API
and thus enables powerful robotics control applications to be developed
easily and quickly.
Full Class Hierarchy at
http://lejos.sourceforge.net/nxt/pc/api/overview-tree.html
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
14
Mindstorms sensors and output devices – Overview
Up to four input sensors can be attached inc:
Touch (digital), Light (analogue), Sound (analogue), Distance (analogue)
‘Digital’ means a true or false, on / off behaviour.
e.g. A switch is either on, or off – there are a finite number of possible states
(often 2, but can be more than 2).
‘Analogue’ means a continuously variable range of values.
E.g. the temperature in a room can be 20.10023 … degrees.
We can never completely accurately state an analogue value as a number.
With the mindstorms sensors, the analogue values are converted to digital
values so you get a numerical output. For example, distance is converted to a
value between 0 and 255. This is a digital representation of the analogue
value, it is less accurate, but in this form it can be used in a computer
program.
Up to three output actuators can be attached inc:
Motors, Lights
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
15
Mindstorms sensors and output devices – Types
Input sensors include:
• Touch / push switch
• Light intensity / Colour sensor
• Sound intensity
• Distance from an object
• 4-button keypad on Brick
Output devices include:
• Motors
• Lights
• LCD display on Brick
• Sound output (sounds, tones, words) on Brick
• Illumination light on Light intensity sensor
Communication:
• Bluetooth
Feedback mechanism:
• Rotation sensor on Servo motor
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
16
Mindstorms Sensors – Servo Motor
Speed, Rotation angle and Rotation direction can be controlled.
Various gears can be used if necessary to change speed and torque ratio.
Built-in rotation sensor:
• Can be used to control the actual distance rotated.
• Can be used as an input sensor – for example as a dial that is turned.
For further technical information see:
http://mindstorms.lego.com/Overview/Interactive_Servo_Motors.aspx
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
17
Motors – Example interfaces and methods
Class Motor
public void forward()
Rotate forward.
public void backward()
Rotate backwards.
public void reverseDirection() Reverses direction of motor.
public void flt()
Motor ‘floats’, loses power, not same as stop.
public void stop()
Motor stops, and resists any further motion.
public void rotate(int angle)
Motor rotates through angle.
public void rotateTo(int limitAngle)
Motor rotates to limitAngle;
public void regulateSpeed(boolean yes)
Turns speed regulation on/off.
public void setSpeed(int speed) Sets speed (degrees/second); Up to 900.
public interface Tachometer (Abstraction for tachometer built into motors.)
int getTachoCount()
Read position / count.
void resetTachoCount()
Set count back to 0.
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
18
Mindstorms Sensors - Distance Sensor (Ultrasonic)
Sends out an ultrasonic sound wave and measures how long it takes to
bounce back.
Speed of sound is approximately 340 meters / second. Based on this known
value, the distance can be computed from the time delay.
The sensor’s electronic circuits, together with the internal methods of the
class, do this computation for you, so you can call a simple method to find out
the distance from the robot to an obstacle.
Sends sound pulse
Receives reflection a short time later
For further technical information see:
http://mindstorms.lego.com/Overview/Ultrasonic_Sensor.aspx
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
19
Distance Sensor – Example methods
public int ping()
Send a single ping.
public int continuous() Continuous ping mode.
Enables continuous ping and capture mode. The default operating mode.
public int getDistance() Return distance to an object (0 -254, 255 if no object)
public int off()
Turn off the sensor.
public int capture()
Capture mode: detect other ultrasonic sensors.
A program can evaluate when it is best to make a new measurement which will
not conflict with other ultrasonic sensors.
Could be used to enable robots to detect each other.
public int reset()
Restores to default state.
After, sensor operates in continuous mode.
Example:
static UltrasonicSensor us = new UltrasonicSensor(SensorPort.S1);
int iDistance;
iDistance = us.getDistance();
20
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Mindstorms Sensors - Touch Sensor
This sensor can be used to detect if a robot has crashed into an obstacle,
or it can be used as an input switch, for example to start or stop a robot.
For further technical information see:
http://mindstorms.lego.com/Overview/Touch_Sensor.aspx
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
21
Touch Sensor – Example method
public boolean isPressed()
Check if the sensor is pressed.
Returns true if sensor is pressed, false otherwise.
Example:
static TouchSensor touch = new TouchSensor(SensorPort.S2);
if(touch.isPressed()
{
StopAllMotors();
}
22
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Mindstorms Sensors - Sound Sensor
Sound sensor readings are displayed in percent
(of value range).
The sound sensor can measure sound pressure
levels up to 90 dB – about the level of a petrol
lawnmower (this would read at 100%).
Other examples:
• 4-5%
A quiet room
• 5-10%
Conversation some distance away
• 10-30%
Normal level conversation or music
• 30-100%
people shouting or loud music
For further technical information see:
http://mindstorms.lego.com/Overview/Sound_Sensor.aspx
23
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Sound Sensor – Example methods
public void setDBA(boolean dba)
Set DB or DBA mode.
dBA: ‘Adjusted decibels’, frequency sensitivity similar to the human ear.
dB: [unadjusted] decibels, all sounds measured with equal sensitivity.
May detect sounds that are outside the human hearing frequency range.
Parameters: dba - true to set DBA mode, false for DB mode.
public int readValue()
Read the current sensor value.
Returns value as a percentage of maximum sound level.
Example:
private static SoundSensor sound = new SoundSensor(SensorPort.S4);
int iSoundLevel;
sound.setDBA(true);
iSoundLevel = sound.readValue();
24
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
Mindstorms Sensors - Light Sensor
• An Analogue input device.
• Provides numerical output; higher light intensity ► higher number.
• ‘Sees’ colours as ‘Grey Scale’ values (example below).
Example uses:
• Detect light / dark environments,
• Detect objects based on their colour,
• Follow a line.
Colours as seen by the eye
Colour equivalents in ‘grey scale’
(i.e. as the sensor sees them)
For further technical information see:
http://mindstorms.lego.com/Overview/Light_Sensor.aspx
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
25
Light Sensor – Example methods
public void setFloodlight(boolean floodlight)
floodlight - true to set floodit mode, false for ambient light
public int readValue() Read the current sensor value. Returns value as a
percentage of difference between the low and high calibration values.
public int readNormalizedValue() Read normalized value. More accurate than
readValue(). Value range 0 to 1023. Typical 145 (dark) to 890 (sunlight).
public void calibrateLow()
Call this when the sensor is reading the low value - used by readValue
public void calibrateHigh()
Call this when the sensor is reading the high value - used by readValue
public void setLow(int low)
Set normalized value corresponding to readValue() = 0
public void setHigh(int high)
Set the normalized value corresponding to readValue() = 100;
Example:
private static LightSensor light = new LightSensor(SensorPort.S1, false);
light.setFloodlight(true);
iValue = light.readNormalizedValue();
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
26
NTX Buttons – Example methods
The four buttons are identified as: ENTER, LEFT, RIGHT, ESCAPE.
Methods include:
public final boolean isPressed()
Check if the button is pressed.
Returns true if button pressed, false otherwise.
public final void waitForPressAndRelease() throws Interrupted Exception
Wait until the button is released.
public static int waitForPress()
Wait for some button to be pressed
Returns the ID of the pressed button
Example:
if (Button.ENTER.isPressed())
{
StartRun();
}
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
27
NTX LCD (100 x 64 pixel monochrome) – Example methods
public static void setPixel(int rgbColor, int x, int y)
Set a specified pixel on the screen.
public static void drawString(String str, int x, int y)
Display a string on the LCD at specified x,y co-ordinate.
public static void drawInt(int i, int x, int y)
Display an int on the LCD at specified x,y co-ordinate.
public static void setDisplay(int[] buff)
Write graphics from a Java buffer to the display.
public static void clearDisplay()
public static void refresh() Update the display.
Example:
LCD.clearDisplay();
LCD.drawInt(iCount, 2, 3);
Embedded Systems Programming II
Richard Anthony, Computer Science, The University of Greenwich
28
Lab (optional for those who have spare time)
Mindstorms challenge – work in groups of about 3, to build a
remote controlled robot using BlueTooth communication (B/T)
There is sample code on the resources website for:
B/T on PC
B/T on Mindstorms brick
In addition there are instructions for using Netbeans for the Java
development of the code for the Mindstorms brick, and a short
video showing an example remote-controlled robot.
The Mindstorms task is not assessed, but very interesting. We
recommend that students who are making good progress with
the coursework do the Mindstorms challenge.
Others should concentrate on the coursework task !
29
Mindstorms NXT to PC
communication via Bluetooth
NXT Brick
Method
PC
Establish NXT Connection
Set up input and output streams both ends of the
connection
Send data to NXT via the streams (could equally be
from NXT to PC) then flush the stream
Continually check the streams for new data, then
flush the streams once the data has been received
Interpret received data to perform desired action (e.g.
when ‘u’ character received, move robot forward)
Close the streams then the connection when done
Important LeJOS method signatures for Bluetooth 1
NXT Only
Class Bluetooth
• public static NXTConnection
waitForConnection(int milli_timeout, int IO_mode)
Class NXTConnection
• public DataInputStream openDataInputStream()
• public DataOutputStream openDataOutputStream()
• public void close()
Important LeJOS method signatures for Bluetooth 2
PC and NXT
Class DataOutputStream
• public void writeByte(int byte)
• public void flush()
//flush the stream ready for the next byte
• public void close()
Class DataInputStream
• public boolean available()
//check if there is any data
waiting in the input stream – normally checked within a loop
• public byte readByte()
• public void close()
Important LeJOS method signatures for Bluetooth 3
PC Only
Class NXTCommFactory
• public static NXTComm createNXTComm(int IO_mode)
// create the NXTComm object
Class NXTComm
• public NXTInfo[] search("NXT") // search for available bricks
• public void open(NXTInfo info, int IOMode) //establish Conn’
• public void close()
// close connection
Class NXTInfo
• public NXTInfo NXTInfo(int IO_mode, String Brick_name,
String NXTMAC)
// create NXTInfo class with the MAC
// address for your specific brick
Important notes
• It is possible to upload your code into the brick via Bluetooth
However in the lab use the USB cable because there will be
multiple bricks in the lab and your code could be uploaded on
your friends’ brick if Bluetooth is used for this part of the
process
• Whilst it is possible to search for all NXT devices to create a
connection, use the MAC address of your brick to ensure
connection to it and not to another brick in the lab, i.e. use:
new NXTInfo(NXTCommFactory.BLUETOOTH,"NXT",NXTMAC);
The Mindstorms Challenge
Build a robot from the Mindstorms kit and control its movement
from your PC keyboard via Bluetooth.
(There is some useful sample code on ‘Richards Resources
Website’)