The Scanning Electron Microscope as Sensor System
for Mobile Microrobots
Ferdinand Schmoeckel, Heinz Wörn, Matthias Kiefer
Institute for Process Control and Robotics
Universität Karlsruhe (TH), Kaiserstr. 12, 76128 Karlsruhe, Germany
Abstract
The presented mobile microrobots are employed inside the vacuum chamber of a scanning
electron microscope (SEM). Even for simple handling tasks more than one robot is required
very often due to the unfamiliar force ratios in the micro world. This paper describes how the
SEM is used as a position sensor system that is a presupposition of the automatic coordination of microrobots. For depth measurements a triangulation principle with the help of
the electron beam is used. First results and the required calibration methods are presented.
Introduction
The scanning electron microscope (SEM) is a very important tool in many areas, where very
small structures are to be investigated. If not the microstructure of large surfaces is of interest
but the samples themselves are smaller than a few millimetres, their handling is very difficult.
At least, if the samples must be manipulated while being in the SEM, e.g. during in-situ experiments or for the assembly of hybrid micro systems, small and flexible manipulating systems are required. The mobile microrobots being developed at Universität Karlsruhe (TH) are
such micro manipulating tools. They are some cube-centimetres sized, piezoelectrically
driven, and the employed slip-stick principle enables them to move large distances with some
centimetres per second with a positioning resolution of about 20 nanometres [1], [2].
Two robot system and teleoperation
If microscopic objects are to be handled, the so-called scaling effects must be faced very often. The most frequent effect caused by these unfamiliar force ratios is that a grasped object
remains sticking at one jaw of the microgripper when trying to drop it. Furthermore, the particles observed with the help of the electron beam can be charged electrically – unless a socalled “Environmental SEM” working at elevated pressure is used. This makes manipulations
unpredictable if no suitable actions are taken to cope with these problems.
As proposed by Miyazaki [3], 1997, one possible approach to face this problem is to involve a
second robot which is equipped with a “helping hand” consisting of a simple needle-shaped
gripper tip. It can brush off the object, minimising the contact faces by the small dimensions
of the needle. As an example, the releasing of a grain of pollen with this technique is shown in
Fig. 1.
Fig. 1: Releasing a grain of pollen, SEM image Kammrath&Weiss GmbH, Dortmund, Germany
Such operations can be done by the two SEM-suitable robots shown in Fig. 2. Both robots
consist of a mobile platform with three degrees of freedom (DOF), which carries a manipulation unit with a microgripper. Supported by three piezo elements, the manipulation unit of
Miniman III consists of a sphere that provides two additional degrees of freedom for the microgripper (one DOF is redundant with the mobile platform) [1]. While Miniman III can grasp
a sample, the successor prototype Miniman IV is used as a helping hand. Its gripper is driven
in z-direction by a linear micro drive manufactured by Kammrath&Weiss GmbH, Dortmund.
It can clamp very small probes (see Fig. 1) that can easily be disposed when worn or dirty.
30 mm
Fig. 2: The microrobot prototypes Miniman III (left) and Miniman IV (right) being employed in the
SEM
In order to integrate two or more robots in the vacuum chamber of an SEM, the robots must
be as small as possible. Similar to Miniman III, the positioning unit of Miniman IV has three
piezo legs for performing a slip-stick movement. However, because of the different manipulation unit, the size of the platform could be reduced (∅ 50 mm). The main difficulty for the
miniaturisation is the connection to the control system. Since the prototypes realised by now
do not carry any onboard electronics, up to 50 wires are required. Therefore, for Miniman IV
long flexible printed circuit boards had been designed replacing bundles of thin wires that are
very fragile especially at the plugs.
For the – presently alternating and open-loop controlled – teleoperation of the robots, a 6Dmouse acts as an intuitive user interface. Together with an automatic coarse positioning into
the field of view of the SEM using a global CCD camera [2], this system is already a helpful
tool for many tasks in scanning electron microscopy.
The SEM as a sensor system
Closed-loop control and automation of the microrobots requires a positioning sensor system.
The employed slip-stick principle provides a very high resolution (ca. 20 nm) while the robot
design is very simple. However, it does not allow any internal position sensors. To use the
SEM itself as a high resolution non-contact positioning sensor is promising. In this case, the
robots’ gripper tips must be recognised and tracked by image processing of the SEM image.
The SEM image can be scanned by the electron beam with a resolution of up to 4096 x 4096
pixels. If the size of this scanning area is set to 2 x 2 mm², the resolution is about 0.5 µm. In
order to be able to acquire and process the SEM image in real-time, the resolution or the field
of view or both must be reduced. The size of this so-called region of interest (ROI) is
256 x 256 pixels for instance.
Scanning area of
the electron beam
Mic
rog
rip
per
ROI
(256² Pixel)
SEM image
(ROI)
Full resolution
(4096² Pixels total)
Fig. 3: Scanning area of the electron beam and zoom principle of the “Region of Interest”
Fig. 3 illustrates the selection of the ROI parameters resulting in a compromise of resolution
and field of view. For the ROI in this example, each third pixel is scanned building the SEM
image. The resolution is reduced while a larger field of view is obtained at a constant number
of pixels.
As the global positioning system provides the position of the microgripper with an accuracy
of about 0.5 mm, the initial ROI can be set ensuring that the gripper tips are visible in this
ROI. The image processing system being currently developed can use the coarse information
about position and orientation of the gripper as well. For the image recognition itself, special
features will be attached to the gripper considering the characteristics of the secondary electron image. If a higher accuracy is required, the field of view can be reduced after successful
recognition of the gripper increasing the resolution.
Electron beam triangulation
As the microscope image provides only two-dimensional position information, the control of
the robots in three dimensions requires an additional sensor for depth measurements. However, the installation of a second electron gun for a second, lateral SEM image like e.g. in [4]
is very expensive. The same applies for stereo SEMs. Moreover, the so-called correspondence
problem to link image details in a stereo pair is not solved generally, yet, and in particular
cases very time consuming.
A usual and fast sensor principle for depth measurement is the laser triangulation, which is
also used for microrobots under the light microscope [5]. Inside the SEM, the electron beam
can be used instead of a laser. The digitally controlled positioning of the electron beam is very
fast and flexible. For this electron beam triangulation, a miniaturised light microscope is
mounted inside the vacuum chamber. It provides the image of the luminescent spot of the
electron beam. This principle is explained in Fig. 4. As the positions of the miniature micro-
scope and the electron beam are known, the height of the electron beam’s spot can be calculated from its image on the CCD chip.
Electron
beam
Spot
Miniature
microscope
Robot
Fig. 4: Electron beam triangulation
For a sufficiently bright spot, the surface to be measured must be coated with cathodoluminescent material (scintillator). To allow the computing of the total robot configuration, scintillator material was attached to the gripper tips in form of a Z-pattern. If the electron beam
scans a line across this pattern, up to six bright spots can be seen by the miniature microscope
(Fig. 5). The co-ordinates of these spots in the microscope image are determined by a simple
image processing software. By triangulating, the height of these points is calculated. Their
distances from each other provide information about the position of the Z patterns and determine almost the total configuration of the gripper including the gripper opening. In order to
find out the remaining degree of freedom, which is the rotation around the axis determined by
the points, different techniques can be considered. If the data from the global positioning system is used, the height of the gripper tips can be obtained with a higher accuracy than that of
the global positioning because of the lever ratios. For instance, if the distance from the luminescent spots to the gripper tips is 2 mm, the total uncertainty is about 50 µm. Alternatively, a
second line can be scanned by the electron beam. Because of the redundancy of the measurements, the accuracy of some parameters can be further enhanced. However, the position of the
Z-patterns must be already know with a certain accuracy, to maximise the distance between
the two lines. The third possibility uses the two-dimensional information from the SEM image
recognition system. A gripper tip lies on the surface of a sphere, which is determined by its
centre and the radius R that are given by the triangulation (see Fig. 5). Hence, the intersection
of this sphere and the line of sight to the SEM image of the tip is calculated to obtain the tip’s
height. The higher the Z-patter is mounted above the tip, the better is the accuracy of this calculation.
Scanning area of the electron beam
Line scan
Miniature
microscope
Mic
rog
ripp
er
SEM image
(ROI)
R
Luminescent spots for
depth measurement
Fig. 5: Sensor principle and section of the microscope image (right)
For the first tests of this sensor principle, silicon chips with 2 mm microstructured grooves
had been glued to the gripper. These grooves had been filled with the scintillator powder P47.
First, the position of the grooves relative to the gripper must be measured as accurately as
possible, e.g. using the SEM. This measurement can be avoided if the grooves are integrated
in a microstructured gripper. In this case, further smaller Z-patters could be integrated closer
to the gripper tips. In connection with a zoom objective for the miniature microscope, different resolutions and respective working ranges could then be selected.
Calibration
All required parameters must be determined by calibrating the SEM image and the miniature
microscope. These are the 11 parameters of Tsai’s camera model [6] in case of the miniature
microscope. They can be obtained by using a small grid shown in Fig. 6. To visualise the successful calibration, the co-ordinate system of the SEM image is overlaid in this camera image.
As the grid is coated with scintillator powder, the matching of the line drawn in the camera
image and the resulting real luminescent line can be additionally observed.
Fig. 6: Camera image of the calibration grid with aligned line scan
The SEM image is calibrated as usual with the help of a microstructured calibration grid. Afterwards, for each focussed free working distance, the size of a pixel is known. Additionally,
the image rotation as a function of the focus must be known. It results form the principle of
electron optics using magnetic lenses and is not automatically compensated by the employed
SEM.
Virtual pivot point
H
Object
A
B
h
Object plane of
the SEM image
Fig. 7: Calibration of the virtual pivot point
Since the SEM image is formed as a central projection, the electron beam seems to come from
the so-called virtual pivot point. To be able to determine the exact position of the electron
beam, this point must be known. Its position is measured using the method shown in Fig. 7.
An object – preferably a very thin wire – is mounted in the centre of the SEM image and in
the height h above the present object plane. If it is moved horizontally by the length B, the
SEM image shows a shift of the length A. This length is measured in the SEM image. Using
the intercept theorems, the height of the pivot point can be calculated.
Results and outlook
Presently, the accuracy of the height measurement is limited to ca. 30 µm, mainly by the employed miniature microscope. Using a higher resolving camera and a smaller field of view, an
accuracy of a few microns will be reached. Beyond this, the depth of focus of the light microscopy becomes the limiting factor. In comparison to the possible resolution of an SEM this
still seems to be low. However, it is sufficient for many tasks in the micro- and millimetre
range, in particular if a contact at the micro gripper’s bottom side is detected during handling
by a future force sensor.
Acknowledgements
This research work has been performed at the Institute for Process Control and Robotics
(Head - Prof. H. Wörn), Computer Science Department, Universität Karlsruhe (TH). The research work is being supported by the European Union (ESPRIT Project “MINIMAN”, Grant
No. 33915).
Reference
[1] Schmoeckel, F.; Fatikow, S.:"Smart flexible microrobots for SEM applications”, Journal
of Intelligent Material Systems and Structures, Vol. 11, No. 3, 2000, pp. 191-198
[2] Fatikow, S.; Buerkle, A.; Seyfried, J.: “Automatic Control System of a Microrobot-Based
Microassembly Station Using Computer Vision”, SPIE's International Symposium on Intelligent Systems & Advanced Manufacturing, Conference on Microrobotics and Microassembly, SPIE, pp. 11-22, Boston, USA, 19-22 Sept. 1999
[3] Miyazaki, H. et al.: “Adhesive forces acting on micro objects in manipulation under
SEM”; Microrobotics and Microsystem Fabrication, SPIE 3202, Pittsburgh, 1997, pp.
197-208
[4] Nakao, M.; Hatamura, Y.; Sato, T.: „Tabletop factory to fabricate 3D microstructures:
nano manufacturing world“, Proc. of the SPIE's Int. Symp. on Intelligent Systems & Advanced Manufacturing, Boston, MA, 1996, Vol. 2906: Microrobotics: Components and
Applications, pp. 58–65
[5] Buerkle, A. and Fatikow, S.: “Laser measuring system for a flexible microrobot-based
micromanipulation station”, IEEE/RSJ Int. Conference on Intelligent Robots and Systems
(IROS), Takamatsu, Japan, 2000, pp. 799-804
[6] Tsai, R.Y.: “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine
Vision Metrology Using Off-the-Shelf TV Cameras and Lenses”; IEEE Journal of Robotics and Automation, No 4, 1987, pp. 323-344