iosB
vis IT
fraunhofer
[ Core competence ]
www.iosb.fraunhofer.de
ISSN 1616-8240
The competence triangle
Optronics
System Technologies
Image Exploitation
Editorial
Notes
CONTENT
Essay
Publisher
Prof. Dr.-Ing. Jürgen Beyerer
Prof. Dr. Maurus Tacke
Editor
Sibylle Wirth
Page 4
IOSB - The Competence Triangle
Maurus Tacke
Layout
Christine Spalek
English Edition
db Sprachenservice GbR
Dresden
Printing
E&B engelhardt und bauer
Karlsruhe
Themes
Page 6
Core Optronics
Reinhard Ebert
Editoral Adress
Fraunhofer Institute of Optronics, System
Technologies and Image Exploitation IOSB
Fraunhoferstr. 1
76131 Karlsruhe
Phone +49 721 6091-300
Fax +49 721 6091-413
[email protected]
Page 10
© Fraunhofer IOSB
Karlsruhe 2012
Page 14
Core System technologies
Olaf Sauer
Core Image exploitation
Markus Müller
Institute of the Fraunhofer-Gesellschaft,
Munich
13. Year
ISSN 1616-8240
Page 18
Contact Person at a glance
Photo acknowledgements
People / product Photos, Key visuals:
indigo Werbefotografie
People Photos, page 7 left, page 17 under:
Volker Steger
page 8: GP Tonello
page 11 left: Daimler AG
page 11 right: MEV
All other illustrations:
© Fraunhofer IOSB
Reproduction of any material
is subject to editorial authorization.
vis IT
Core competence
2
Editorial
Dear Friends of the IOSB,
The name of our institute reflects our three core competences. Two of which are
nearly self-explanatory:
We understand optronics as electro-optical systems and methods for signal and
image acquisition ranging from ultraviolet to thermal infrared.
Image exploitation includes conditioning, real-time processing, automatic and
interactive information acquisition from images and videos.
System technology may at first glance appear to be highly abstract; however,
it is a cross-sectional competence when it comes to finding holistic solutions to
challenging, complex problems.
System technology covers everything required for analysis, understanding, modeling,
development and the management of complex systems.
Prof. Dr.-Ing. Jürgen Beyerer
The class of problems, our institute is concerned with in its research and development
work, primarily requires the capability of dealing with highly complex subjects and
of integrating different scientific disciplines in the process of problem solving. With
challenges of this type, the potential for advancement and benefit mostly lies more in
the systematic combination of different components and sub-systems to form a
concerted whole than in any particular optimization of the components. It is typical
for problems with a certain level of complexity that in most cases optimal solutions
are not mere automatisms but human-machine systems, where organization into
a comprehensive process is essential in addition to hardware and software.
Our core competence system technology, in particular, includes the following:
software engineering, information and knowledge management, network and
web technologies, IT security, control and automation technology as well as
human-machine interaction.
However, profound knowledge in these fields is not sufficient. It is of vital importance
as well to have expert competencies in using the respective tools of the latest
state-of-the-art, if affordable contract research and development services are to
be provided.
In this issue of visIT, we would like to provide you with a detailed introduction to
the core competences of the IOSB.
Karlsruhe, August 2012
Prof. Dr.-Ing. Jürgen Beyerer
Prof. Dr. Maurus Tacke
3
vis IT
Core competence
Prof. Dr. Maurus Tacke
Essay
IOSB – THE COMPETENCE TRIANGLE
Figure 1: The key visuals stand for the 3 core competences Optronics, System Technology and Image
Exploitation.
ANALOGIES OFTEN
HELP TO MAKE THINGS
COMPREHENSIBLE,
they convey findings from one wellknown field to another less elaborated
one, where they help to structure new
issues and, in many cases, allow for
comprehensible accessibility. This is how
the IOSB competence triangle developed.
C O N TA C T
At our first retreat after forming IOSB
we tried to get a consistent view of
the institute. We wanted to define our
competences in order to set up a basis.
Numerous aspects were discussed and
many facts gathered, but putting it
all together proved to be a challenge.
Finally, someone (I can’t remember
who it was) suggested that the competences Optronics, Image Exploitation
and System Technology should be
arranged in a triangular form.
Prof. Dr. Maurus Tacke
Director
Fraunhofer IOSB
Phone +49 7243 992-130
[email protected]
www.iosb.fraunhofer.de
The triangular diagram promptly made
us think of the color triangle, and we
were pleased with this analogy. Just like
the three primary colors that enable any
color variety within the color triangle,
our three competences in the triangle
allow for solutions with regard to any
problems that may be managed by imaging sensors, intelligent exploitation and
system expertise. In a little while the final
vis IT
Core competence
4
diagram developed on the board and
the departments selected their positions
according to their shares of the three
core competences. Two years have gone
since then. The competence triangle has
evolved with the dynamic development
IOSB sees, and in its recent version is
shown in figure 2; it in fact helped us and
others to visualize the institute character.
COLORS STIMULATE
THE SENSES
Colors have always appealed to mankind and we have learned to mix them
and paint with them. It was soon discovered that any color could be mixed by
using the three primary colors only; and
already in the Middle Ages early versions
of the color triangle were known. An
example for a modern version is Maxwell’s color triangle, which we selected
to design our competence triangle.
Newton showed that it is possible to disperse light of any color into primary spectral colors and he also showed how to
produce any color out of these pure spectral colors. This is the physical side of the
game. Von Helmholtz further developed
the color triangle into a colored area with
its frame being formed by the pure spectral colors. This results in the fusion of
physical nature and sensation of colors.
We know that the human eye is
capable of recognizing three primary
colors and that these three sensory
stimuli generate our color perception.
No wonder that three well selected
colors also suffice for mixing the entire
palette. But it is quite impossible to
illustrate three independent values in a
two-dimensional way; the actual result
is a three-dimensional color space.
Through scientific findings Von Helmholtz succeeded in developing the illustration common to us today, based on
the thesis that colors are perceived alike,
irrespective of the light intensity.
Therefore, it is possible to illustrate the
whole color variety in a two-dimensional
diagram, but it would be impossible
for us to perceive colors in a threedimensional space.
Understanding color perception is
complicated because the color sensors
of our eyes do not analyze pure colors,
but rather mixtures, which in addition
do overlap. And natural colors are not
pure colors, but in part wild mixtures
of different primary colors. Colors are a
sensation and not a physical parameter.
Everybody easily sees and names colors,
but color perception is very complex.
SIGNIFICANCE
FOR THE IOSB
An analogy, however, is not a panacea.
Analogies are not suited for drawing
logical conclusions, but they prove
auxiliary in understanding highly complex
interrelations and relational levels.
Nonetheless, the analogy of color space
to our competence triangle goes further
than one would think at first sight.
Upon closer inspection, it discloses
more than merely the positions of the
departments and the miscibility of their
capabilities. People and their competences stand behind our triangle. And
these people are not like points corresponding to spectrally pure colors, but
they are people with their own profiles.
These profiles are the crucial factors.
Combining them properly to create
concerted solutions is not a technical
problem, but rather an art, which all
those involved contribute to. A further
development of these thoughts ends up
in multilayer matrix structures, which can
be used in an interdisciplinary respect
and reveal a considerable potential for
synergies.
As to IOSB: the primary colors are mixed;
the palette is now being developed and
is displaying a few new colors already.
We expect impressive results that might
be of use to you in a variety of ways.
Optronics
OPT: Optronics
SIG:
Signatorics
SPR:
Visual Inspection Systems
System technologies
NRG: Energy
WMS: Water and Mobile Systems
ILT:
Information Management and
Production Control
INA:
Fraunhofer Application Center
Industrial Automation
MRD: Systems for Measurement,
Control and Diagnosis
SKA: Secure Communication Architectures
Image exploitation
IAD:
Interactive Analysis and Diagnosis
IAS:
Interoperability and Assistance Systems
OBJ: Object Recognition
SZA: Scene Analysis
VID:
Video Exploitation Systems
VBV: Variable Image Acquisition and
Processing (Research Group)
Figure 2: The IOSB departments and their positions within the competence triangle.
5
vis IT
Core competence
OPTRONICS
CORE COMPETENCE OPTRONICS
Reflection-Signature
Vibrations-Signature
Photo
Frequency
Visualization of information acquired with a 1.5 µm laser Doppler vibrometer: the reflection image and
vibration pattern with the frequency-resolved vibration patterns. The different mode structures are highly
visible while the vibration amplitude is color-coded (white: intense vibration, black: no vibration).
ACTIVE AND PASSIVE SENSORS IN SYSTEMS
C O N TA C T
The examination, development and
assessment of optronic systems is one
of the key competences of IOSB. This
includes a number of complex tasks,
for example the use as well as the qualitative and quantitative performance
assessment of active and passive electrooptical systems for application in the
areas of optical metrology, reconnaissance, surveillance, security, inspection
and protection.
Dr. rer. nat. Reinhard Ebert
Optronics OPT
Fraunhofer IOSB
Phone +49 7243 992-140
[email protected]
www.iosb.fraunhofer.de
Reliable predictions on the useful range
of a system are a typical example of
the goals of performance assessment.
»Range« in this context is defined as
that distance up to which an observer
or sensor is capable of performing a
given task (like detection, recognition,
identification of vehicles, persons or
obstacles) with a given probability.
vis IT
Core competence
6
Additionally, performance assessment
can be important in the context of
industrial production, where it is
necessary to ensure that an inspection
system can keep pace with the speed
of production, especially when used
for the detection of faulty products.
As an example, packaging machines
in the pharmaceutical industry need to
ensure that no broken, contaminated
or mistaken pills are present in the
final product.
The performance limits of optronic
systems are determined experimentally
in our laboratories. Frequently, extensive field experiments or first tests at
the production facilities of customers
are done to ensure proper functioning
of the equipment.
Methods for experimental and modeled performance evaluation and
optimization of passive and active optronic systems and their transition
to industrial applications.
This image shows the construction of high-performance optical equipment for operating a gated
viewing system for detecting objects in motion.
The purity inspection system detects even the
slightest flaws in transparent materials.
MODELING
Right from the start of the design phase,
a reliable quantitative assessment or
performance prediction of electrooptical systems is highly desirable.
For this purpose, the IOSB develops a
multitude of modeling and simulation
tools that can be used to predict system
performance analytically as well as to
generate imagery with the anticipated
properties of the production system.
Even three-dimensional infrared scenarios can be rendered in a realistic
fashion.
tions are investigated at the IOSB using
both software and hardware based
solutions in optronic systems
(e.g. adaptive optics).
Significant distortions in imaging
through the atmosphere can be caused
by turbulent small-scale variations and
large-scale height-dependent variations
of the index of refraction; in extreme
cases, this leads to mirages. The causes
and the compensation of these distor-
PROTOTYPES
The long experience of the IOSB with
past and present optronic systems
together with extensive in-house databases of environmental data, taken
from decade-long measurements of
atmospheric transmission and turbulence strength, ensure the high fidelity
of the modeling and simulation tools.
Feasibility studies, done by IOSB’s
optronics, often serve as stepping stones
to prototypes. Material samples can be
used to decide if and how different
substances can be discriminated with
optronic sensors; based on these results,
solutions for product quality assurance
or automatic sorting are then developed.
An example for this development process is the sorting of heat-resistant
scarp glass: an IOSB study showed that
an illumination of glasses with a specific
combination of ultraviolet wavelengths
enabled the discrimination between
ordinary and heat-resistant glasses (as
used in oven windows or ceramic stove
tops), which was not readily apparent
with visual light.
For the development of visual inspection
systems, a multispectral workbench was
recently added to the existing image
exploitation center in order to tap into
new industrial applications. Now,
different substances and materials (e.g.
minerals, foods, plastics, glasses, etc.)
7
vis IT
Core competence
OPTRONICS
YOUR CONTACTS:
Dr. rer. nat. Reinhard Ebert
Optronics
+49 7243 992-140
Dr. rer. nat. Karin Stein
Signatorics
+49 7243 992-120
Severe turbulence is visible under the exhaust trail of an aircraft. Photo: GP Tonello
Prof. Dr.-Ing. Thomas Längle
Visual Inspection Systems
+49 721 6091-212
can be examined using hyperspectral
imaging sensors that cover the spectral
range from the ultraviolet to the near
infrared; this way, optimal discrimination criteria for inspection tasks can be
readily identified.
METHODS
Other groups continuously develop and
improve laser radar methods, relying on
both the incoherent and coherent properties of laser light. These methods
enable for example the contactless
measurement and analysis of surface
vibrations to sub-micrometer accuracy,
even when distances to the measurement object exceed several kilometers.
Other methods facilitate three-dimensional scanning of the environment even
under adverse conditions, for example
submerged sensors or limited visibility.
INTERFERENCES
Intensive natural light sources, laser
beams and strong lamps can interfere
with the proper functioning of sensitive
optronic sensors; the sources of inter-
vis IT
Core competence
8
ferences potentially cover the whole
spectrum from ultraviolet light to the
thermal infrared. The use of pulsed highpowered laser radiation increases the
risk of damage considerably. Lasers and
other measurement equipment enable
IOSB to assess the sensitivity of electrooptical equipment to these factors.
It is also possible to actively protect
sensors from damage. Current research
is investigating protection techniques
for all relevant spectral ranges and time
scales. A recent problem is the threat
by ultra-short laser pulses which cannot
be blocked by conventional, slowlyreacting filters.
REAL-TIME OPERATION
Real-time capability is important for
various applications in the competence
field of optronics. In the context of manmachine interaction, the identification
of obstacles, vehicles or persons has to
be fast enough to allow for adequate
and timely reactions by the operator;
for automatic operation, e.g. production processes, the time available for
47,9 °C
30,4 °C
This is a Stove with hot stovetops and heating of the wall to the back, the impression
of a comparatively cold hand and reflection of the heat radiation from a pot on the
right side of the image.
the whole chain from image recording
to action is often limited by external
factors. In order to reach the goal of
real-time operation, IOSB uses various
techniques such as parallelization, multicore architectures and sensor fusion.
SIGNATURES
Signature management examines the
appearance of persons, vehicles and
buildings in various spectral ranges. This
research includes the investigation of
methods to influence this appearance,
e.g. with the goal to diminish the
detectability of a target.
WARNING SENSORS
Optronic systems for threat detection
and warning are developed in addition
to the signature management measures
described above. The understanding
of background properties and the identification of possible sources of false
alarms supports the selection of suitable
methods, increasing the trust of the user
in the equipment.
The IOSB’s combination of research into
signature reduction on the one hand
and the development of methods for the
reliable detection of persons, vehicles
and buildings on the other hand, results
in a very productive co-operation that
guarantees the high quality of both
optronic warning sensors and countermeasures.
The entire optronic chain is covered
by the competence fields of optronics
at the IOSB: starting from the signal
properties of an object and its interaction with its immediate background,
through atmospheric distortions and
modifications, interferences up to sensor effects and data acquistion. In this
way, Optronics serves a starting point
for the subsequent steps of image
processing and system technology.
The difference between an object and its
background is the decisive factor for the
conspicuity of an object. Thus, a good
understanding of the characteristics of
the various possible backgrounds across
the spectral ranges is mandatory, including the influence of weather effects like
wind and sunlight. Specialized laboratories and measurement equipment is
employed by IOSB to arrive at definite
answers to these questions.
Left: vehicle without camouflage in the middle of the image; with adaptive camouflage on the right.
9
vis IT
Core competence
SYSTEM
TECHNOLOGIES
THE CORE COMPETENCE OF SYSTEM
For the IOSB, the core competence of
system technology refers to the ability
to analyze, model, design, optimize,
create, commission and successfully
operate complex systems over the long
term. This is high standard, particularly
for an institute whose primary business
constitutes its activities as a research
and development service provider.
System-technical competence is mainly
needed as a backbone in the fields
of automation, energy, water / the
environment and secure IT.
C O N TA C T
AUTOMATION
Dr.-Ing. Olaf Sauer
Business unit automation
Fraunhofer IOSB
Phone +49 721 6091-477
[email protected]
www.iosb.fraunhofer.de
Topics relating to automation have
always been important research and
development topics for the IOSB:
From measurement and control technology to embedded systems, complex
control and systems for manufacturing
operations management, the IOSB has
developed and provided pioneering
contributions for various industrial
applications. The new institute is now
bundling all of its competences relating
to automation in order to serve industrial
customers and clients from the public
sector form one source.
The IOSB has competences in the
area of automation, particularly in
the following fields:
• adaptivity: Mechanical transformation
ability of production systems has been
resolved for the most part, while the
IT-based ability to automatically adapt
to changes in production is still far
from completion. For this reason, the
IOSB is developing technologies that
vis IT
Core competence
10
enable adaptivity of all productionrelated software components in a
factory on a permanent basis. Adaptivity also applies to all plant objects:
from a single field unit to the complete plant. Changes to individual
instances may have extensive effects
on all levels of the plant. In practice,
changes to production equipment
may not only »displace« systems
within a plant, but also result in an
increased number of software adjustments, for example:
- on the basis of embedded software
in field units linked via the field bus,
e.g. in sensors, actuators, drives,
valves etc.,
- for controlling software for machines
and systems, e.g. programmable
logic controllers (PLCs),
- for software overlaying direct system
controllers, e.g. manufacturing
execution systems (MES).
• Measurement and control technology:
System technology methods are
researched and implemented with
areas of focus being discrete and
continuous manufacturing processes,
autonomous mobile systems and
environmental processes. Analytical,
knowledge-based and data-driven
procedures and experimental system
analyses are utilized methodically. This
leads to block-oriented models and
finite element models, e.g. MATLAB /
SIMULINK, COMSOL, FEFLOW. Fields
of application include e.g. modeling
of complex continuous production
processes. In control technology, we
focus on the field of application of
M TECHNOLOGY IS THE BACKBONE AND BRACE
C class, W 203 production in the Bremen plant.
Photo: Daimler AG
industrial processes for model and
knowledge-based control concepts,
e.g. model-predictive controls, fuzzy
control or multi-criteria optimization
for continuous production processes.
In this case, the goal is to optimize
processes and products in process and
manufacturing technology. For applications involving robotics, e.g. humanoid robots, controls with imaging
sensors (visual servoing), multimodal
discreet-continual control concepts
and environmental-interactive controls
are researched and implemented.
Methods for navigation and path
planning (simultaneous localization
and mapping, SLAM) and geometric
environmental modeling are also being
developed for autonomous robots.
• State-oriented maintenance, condition
monitoring and data mining: Droves
of heterogeneous data bases exist in
manufacturing companies; however,
most of them are not linked and are
only for short-term use. For this reason,
the IOSB is developing software tools
capable of linking and interpreting
such data bases. The goal is to monitor
complex production processes online,
IOSB simulator for evaluating complex control
processes.
to analyze error statuses and support
the system user when making decisions. Classification procedures,
machine learning procedures and
data mining methods are used to
analyze process or product data.
The fields of application are processengineering and manufacturing
equipment, e.g. for online process
management or condition monitoring.
• Monitoring and MES systems: MES
systems are the backbone for all information in a factory. The IOSB is developing individual customer solutions
with standard interfaces to the automation and ERP level. The institute
has long been a pioneer for such systems, which are then subsequently
adopted by software companies or
the manufacturing industry.
• Vertical integration and semantic
interoperability: The IT architecture
in manufacturing companies in most
cases has grown in a heterogeneous
and historical manner, for which
reason it was not integrated. The
IOSB develops solutions relating to
the topic of interoperability, which
Considerable growth in regenerative energies
requires intelligent energy management.
establishes the prerequisite for the
seamless interaction of different IT
systems across all factory levels.
• System technology for automation:
The IOSB considers automation to
be a product, meaning that new
automation solutions are developed
in this area.
With this new, bundled range of services,
the Fraunhofer IOSB offers forwardlooking solutions for manufacturing
companies from the manufacturing and
process industry, for system integrators
and automation providers.
ENERGY SYSTEMS
The energy markets in Europe have
changed dramatically as the result of
deregulation and the forced expansion
of regenerative, decentralized energy
producers in recent years. In order to
continue to master the challenges of
economic, ecological, efficient and
secure energy supply in a liberalized
environment in the future, a qualified
evaluation of the entire energy supply
system is necessary.
11
vis IT
Core competence
SYSTEM
TECHNOLOGIES
YOUR CONTACTS:
Dr.-Ing. Olaf Sauer
Business unit automation
+49 721 6091-477
Dr.-Ing. Michael Heizmann
Systems for Measurement,
Control and Diagnosis
+49 721 6091-329
Dr.-Ing. Peter Bretschneider
Ressource Management Energy
+49 3677 461-102
Dr.-Ing. Thomas Usländer
Information Management
and Production Control
+49 721 6091-480
Dipl.-Inform. Thomas Kresken
Secure Communication Architectures
+49 721 6091-273
The IOSB and its system technology
application center (AST) develop innovative solutions which are optimized
with respect to system technology for
all energy sources with a special focus
on electric energy.
In addition to the supply process,
aspects relating to information and
communication must also be considered
as well as the various dimensions of
business processes.
feeders in virtual power plants.
The IOSB has specialized in computerbased, intelligent and optimized process
management for water management
for the topics of the supply of fresh
water and wastewater treatment. An
innovative software family that can be
deployed flexibly covers the numerous
fields of applications for the supply of
water and wastewater treatment almost
in their entirety.
One of the IOSB’s focuses of research
and development regarding the topic
of energy and environment involves
the effective and sustainable use of
socially and economically critical resources through the use of models and
methods that employ technical systems.
The solutions developed at the IOSB are
utilized around the world, for example
decision support systems for optimal
water management in Mongolia or
China, energy management solutions
for distribution system operators in
Germany, Austria, Portugal and Hungary,
software-supported leak detection in
the »Great-Man-Made-River« project
in Libya or innovative electric wheelchair
remote maintenance components in
Finland. Other projects are marketoriented initial research in the field
of »Advanced Energy Storage« with
development focusing on scalable,
stationary energy storage devices and
the ICT energy lab for developing
innovative technologies for the management, supervision and monitoring of
energy systems for liberalized energy
markets and coordinated operations
management of decentralized allocated
The aforementioned methods for navigation, path planning and geometric
environmental modeling are also used
in this field of application. These technologies are researched both for terrestrial
and underwater craft, e.g. for inspecting
fresh water pipelines and sewage conduits or for monitoring properties.
vis IT
Core competence
12
ENVIRONMENTAL
SYSTEMS
Another focus, which the IOSB has
pursued for a number of years, is the
development of environmental information systems (EIS).
EIS are essential tools for users on all
levels of environmental administration.
They enable short and long-term
monitoring and analysis of environmental states, assist environmental and
health authorities in planning measures
and complying with reporting duties
vis-à-vis the German Federal Environmental Agency and the EU. The IOSB
covers the entire development cycle,
from the analysis of requirements via
the software architecture design and
Support of environmental and health authorities in implementing European guidelines using WaterFrame® information systems.
SECURE COMMUNICATION
SYSTEMS
programming on the basis of their own
software frameworks up to and including long-term support and further
development. With information systems,
the focus lies on the areas of ground
water, surface water, fresh water and
business inspectorate.
Recent developments involve the use
of semantic technologies and integration in service-oriented architectures and
online sensor networks on the basis of
international standards of the Open
Geospatial Consortium (OGC). The
design of open system architectures and
provision of generic services creates the
basic prerequisite for the interdisciplinary
approach of IOSB projects and for the
interoperability of energy and environmental systems.
The competences of information security,
identity management and networks are
essential for system technology.
The dependency on communication systems requires a high level of security –
whether for protection and maintenance
the confidentiality of communication
processes themselves or authenticity of
communication partners. At the same
time, users require easy, transparent
access to information and services.
In the process, only authorized persons
may gain access and their management
rights must be mapped such that they
are guarded against manipulation. It is
furthermore expected that misuse of
personal data is prevented.
Information security focuses on
security management:
We consider a company’s strategy, its
processes, IT policy and legal boundary
conditions. In addition to solutions for
supporting the collection and processing
of personal data, the IOSB also develops
solutions for meeting needs in terms of
privacy and the minimization of data.
We research and integrate solutions that
monitor and assure the agreed handling
of data in surveillance systems on the
technical level.
The IOSB is responsible for the secure
provision of communication services
and solutions for the institutes of
the Fraunhofer-Gesellschaft. It also
maintains one of two trust centers
of Fraunhofer’s own PKI, which the
IOSB helped to establish.
13
vis IT
Core competence
IMAGE
EXPLOITATION
IMAGE EXPLOITATION AND ITS
Interactive image exploitation on the digital situation table.
C O N TA C T
Following its merger in early 2010, the
IOSB now ranks among Europe’s largest
research institutes working on image
exploitation, sensor fusion, and pattern
recognition. New image exploitation
methods for a wide range of applications have been developed for decades
at the locations in Karlsruhe and
Ettlingen. The range of services offered
by the IOSB covers the entire chain from
image acquisition, image exploitation up
to and including support of the human
decision maker or fully-automated
decision-making in a real-time closed
loop process.
Dipl.-Inform. Markus Müller
Video Exploitation Systems
(VID)
Fraunhofer IOSB
Phone +49 721 6091-250
[email protected]
www.iosb.fraunhofer.de
Development focuses on the recognition
of objects and situations in individual
images and image series with input from
multiple sensors. In addition to classic
sensor technology (visual-optical – VIS),
(thermal) infrared (IR), synthetic aperture
RADAR (SAR), ultra violet (UV), gated
viewing, deflectometric or macroscopic
systems etc. are also utilized. Sensor
technology can be firmly installed in
vis IT
Core competence
14
the process or located on (mobile)
piloted or autonomous platforms
(space, land, air, water).
The range of applications involves the
analysis of material surfaces, protection
and security, disaster protection, driver
assistance systems, robotics and many
other fields.
Model-based methods for pattern
recognition are predominantly used for
application-specific image and signal
exploitation. These methods are for
the most part implemented with realtime capabilities. Examples include
driver assistance systems designed to
respond in the event of an imminent
crash, the detection of flaws on reflective surfaces or identification of damages after a disaster. As data from a single
sensor is not sufficient for many applications, methods for merging networked
multi-sensor systems are aggregated
in order to provide a better informational basis.
VERSATILE APPLICATION POSSIBILITIES
INTERACTIVE IMAGE
EXPLOITATION
Projects focusing on image exploitation
for human-machine interaction form
the visual pillar of intuitive and consequently easy-to-use multi-modal interaction for human users.
In addition to speech, people also
communicate intensively using gestures.
A communication intention can be
determined based on the different body
movements and gestures of a person.
This development work focuses among
other things on interaction with applications from the field of situational
analysis for managing events involving
large-scale damages and the interactive
exploitation of air and satellite images.
Communication with the machine can
be done by large, distributed screens.
In one application line, which is represented by the »SmartControlRoom« of
the IOSB, a group of people interacts
within an observational room such as
a situation room, preferably in front of
large screens. People are identified by
means of cameras allocated throughout
the room both via face recognition,
reconstructed using 3D voxel models,
tracked with respect to their position
in the room and their pointing gestures
and head position are surveyed so that
individual and collective interaction
with the visualized application is
enabled based on needs.
In the second application line, interaction on table displays, e.g. the digital
situation table (DigSt), is supported for
a wide range of tasks such as aerial
image interpretation or plant planning.
The hand-gestures of human users are
interpreted in the process and surveyed
exactly with respect to their position
above the table. This enables highly varied
gestural interaction with the system.
SmartControlRoom.
15
vis IT
Core competence
IMAGE
EXPLOITATION
YOUR CONTACTS:
Dr. Elisabeth Peinsipp-Byma
Interactive Analysis and Diagnosis
+49 721 6091-262
Dr.-Ing. Rainer Schönbein
Interoperability and Assistance Systems
+49 721 6091-248
Collection of 3-D data and scene objects from a car in motion.
Dr. Michael Arens
Object Recognition
+49 7243 992-128
Dr.-Ing. Karsten Schulz
Scene Analysis
+49 7243 992-106
Dipl.-Inform. Markus Müller
Video Exploitation Systems
+49 721 6091-250
Dr. Alexey Pak
Variable Image Acquisition and
Processing Research Group
+49 721 608-45912
PROTECTION AND
SECURITY
Civil and military task forces are
confronted with the consequences of
disasters, terrorist activities, and other
threatening scenarios both nationally
and internationally. The security situation is tense, for which reason, the EU,
the BMBF, traditionally the BMVg, and
the Fraunhofer-Gesellschaft have initiated projects for security research, which
involve the IOSB in various respects.
The initial situation described poses very
high demands on image-based reconnaissance, the protection of task force
members and bystanders as well as the
management and execution of corresponding task force missions. The IOSB
takes on these challenges with its
extensive competence in the fields of
optronic data collection, real-time sensor data exploitation and interoperable
result distribution for decision-makers.
vis IT
Core competence
16
Multi-sensor platforms enable rapid
three-dimensional and graphic scanning of an operational area. Methods
for generating and processing such
data as well as various systems on their
exploitation have been developed for
this purpose. Various systems from the
IOSB are already in use such as:
• ABUL – Video exploitation for reconnaissance and surveillance purposes.
• CSD – interoperable exchange of
data and information.
• RecceMan® – Image exploitation assistance for image interpreters.
Ultra-modern feature-based approaches
as well as fusion and registration methods
make it possible to detect changes e.g.
in urban structures, buildings, or along
streets through the automatic comparison of data records collected at different
times. This enables rapid detection of
damage and critical points in disaster
situations. In other applications, scene
changes that are detected may indicate
special threats (e.g. improvised explosive
devices (IEDs), which have been deposited).
Innovative biometric methods also
address forms of threat analysis, e.g. by
determining the movement behavior of
single or groups of persons. This is
important for applications such as mass
panic detection, violent crimes, but
also for acquiring evidence pointing
to potential terrorists.
Pirate boats in pursuit or floating sea
mines pose a serious danger in the
area of marine transportation. Slight
differences in intensity on infrared
Real-time-tracking of vehicles.
Distribution of reconnaissance data via a coalition shared data (CSD) server.
images contribute to the detection
of such objects by means of a series
of specially adapted methods of automatic image series exploitation.
Advanced exploitation methods can,
together with special sensor technology (e.g. gated viewing or SAR), also
prove effective in otherwise hopeless
situations. This involves for example
seeing through fire in order to acquire
decisive image information on what is
happening behind the fire curtain in
the event of fires in buildings, tunnels,
or forests.
SAR sensors are particularly suitable for
real-time remote reconnaissance as they
are not only capable of seeing through
fires, but can also be used at any time
of day and night regardless of weather.
The IOSB has developed SAR exploitation
methods for numerous applications
(e.g. object detection) and is continually
working on new and further developments. SAR simulators are available for
training purposes and for tests of automatic exploitation methods.
An important task of reconnaissance
is the protection of critical infrastructures. Automated image exploitation
provides the analyst with relief when
solving partial tasks, but is not capable
of performing the entire analysis to date.
The »SiteAnalyst« software also harnesses the different strengths of
humans and machines. Humans assume
the task of recognition in this respect
while the computer is able to verify
thousands of possible interpretations
with respect to object allocation in
fractions of a second.
The deployment of task forces often
occurs within a heterogeneous network. Interoperability is of particular
importance in this respect. For this
purpose, the IOSB developed a CSD
(Coalition Shared Database), which
enables the linking and configurable
synchronization of complex system
networks. A CSD acts as the core of
needs-based networking of stationary
and mobile sensor technology with
exploitation and guidance systems for
reliable decision making.
17
vis IT
Core competence
YOUR CONTACTS
OPTRONICS
SYSTEM TECHNOLOGIES
Optronics (OPT)
Energy (NRG)
Dr. rer. nat. Reinhard Ebert
Dr.-Ing. Peter Bretschneider
Phone +49 7243 992-140
Application Center System Technology (AST)
[email protected]
Phone +49 3677 461-102
[email protected]
Signatorics (SIG)
Water and Mobile Systems (WMS)
Dr. rer. nat. Karin Stein
Prof. Dr.-Ing. Thomas Rauschenbach
Phone +49 7243 992-120
Application Center System Technology (AST)
[email protected]
Phone +49 3677 461-124
[email protected]
Visual Inspection Systems (SPR)
Information Management
Prof. Dr.-Ing. Thomas Längle
and Production Control (ILT)
Phone +49 721 6091-212
Dr. Thomas Usländer
[email protected]
Phone +49 721 6091-480
[email protected]
vis IT
Core competence
18
IMAGE EXPLOITATION
Application Center Industrial
Interactive Analysis and Diagnosis (IAD)
Scene Analysis (SZA)
Automation (INA)
Dr. Elisabeth Peinsipp-Byma
Dr.-Ing. Karsten Schulz
Prof. Dr.-Ing. Jürgen Jasperneite
Phone +49 721 6091-393
Phone +49 7243 992-106
Phone +49 5261 702-572
[email protected]
[email protected]
Systems for Measurement,
Interoperability and Assistance
Video Exploitation Systems (VID)
Control and Diagnosis (MRD)
Systems (IAS)
Dipl.-Inform. Markus Müller
Dr.-Ing. Michael Heizmann
Dr.-Ing. Rainer Schönbein
Phone +49 721 6091-250
Phone +49 721 6091-329
Phone +49 721 6091-248
[email protected]
[email protected]
[email protected]
Secure Communication
Object Recognition (OBJ)
Variable Image Acquisition
Architectures (SKA)
Dr. Michael Arens
and Processing (VBV) Research Group
Dipl.-Inform. Thomas Kresken
Phone +49 7243 992-147
Dr. Alexey Pak
Phone +49 721 6091-273
[email protected]
Phone +49 721 608-45912
[email protected]
[email protected]
[email protected]
19
vis IT
Core competence
Karlsruhe
fraunhofer institute of optronics,
system technologies and image
exploitation iosB
fraunhoferstraße 1
76131 Karlsruhe
Phone +49 721 6091-0
fax +49 721 6091-413
[email protected]
www.iosb.fraunhofer.de
ettlingen
fraunhofer institute of optronics,
system technologies and image
exploitation iosB
gutleuthausstr. 1
76275 ettlingen
Phone +49 7243 992-130
fax +49 7243 992-299
www.iosb.fraunhofer.de
ilmenau
fraunhofer application center
system technology
am Vogelherd 50
98693 ilmenau
Phone +49 3677 4610
fax +49 3677 461-100
[email protected]
www.iosb-ast.fraunhofer.de
lemgo
fraunhofer application center
industrial automation
langenbruch 6
32657 lemgo
Phone +49 5261 702-572
fax +49 5261 702-5969
[email protected]
www.iosb-ina.fraunhofer.de
Beijing
fraunhofer office for Production
and information technologies,
Beijing representative office
unit 0610, landmark tower ii
8 north dongsanhuan road
chaoyang district
100004 Beijing, Pr china
Phone +86 10 6590 0621
fax +86 10 6590 0619
[email protected]