Advantages of Spatial-temporal Object Maps
for Service Robotics
Zoltan-Csaba Marton, Nico Blodow, and Michael Beetz
{marton, blodow, beetz}@cs.tum.edu
Motivation
Environment models
are resources that
enable robots to perform their tasks more
reliably, efficiently, and
competently by using
information about the
environments. Robots
that are to perform everyday manipulation
tasks in human living
environments
must
have
environment
models that are much
more comprehensive
and richer than the
models that have
been employed by
robots so far. It is not
sufficient to simply
have a representation
of the spatial layout of
the environment together with perceptual descriptions of locations as it is sufficient
for navigation. Also, environment models that contain object models and semantic
information are not sufficient for carrying out activities of daily life.
Consider, for example, a household robot that is to set the table, to load the dishwasher and to clean up. Such a robot must know that clean glasses can typically be
found in the cupboard, that glasses on the table are often in use, intended to be used
by somebody or dirty, that glasses on the table suggest that people sit or have been
sitting there, that glasses are put into the dishwasher to clean them, etc.
In human environments, almost every object is designed for certain actions and purposes. As a consequence, there are strong correlations between objects, their locations, their state, and the activity context. These correlations must be known to the
robot in order to perform the right action on the right object in the right way.
Use of Semantic Object Map for
Task-Relevant Object Detection
Exploiting Gathered Knowledge for
Environment Adaptation
Such a model makes
it possible to define
regions of interest
in which objects are
expected by encoding
a large amount of
knowledge about the
environment.
Semantic queries can
be performed by the
robot to extract taskrelevant information,
like opening trajectory
for a drawer for example.
Additionally,
it can be used by the
knowledge base in
order to reason about
the plausibility of tasks and object locations. It is important to be integrate Spatialtemporal Object Maps as well in perception tasks as in the planning of actions. For
example, learning of household object organizational principles allows inferring likely
object locations based on where other, similar objects are stored.
Observation
Ontology &
Semantic Similarity
Inference Result: Fridge
Thing
Þ
Þ
SpatialThing
FoodOrDrink
Milk
Carbonated
Beverage
WholeMilk
Fanta
Learning Relevant Objects and Task
Execution Parameters from Humans
Cylindrical grasp
Inclination angle
Height
Pancake size
Service robots, such as household robots, perform the same kinds of tasks with the
same objects in the same environment over and over again. This enables them to
learn and make use of more specific perception mechanisms for the particular objects
and environments through task and environment adaptation. This enables the robot
to better perceive the objects in its environment by exploiting its experience and considering only relevant objects. One example of such an environment adaptation is that
the a robot performing daily chores in a specific kitchen environment exploits the fact
that it knows almost all objects in its environment. It can then use this assumption in
order to specialize its perception mechanisms. It might turn out that cups are the only
yellow objects in the environment. Thus in order to find the cups it suffices to look
for yellow objects. The advantage of such specialized perception mechanisms is that
they are often more reliable, more accurate, and more resource efficient than their
general counter parts. However, as the knowledge of all objects is only an assumption and will often be violated as new objects are acquired and old ones thrown away,
the robot must be prepared that its perception routines might not work as intended
and adapt its perceptual apparatus in a lifelong learning process.
The joint probability distributions of Spatial-temporal Object Maps are to represent the
dynamic aspects of the environment in the context of everyday activities: the typical
locations and arrangements of objects, the roles that objects play in activities, and
the changes of objects’ state and position over time. The symbolic knowledge can be
represented in K NOW R OB, which supports reasoning about the environment model,
about human activities and about actions the robot had previously performed.
References
[1] Z.-C. Marton, D. Pangercic, N. Blodow, and M. Beetz, “Combined 2D-3D Categorization and Classification for Multimodal Perception
Systems”, in The International Journal of Robotics Research, pp. 1378 - 1402, Volume 30, Issue 11, September 2011.
[2] S. Albrecht, K. Ramirez-Amaro, F. Ruiz-Ugalde, D. Weikersdorfer, M. Leibold, M. Ulbrich and M. Beetz, “Imitating human reaching
motions using physically inspired optimization principles”, in 11th IEEE-RAS International Conference on Humanoid Robots, Bled,
Slovenia, October 26-28, 2011.
[3] N. Blodow, L. C. Goron, Z. C. Marton, D. Pangercic, T. Rühr, M. Tenorth, M. Beetz, “Autonomous Semantic Mapping for Robots
Performing Everyday Manipulation Tasks in Kitchen Environments“, in IEEE/RSJ International Conference on Intelligent Robots and
Systems (IROS), San Francisco, California, September 26, 2011.
[4] R. B. Rusu, Z. C. Marton, N. Blodow, M. E. Dolha, M. Beetz, “Functional Object Mapping of Kitchen Environments“, in IEEE/RSJ
International Conference on Intelligent Robots and Systems (IROS), Nice, France, September 22-26, 2008.
[5] M. Tenorth, L. Kunze, D. Jain, and M. Beetz, “Knowrob-map knowledge-linked semantic object maps”, in Proceedings of 2010
IEEE-RAS International Conference on Humanoid Robots, Nashville, TN, USA, December 6-8 2010.
[6] Ian Horswill. “Integrating vision and natural language without central models”, in Proceedings of the AAAI Fall Symposium on Embodied Language and Action, 1995.
http://ias.cs.tum.edu
Intelligent Autonomous Systems
Technische Universität München, Munich, Germany