IEEE International Conference on
Robotics and Automation (ICRA 2017)
Singapore, May 29 - June 3, 2017
Learning and Control of Robots in
Interacting with Unknown Environments
Yanan Li PhD BSc, Imperial College London
Shuzhi Sam Ge FIEEE, FIFAC, FIET, FSAE, PhD, DIC, BSc
National University of Singapore
http://robotics.nus.edu.sg
Vision of Social Robots
• Social robots: Robots that are able to interact and
communicate among themselves, with humans, and with
the environment, within the social and cultural structure
attached to their roles.
-International Journal of Social Robotics
• More robots out of the laboratory into schools, homes
and clinic places…
• Interacting with people in their own space
• Changing everyday lifestyle
International Journal of Social Robotics
Editor in Chief:
Shuzhi Sam Ge, NUS
Co-Editor in Chief:
Oussama Khatib, Stanford University, USA
Aims and scope:
 Provide a platform for presenting findings and
latest developments in social robotics,
covering relevant advances in engineering,
computing, arts and social sciences.
 Provide an overview of the current state of
the social robotics scene.
 1 Volume(-s) with 5 issue(-s) per annual
subscription
 IF: 1.407
Outline
•
•
•
•
•
Introduction
Impedance Learning
Intention Estimation
Zero Force Regulation
Conclusion and Future Work
Control: Industry vs Social
• Accuracy
• Fast response
• Fixed environment
• Interaction
• Safety
• Compliant behavior
• Unknown environment
Intelligent Control
http://cnx.org/content/m42183/latest/?collection=col11406/latest
http://www.rocketsticks.com/science
Human Limb
Robot Arm
•
Human adjusts their limb
impedance and trajectory
when catching a ball.
•
It is possible to apply
impedance and trajectory
learning skills to robot
control.
Intelligent Control
Human-Robot
Collaboration
Mass-DampingStiffness System
• Robot will act as a load to the
human partner if it plays a follower
role.
• Robot’s trajectory is adapted
according to human’s intention.
Central Nervous System:
Motion Intention
Intelligent Control
• Hybrid Position/Force Control: controlling force and
position in a non-conflicting way
• Impedance Control [N. Hogan, 1985]: developing a
relationship between force and position
Robot
Passive
Environment
Passive
Passive systems include
arbitrary combinations of
masses, springs and dampers,
linear or nonlinear.
Stable [J. E. Colgate, 1988]
Problems and Solutions
• How to impose a target impedance model on a robot arm, in
the presence of uncertainties and unknown robot dynamics?
– Impedance control design (available in the literature)
• How to find a target impedance model?
a. How to determine the impedance parameters, subject to
unknown environment dynamics?
– Impedance learning
– Impedance adaptation
b. How to determine the rest position/desired trajectory in the
impedance model?
–
–
Intention Estimation
Zero Force Regulation
Outline
•
•
•
•
•
Introduction
Impedance Learning
Intention Estimation
Zero Force Regulation
Conclusion and Future Work
Related Works
• Damping control
– Passivity assumption
– Modest performance [S. P. Buerger]
• Known environment dynamics
– Stability
– Optimal performance [R. Johansson; M. Matinfar]
• Unknown environment dynamics
– Environment identification and modeling
– Impedance learning and adaptation
a. Earlier works: simple tasks
b. State-of-the-art: reinforcement learning – complex computation [J. Buchli]
c. Still an open problem
Human Learning Skill
• A new door/ball is an unknown environment.
• Human beings learn to adjust their limb
impedance iteratively when opening a
door/catching a ball.
• It is possible to apply impedance learning
skill to robot control.
Problem Formulation
The dynamics of a robot arm follow an impedance
model:
Consider that the environment is unknown and
dynamically changing, and it is described by
Problem: How to determine
so that a
desired interaction performance is guaranteed?
Define a cost function to measure the interaction
performance:
Impedance Learning
1. According to gradient-following scheme:
2. The target impedance model:
3. How to obtain
?
Impedance Learning
Consider the environment dynamics:
Choose states
The environment is described as
Define
The environment dynamics become
which is a time-varying system.
Betterment Scheme
Theorem [S. Arimoto, 1984]:
Consider the linear time-varying system described by
The control input is iteratively updated in the following manner
The betterment process is convergent in the sense that
as
Convergence guarantee:
Impedance Learning
According to the betterment scheme, we have
By employing gradient-following and betterment schemes, impedance
parameters are updated as:
Experiment Study
ATI 6 axis force/torque
sensor
• Scenario: In each iteration with a period of 18s, the interaction
starts at t = 5s and ends at t = 16s.
• Unknown environment: human hand
• Initial impedance parameters:
• Learning rate:
The First Case
Cost function:
Cost Function
Stiffness
The First Case
Cost function:
Tracking Error
Interaction Force
The Second Case
Cost function:
Cost Function
Stiffness
The Second Case
Cost function:
Tracking Error
Interaction Force
Discussion
• By choosing different cost functions, it is determined that
the control objective can be trajectory tracking, integral force
tracking or the combination/compromise of these two.
• The proposed impedance learning guarantees that the
defined cost function becomes smaller iteratively, subject to
unknown dynamic environments, and the expected
interaction performance has been achieved.
Discussion
• The advantage of the proposed impedance learning over
impedance control with fixed impedance parameters lies in:
a modest performance can be obtained if a good set of
fixed impedance parameters is predefined (when k = 0),
and a better performance can be obtained only with variant
impedance parameters because the environments are
dynamically changing.
• While impedance parameters have been obtained through
iterative learning, how to determine the rest
position/desired trajectory in a target impedance model?
Outline
•
•
•
•
•
Introduction
Impedance Learning
Intention Estimation
Zero Force Regulation
Conclusion and Future Work
Scenario: Human-Robot Collaboration
• Most tasks that are either too complex to automate or too heavy to
manipulate manually are impractical and even impossible to be solely
taken by either fully automated robots or human beings.
• Robots and human beings may share the same workspace and they
have complementary advantages.
Scenario: Human-Robot Collaboration
• The robot will act as a load to the human partner if x0 is far away from x.
• x0 is supposed to be adapted according to human partner’s motion
intention.
• Trajectory adaptation is another human beings’ learning skill which can
be applied to robot control.
Related Works
• Intention estimation
– Motion characteristics of the human limb: minimum jerk model [B.
Corteville, 2007]
– Motion intention state: hidden Markov model [Z. Wang, 2009]
– Intentional walking direction: Kalman filter [K. Wakita, 2011]
– Robot and human partner: separately analyzed
• Force regulation
– Force regulation under impedance control [K. Lee, 2008]
– Existing works: constant rest position and only stiffness in the
environment (human limb)
Problem Formulation
Human limb model [M. M. Rahman 2002]:
motion intention planned in
the central nervous system
Assumption: In a typical collaborative task, the motion intention of the
human partner (in each direction) is determined by the interaction force,
position and velocity at the interaction point (in the corresponding
direction) of the human limb and robot arm.
intention
estimation
Intention Estimation
NN is employed to estimate human partner’s motion intention, as below:
Define a cost function:
Updating law:
Intention Estimation
Experiment Study
• Scenario: The left wrist of Nancy is
moved by human being’s hand.
• Note: The actual trajectory of the robot
arm cannot be compared with the motion
intention directly.
Impedance parameters:
NN setting:
Point-to-Point Movement
Joint angle
Point-to-Point Movement
External torque
Time-Varying Trajectory
Joint angle
Time-Varying Trajectory
External torque
Discussion
• The motion intention of the human partner has been
observed by employing the human limb model.
• With the proposed trajectory adaptation, the robot arm is
able to “actively” follow its human partner.
• Human partner and robot are considered to be two
separated subsystems, and the performance of the whole
coupled interaction system is yet to be rigorously analyzed.
Outline
•
•
•
•
•
Introduction
Impedance Learning
Intention Estimation
Zero Force Regulation
Conclusion and Future Work
Problem Formulation
Human limb model [M. M. Rahman 2002]:
Control objective: design x0 in
so that
.
Three cases:
• Point-to-point movement
• Periodic trajectory
• Arbitrary continuous trajectory
Zero Force Regulation
Trajectory adaptation:
Point-to-point movement:
Periodic trajectory:
Arbitrary continuous trajectory:
Similar to the
intention estimation
method!
Main Result
Theorem:
With the above trajectory adaptations, the following performance of the
closed-loop interaction system can be guaranteed:
The robot is able to “actively” follow its human partner.
Lemma:
The above result will not be affected by the transient performance of the inner
position control loop, if it is asymptotically stable.
Experiment Study
• Scenario: The left wrist of Nancy is
moved by human being’s hand.
• Note: The actual trajectory of the robot
arm cannot be compared with the motion
intention directly.
Impedance parameters:
Point-to-Point Movement
Joint angle
Point-to-Point Movement
External torque
Periodic Trajectory
Joint angle
Periodic Trajectory
External torque
Arbitrary Continuous Trajectory
Joint angle
Arbitrary Continuous Trajectory
External torque
Discussion
• Zero force regulation has been investigated for humanrobot collaboration, so that the robot is able to “actively”
follow its human partner.
• Adaptive control has been proposed to deal with the
point-to-point movement, and learning control and NN
control have been developed to generate periodic and nonperiodic trajectories, respectively.
Outline
•
•
•
•
•
Introduction
Impedance Learning
Intention Estimation
Zero Force Regulation
Conclusion and Future Work
Conclusion
• Impedance learning is developed to obtain desired
impedance parameters when robots interact with
unknown and dynamically changing environments.
• Trajectory adaptation is investigated for human-robot
collaboration, so that robot is able to “actively” follow
its human partner.
Future Work
• Simultaneous learning/adaptation of impedance and
trajectory will be studied.
• Interaction control with other sensory information (e.g.,
image) will be investigated.
• The applications of the proposed methods in specific areas
(e.g., rehabilitation, tele-operation, and human-robot
collaboration) will be considered.
• Critical issues in practical implementations, such as time-delay
and human factor, will be investigated.
References
• C. Wang, Y. Li, S. S. Ge and T. H. Lee, “Reference Adaptation for Robots in
Physical Interactions with Unknown Environments,” IEEE Transactions on
Cybernetics, 2016
• Y. Li and S. S. Ge, “Human-Robot Collaboration Based on Motion Intention
Estimation,” IEEE Transactions on Mechatronics, 2014
• Y. Li and S. S. Ge, “Impedance Learning for Robot Interacting with
Unknown Environments,” IEEE Transactions on Control Systems
Technology, 2013
• S. S. Ge and Y. Li, “Force Tracking Control for Motion Synchronization in
Human-Robot Collaboration,” Robotica, 2013
• S. S. Ge, Y. Li and C. Wang, “Impedance Adaptation for Optimal RobotEnvironment Interaction,” International Journal of Control, 2013
Published Books
• Published Books
 Intelligent Control:
54
Published Books
Switched Dynamical
Systems:
Robotic Systems:
55
International Journal of Social Robotics
Editor in Chief:
Shuzhi Sam Ge, NUS
Co-Editor in Chief:
Oussama Khatib, Stanford University, USA
Aims and scope:
 Provide a platform for presenting findings and
latest developments in social robotics,
covering relevant advances in engineering,
computing, arts and social sciences.
 Provide an overview of the current state of
the social robotics scene.
 1 Volume(-s) with 5 issue(-s) per annual
subscription
 IF: 1.407
Welcome
2017 International Conference on Social
Robotics (ICSR 2017) will be held in Tsukuba
City, November 22-24, 2017—first time in
Japan
http://www.icsr2017.org/committees.html
Acknowledgement
•
•
Machine Technology
– Intelligent Control
– Robot Sensing
– Machine Learning
– Cooperation Mechanisms
Affective Sciences
– Behavioral Studies
– Psychological Impact
– Ethical Implications
– Interface Design