Collision Safety for
Physical Human-Robot Collaboration
IROS 2015 Workshop “Physical
Human-Robot Collaboration”
Jae-Bok Song
School of Mechanical Engineering
Korea University
Seoul, Korea
Outline
 Human-Robot Interaction
 3-Step Safety Strategy
•
Collision Prediction & Avoidance
•
Collision Detection & Reaction : Active Safety
•
Collision Absorption : Passive Safety
 Advanced Collision Detection
•
Sensorless Collision Detection
•
Collision Detection Index (CDI): Frequency-based
•
Collision Detection Index (CDI): Projection-based
 Collision Analysis & Simulation
 Summary
2
Collision Safety
3
 Physical Human-Robot Interaction
• Frequent contacts between humans and robots
• Sharing the same workspace  Collaborative robots
Safety Strategies
Before collision
Avoidance
Vision
Collision
Detection
Torque sensing
After collision
Absorption
Spring
Human-robot collision
Need for collision safety
Safe Physical Human-Robot Interaction (pHRI)
4
 3-Step Safety Strategy
Approach
to Human
Step 1
Prediction
Step 2
Active Safety
Step 3
Passive Safety
Collision Prediction
Non-contact sensors
Collision Detection
Sensor: JTS, skin
Sensorless:
current monitoring
Collision Absorption
Safe Joint Mechanism
Path Regeneration
Path planning
Safe Motion
Collision avoidance
Success
Fail
Collision Reaction
Limit switch
Fail
Success
Collision Safety !!
Collision Reaction
Emergence stop
Reflex motion
Success
Step 1: Collision Prediction and Avoidance
5
 Collision Prediction
• Based on noncontact sensors
- Vision sensors, Kinect sensors
Camera
Kinect
- Ultrasonic sensors
Capacitive sensor
Ultrasonic sensor
Collision prediction and avoidance
No
Human
approach
Original
path
Danger?
Yes
Trajectory
generation
New path
<Collision avoidance using Kinect>
Step 1: Collision Prediction and Avoidance
 Problems with Vision System

Use of Ultrasonic Sensors
• Occlusion  Multiple cameras
•
Multiple sensors needed
• Sensitive to lighting conditions
•
d < 0.3m → Warning
•
d < 0.1m → Emergence Stop
<Collision avoidance using ultrasonic sensors>
6
Step 2: Active Safety
7
 Collision Detection & Reaction @ KU
• Detection: Disturbance observer + JTS
(Joint Torque Sensor)
• Reaction: Different reaction modes
<Collision detection with styrofoam>
<Collision detection & reaction with chest>
Collision Detection using Disturbance Observer (DOB)
8
 Principle of collision detection
• Human-robot collision
 External force applied to a robot
 External torque generated at each joint
* τj : joint torque, τext : external torque
Normal operation
Human-robot collision
τ j = M (q)q + C (q, q )q + g (q)
τ j − τ ext = M ( q)q + C ( q, q )q + g ( q)
Collision can be detected by monitoring external torque.
Collision Detection using DOB
9
 External torque estimation
 + C ( q, q ) q + g ( q)}
• External torque: τ ext = τ j − {M ( q ) q
- Sensor based solution
- Measurement of acceleration
 Use of additional sensors
 Impractical solution
<Joint torque sensor (JTS)>
<Joint module>
- Sensorless solution
- Computation of acceleration
 Numerical differentiation of
encoder signal
 Noise due to differentiation
External torque estimation
<Motor current & Friction model>
for collision detection
Collision Detection using DOB

10
Disturbance observer (DOB)
• Basic disturbance observer
Applications
Robust control
Adaptive control
Fault Detection & Isolation
Dˆ ( s )
Human-Robot
Collision Detection
• Disturbance observer
 G( s)


1
G( s)
ˆ


− 1 U ( s) +
N ( s) +
D ( s )
D( s) = Q( s) 
Gn ( s)
Gn ( s )
 Gn ( s) 

Dˆ ( s ) ≈ Q ( s ) D( s )
(if G(s)/Gn(s) ≈ 1, N(s) ≈ 0 )
Collision Detection using DOB
11
 External Torque Estimator
• External torque estimator based on disturbance observer
System (robot arm joint)
External torque estimator
q(s)
Input  Joint torque
Output  Joint velocity
Disturbance  External torque
τˆext(s)
• External torque estimate: τˆext ( s) = Q( s)τ ext ( s)
Dˆ ( s ) = Q( s) D( s)
τˆext ( s ) =
K
τ ext ( s )
s+K
(Q(s): Low pass filter)
Collision Detection using DOB
Collision detection based on external torque
• External torque estimate in time domain
τˆext ( s ) =
K
τ ext ( s )
s+K
τˆext = K  [τ j − {M ( q)q + C ( q, q )q + g ( q)} − τˆext (dl )] dl
• Generalized momentum: p = M ( q ) q
(De Luca, 2003)
τˆext = K [  (τ j + C T ( q, q ) q − g ( q) − τˆe )dt − p ]
 External torque estimation without the acceleration information
τˆext
|τˆext |≥τth ?
τˆext

12
<Collision detection algorithm>
<Example of typical case>
Collision Detection
13
 Demonstrations
7 DOF manipulator
Weight
Specifications
15 kg TCP speed
1 m/s
Payload
7 kg
Acc.
5 m/s2
Reach
780 mm
DOFs
7
RTOS
TwinCAT Control period 1 ms
Step 3: Passive Safety
14
 Safe Joint Mechanism (SJM)
• Passive joint mechanism consisting of
springs and cam-cam follower mechanisms
• Nonlinear spring system
• High stiffness for positioning accuracy
• Low stiffness for collision safety
• Small & Lightweight
• Automatic return to home position
60
Unsafe region
50
 Operation of SJM
40
• Normal operation
30
 stiff arm  accurate positioning
• Emergency (large impact)
 soft arm  shock absorption
Dangerous
20
10
Low stiffness spring
Certain collision force
High stiffness
spring
Safe region
Working region
00
10
20
30
Inaccurate Displacement (mm)
positioning
40
Passive Safety: Demo
<Balloon & can>
15
Static collision
<Shoulder collision>
<Industrial robot with SJMs>
16
Advanced Collision Detection
1. Sensorless Collision Detection
2. Collision Dection Index (CDI)
• Frequency-based CDI
•
Projection-based CDI
Sensorless Collision Detection
17
 Drawbacks of Sensor-based Collision Detection
• Costly solution due to the use of sensors
• Not applicable to industrial manipulators
 Need for collision detection without the use of extra sensors
 Sensorless Collision Detection
• Estimation of joint torques using the motor current and friction model
Estimation of joint torques without sensors
Sensorless
<Joint torque sensor>
<Motor current>
<Friction model>
Sensorless Collision Detection
18
 Estimation of joint torque
Power transmission
τm : Motor torque
α : Torque constant
i : Motor input current
n : Speed reduction ratio
τm =α i,
τ j = nτ m − τ f
τ ext + τ f = nα i − (M (q )q + C (q, q )q + g (q ) )
Friction torque
 Friction torque model
τ c sgn(τ h ),

τ f = τ s sgn(τ h ),
τ sgn(q ) + τ ( q ),
v
 c
if | q |< ε and qd = 0
if | q |< ε and qd ≠ 0
if | q |≥ ε
Identification of
unknown parameters
IROS 2015, S.D. Lee, M.C. Kim, J.B. Song “Sensorless Collision Detection for Safe
Human-Robot Collaboration”
Sensorless Collision Detection
19
 Estimation of joint torque
Friction torque identification using least-squares technique
Friction torque observer
Analysis on friction torque
q(s)
rˆ ( s )
Identification
Friction model
τ c sgn(τ h ),

τ f = τ s sgn(τ h ),
τ sgn(q) +τ (q),
v
c
if | q |< ε and qd = 0
if | q |< ε and qd ≠ 0
if | q |≥ ε
Regressor
LS technique
Data
=
acquisition
=
̅
Data set
′ =
Sensorless Collision Detection
20
 Demonstrations
Human-robot collision
7 DOF robot arm
Weight
Specifications
15 kg
TCP speed
1 m/s
Payload
7 kg
Acc.
5 m/s2
Reach
780 mm
DOFs
7
RTOS
TwinCAT Control period 1 ms
Collision detection without the use
of any extra sensors
Sensorless Collision Detection
21
 Demonstrations
6 DOF industrial manipulator
5 DOF collaborative robot arm
Specifications
Specifications
Weight
33 kg
TCP speed
1 m/s
Weight
125kg
Payload
6 kg
Acc.
5 m/s2
Payload
15 kg
Acc.
5 m/s2
Reach
1044 mm
DOFs
6
Reach
2105 mm
DOFs
5
TCP speed 1.15 m/s
Collision Detection for Human-Robot Collaboration
22
 Motivation
SAFE
Various tasks of collaborative robots
Human-robot cooperation
DANGER
Contact task
Handling of payload
τext
Unexpected collision
τext
Physical interaction
Generation of external torque
 collision ?
Need for New Collision
Detection algorithm
Frequency-based Collision Detection Index
23
 Frequency-based Approach
• Rate of change of external force: Frequency-based Collision Detection Index
- Safe Intended Contact : Relatively slow rate of change
- Dangerous Unexpected Collision : relatively fast rate of change
 Need for an observer that detects only the fast-changing external torque
Torque (Nm)
 Add a high-pass filter to the conventional collision detector
Frequency-based Collision Detection Index
 Collision detection of unexpected collision
• Threshold: ±0.5 Nm
• Intended contact
- Maximum Residual: 0.2 Nm < threshold
• Unexpected collision
- Maximum Residual: 2.2 Nm > threshold
Intended contact
Collision
24
Frequency-based Collision Detection Index
25
 Limitations of Frequency-based Approach
• No guarantee that intended contact force is always low frequency
• No guarantee that unexpected collision force is always high frequency
 Examples: Collisions in low velocity, clamping
• No clear frequency threshold to distinguish collision from external torque
Box assembly
Measured contact force
Fx
Fy
Fz
Need for more accurate but practical solution
Projection-based Collision Detection Index
26
 Subspace Projection based Approach
• Types of tasks for human-robot collaboration
Collision
Source of τext
w/o collision w/ collision (EE or Body)
Cases
1.
case 1-1
Fcb
none
Fce
2.
Fce
Fi
Fp
3.
τp
τp + τce
case 2-2
τp + τcb
Position control
(painting, welding)
Position control with
payload
(pick-and-place,
material handling)
case 3-1
Fcb
Fe
case 1-2
case 2-1
Fg
Fce
τce
τcb
Fcb
Payload
Applications
τe
τe +τce
case 3-2
τe +τcb
Force control
(grinding, hand
guiding),
Projection-based Collision Detection Index
27
 Subspace Projection based approach
• Collision detection strategy for human-robot collaboration
Cases
1.
Collision detection index
Detectable
collision
Available arms
Fcb
τext
Any robot arms
Fce
2.
Fcb
+
Payload
Fce
Fi
Fp
3.
( I − J p J p )τ ext
6~7 DOF robot arms
( I − J T ( J T ) + ) τ ext
7 DOF robot arms
Fg
Fcb
Fce
Fe
Projection-based Collision Detection Index
28
 Projection based Approach
• Main idea of proposed collision detection method
Example of subspace projection (in Cartesian space)
- Fp = (0, 1, 1) in the yz plane ( only payload)
- Fext = (1, 1, 1) in the xyz space ( col. Included)
- Projection of Fext into the x axis (orthogonal to
the yz plane)
Fext = Fc + Fp
- Collision force  Fc = (1, 0, 0)
• If Fext = Fp
F ext
Fext
 CDI : zero vector
• If Fext ≠ Fp
 CDI : not zero vector
Projection-based Collision Detection Index
29
 Collision Detection for Handling a Payload (Case 2)
τ ext = τ c + τ p
• Available for 6 – 7 DOF robot arms
dim(⊥ S p ) = n − m
CDI = ( I − J p J p + )τ ext
≈ ( I − J p J p + ) τ c + ( I − J p J p + )τ p
≈ ( I − J pJ p+ ) τ c
 CDI : decoupled with τp & sensitive to τc
Projection-based Collision Detection Index
 Experimental results
• Collision detection for various payloads (w/o payload  1kg  2kg)
20
10
0
-10
1kg
0
2
4
2kg
6
8
10
12
CDI (Nm)
-20
The developed CDI can detect a collision for unknown payloads.
30
Collision Detection for Human-Robot Collaboration
31
 Collision detection for Contact Task (Case 3)
• Physical interaction based on force applied to its end-effector
• External force on the end-effector  intended interaction force
τ ext = τ c + τ e
• External force on the body  unexpected collision force
CDI = ( I − J T ( J T ) + )τ ext
≈ ( I − J T ( J T ) + )τ c + ( I − J T ( J T ) + )τ e
≈ ( I − J T ( J T ) + )τ c
 CDI : decoupled with τe & sensitive to τc
Collision Detection for Human-Robot Collaboration
 Experimental results
• Collision detection during hybrid force/position control
•)
Hybrid force/position control
Collision detection
- Intended interaction force for
impedance control in the x direction
- Collision between human and
manipulator
< Written letters: IRL >
32
Projection-based Collision Detection Index
33
 Scenario for human-robot collaboration
Human-robot collaboration in car assembly line
Human-robot
collaboration
Case 1: Approaching
(Position control)
Case 2: Handling of payload
(Pick and place)
Case 3: Physical interaction
(Hand guiding)
Collision Detection for Human-Robot Collaboration
34
 Collision detection strategy
Case 1: Approaching
Case 2: Handling of payload
Case 3: Physical interaction
Normal operation:
Normal operation:
Normal operation:
Collision:
Collision:
Collision:
τ ext = 0
τ ext = τ c
CDI
τ ext
Detectable collision
τ ext = τ p
τ ext = τ c + τ p
CDI
+
( I − J p J p )τ ext
Detectable collision
τ ext = τ e
τ ext = τ c + τ e
CDI
( I − J T ( J T ) + )τ ext
Detectable collision
35
Collision Analysis & Simulation
Various Safety Criteria
36
 Safety criteria for safety evaluation
• ISO 10218-1
- Collaborative operation with humans
- vTCP<0.25m/s, FTCP<150N, Pmax<80W
• Human pain tolerance [Yamada, 1996]
- Static collision (v<0.6m/s)
• Too restrictive criteria
 Limitation of performance
- F<50N
• Head Injury Criterion (HIC)
- Automobile crash test
- HIC<650  prob(AIS≥3)<0.05
• Too generous for a robot arm
- Low collision speed
- HIC saturation with increasing mass
 No robots become dangerous at
2m/s. [Haddadin, 2008]
- Used to be the most popular index
Safety Evaluation
37
 Safety evaluation of human-robot collision
• Real impact test & evaluation
• Using a crash-test dummy
Real impact test
Simulation S/W
Collision analysis
DLR – Haddadin
• Features
+ Most realistic data available
- Considerable cost and time for tests
- Need to construct a robot
Safety Evaluation
38
 Safety evaluation of human-robot collision
• Collision simulation
• Using simulation S/W
Real impact test
Simulation S/W
Collision analysis
MADYMO S/W
• Features
+ Relatively reliable results
+ No need to construct a robot
- Expensive S/W
Safety Evaluation
39
 Safety evaluation of human-robot collision
• Collision analysis and evaluation
• Analytic method
Real impact test
Bicchi ‘04
Simulation S/W
Morita ‘00
Collision analysis
• Features
+ No need to construct a robot
+ Low cost and easy application
- Less reliable data
Various Safety Criteria
 Injury tolerance of body parts
40
Neck (indirect impact)
Injury tolerance
Cranial bone
[SAEJ885, 1980]
Fracture tolerance
Frontal
Temporal
Occipital
4.0 kN
3.12 kN
6.41 kN
Facial bone
[Nahum, 1972 & 1976]
Fracture tolerance
Mandible (C)
Mandible (L)
Zygomatic
Maxilla
Nasal
1.89 kN
0.82 kN
0.85 kkN
0.62 kN
0.342 kN
Compression
[Mertz, 1993]
Extension
[Mertz, 1967]
Flexion
[Mertz, 1967]
Bending angle
[Gadd, 1971]
Chest
Injury tolerance
Neck (direct impact)
Injury tolerance
22mm
Thyroid and cricoid
[Melvin, 1973]
0.337 kN
0.5m/s
Lower extremities
[Devore, 1999]
Injury tolerance
Abdominal
[Miller, 1989]
Injury tolerance
Femur
Tibia
3.8kN
5.4kN
Liver
Lower abdomen
310kPa
3.76kN
Upper extremities
[Begeman, 1999]
Injury tolerance
Humerus
Elbow
Forearm
1.96kN
1.75kN
1.37kN
Compression criterion
[Lau, 1983]
Viscous criterion
[Lau, 1986]
- KR6@2m/s  No injury[Haddadin, ‘09]
Shear
[Mertz, 1993]
Tension
[Mertz, 1993]
3.1kN @ 0msec
1.5kN @ 25-35msec
1.1kN @ 45msec
3.3kN @ 0msec
2.9kN @ 35msec
1.1kN @ 60msec
4kN @ 0msec
1.1kN @ 30msec
57Nm
87.8Nm
Extension: 80°
Lateral: 60°
Safety Criteria
 Safety criterion for service robots (blunt impact)
Safety criteria (Collision force)
• Head injury
• Nasal bone
- Protrusion of head
- Weakest bone of head
- Fracture force : 342 N
 Comminuted fracture
• Neck injury
• Thyroid and cricoid cartilages
- Upper end of airway passage
- Fracture force : 337 N
 Obstruction of airflow
41
HuRoCol: Model Parameters
42
 HuRoCol (Human-Robot Collision Analysis)
 Parameters of collision model
• Human (Hybrid III 50th percentile male)
• Weight: 4.5kg(head), 1.5kg(neck), 71kg(body)
• Neck stiffness: 0.44Nm/deg
• Robot arm
Robot arm model
Hybrid III
HuRoCol: Collision model
43
 Head-Neck Model (3 DOF)
• Head: Revolute joint (OC), Neck stiffness
• Neck: Revolute joint (C7), Neck stiffness
• Body: Prismatic joint
Collision model
Human model
HuRoCol: Collision model
44
 Chest Model
• Lobdell [17]): 2 DOF
• Lumped-mass model of anteroposterior thoracic impact
• To obtain uncoupled inertia matrix
 Dummy mass is added between kve and cve
y
kr
x
cb
kve cve
x6
x7
- kr : rib cage and directly coupled viscera
- cb : air in lungs and blood in the vessels
- kve and cve : viscoelastic tissue such as thoracic muscle tissue
x5
HuRoCol : Collision model
45
 Various collision cases
Unconstrained human
Impact to head
Collision model
Impact to neck
Constrained human
Partially constrained human
Wall
z
z
x
Impact to head
Impact to neck
Wall
Impact to head
x
Impact to neck
HuRoCo : Solution Method
46
 = M (q) −1 (F − C (q, q ) − K (q) − G (q) − D(q ) )
 Solution: q
• Matlab/Simulink
- 4th and 5th-order Runge-Kutta method
Robotica 2015, J.J. Park, J.B. Song, S. Haddadin, “Collision analysis and
safety evaluation using a collision model for a frontal robot-human impact”
HuRoCol : Analysis Results
 Collision with unconstrained human
Impact to head
C7
Robot link
z
x
Body
Collision force (N)
O.C.
500
400
407 N
300
Nasal bone
fracture
342 N
200
100
0
0.4
0.6
0.8
1.0
Time (s)
Impact to neck
47
1.2
Displacement (cm) Angle (deg)
- Impact to the neck is more dangerous than impact to the head.
( airway obstruction)
80
60
40
20 Collision
0
0
0.5
0.6
0.4 Collision
0.2
0
0
0.5
O.C.+C7
C7
O.C.
1.0
1.5
Time (s)
2.0
Body
1.0
1.5
Time (s)
2.0
HuRoCol : Analysis Results
 Collision with partially constrained human
- Impact to the neck is more dangerous than impact to the head.
z
x
Angle (deg)
Wall
Collision force (N)
Impact to head
z
x
Angle (deg)
Wall
Collision force (N)
Impact to neck
48
HuRoCol : Analysis Results
 Collision with constrained human
- Impact to the neck is more dangerous than impact to the head.
Impact to head
500
425 N
400
Wall
z
x
Nasal bone
fracture
300 342 N
200
100
0
Impact to neck
49
0.4
0.6
0.8
1.0
Time (s)
1.2
HuRoCol : Design of Safe Robot Arm
 Design of safe robot arm
- Design of the robot arm can be modified according to analysis results.
- Mass (inertia), length, velocity…
Impact to neck
500
423 N
400
Thyroid & cricoid
fracture (337N)
500
423 N
400
2.5 kg
300
337 N
200
289 N
100
2.1 kg
0.4
0.6
1.5 m/s
1.2 m/s
200
210 N
100
0
300
2.3 kg
344 N
Thyroid & cricoid
fracture (337N)
0.8
1.0
Time (s)
Inertia of robot link
(1.5m/s)
1.2
0
0.4
1.0 m/s
0.6
0.8
1.0
Time (s)
Velocity of robot
(2.5kg)
1.2
HuRoCol : Design of Safe Robot Arm
• The robot arm with SJM can provide much higher safety.
• Design of the robot arm can be modified according to analysis results.
Impact to head
O.C.
C7
Robot link
z
x
Body
Impact to neck
HuRoCol : Verification 1
 Analysis versus Dummy crash-test
• KUKA KR6 (inertia: 67 kg)
• Unconstrained human
 Close agreement with dummy crash-test data
O.C.
C7
Robot link
z
x
Body
Collision force (N)
Impact to head
Haddadin, ICRA ‘09
HuRoCol : Verification 2
 Analysis versus Dummy crash-test
• KUKA KR500 (Refl. inertia: 1870 kg)
• Unconstrained human
 Close agreement with dummy crash-test data
Impact to head
O.C.
C7
Robot link
z
x
Body
Haddadin, ICRA ‘09
Summary
54
• Safe Joint Mechanism: passive approach, infinite bandwidth
• Frequency-based Collision Detection
 Intended contact: low frequency & Collision: high frequency
• Projection-based Collision Detection Index
 Any collision regardless of frequency and magnitude of collision
• Safety Criterion: fracture force of thyroid & cricoid cartilages for neck injury
- The most appropriate safety indicator for a service robot
• Proposed collision model and analysis
• Accurate model
- More reliable analysis results for human-robot collisions
• Evaluation in the robot design phase
- Can save time and cost associated with collision tests
Q&A
Thank you!