Haptics Technologies:
Bringing Touch to Multimedia
C1: Haptics: General
Principles
Roadmap of the course/book






Haptic Applications (Chapter 2)
General Principles Human Haptic Perception (chapter 3)
Haptic interfaces and hardware (chapter 4)
Computer Haptics (chapter 5)
Multimedia Haptics (chapter 6)
Summary, conclusions, and future directions (chapter 7)
Introduction

Vision




Audition




Via human eyes and most information rich media
Humans perceive between 0.3 and 0.7 microns wavelength
Temporal and spatial limitations define visual displays
Via human ears and very effective
Humans perceive 16 Hz to 20000Hz (1000Hz and 4000Hz)
Auditory interfaces are very mature (2D and 3D sound effects)
Touch



Distributed all over the human body, up to 1 KHz sensitivity
Tactile and kinesthetic
Bi-directional flow of information and energy
Haptics exploration
Haptics refers to the science of manual sensing (exploration for information
extraction) and manipulation (for modifying the environment) through touch
Force representation in a virtual
world
1. The user manipulates the
generic probe of the haptic
device,
2. the device tip position is read
and conveyed to the
computer.
3. the computer checks for
collisions between the
simulated stylus and the
virtual objects populating the
virtual environment.
4. If a collision has occurred,
reaction forces/torques will be
computed and applied.
5. In the case that no collision is
detected, no forces will be
computed/applied
Concepts and Terminology -1




Haptic: the science of applying tactile, kinesthetic, or both
sensations to human-computer interactions. It refers to the ability of
sensing and/or manipulating objects in a natural or synthetic
environment using a haptic interface.
Cutaneous: Relating to or involving the skin. It includes sensations
of pressure, temperature, and pain.
Tactile: Pertaining to the cutaneous sense, but more specifically the
sensation of pressure rather than temperature or pain.
Kinesthetic: relating to the feeling of motion. It is related to
sensations originating in muscles, tendons and joints.
Force Feedback: Relating to the mechanical production of
information that can be sensed by the human kinesthetic system.
Concepts and Terminology -2




Haptics or haptic technology: an emerging interdisciplinary field that
deals with the understanding of human touch sensory (Human
Haptics), motor characteristics (Machine Haptics) and with the
development of computer-controlled systems (Computer Haptics)
Haptic communication: the means by which humans and machines
communicate via touch
Haptic device: is a manipulator with sensors, actuators, or both. The
most popular are tactile-based, pen-based, and 3 degrees of
freedom (DoF) force-feedback devices; more degree of freedom
devices exist.
Haptic interface: consists of a haptic device and software-based
computer control mechanisms. It enables human-machine
communication through the sense of touch.
Haptic perception: the process of perceiving the characteristics of
objects through touch
Concepts and Terminology -3



Haptic rendering: the process of calculating the interaction force. Haptic
rendering is therefore a system that consists of three parts, a collision
detection algorithm, a collision response algorithm, and a control algorithm.
Sensors and Actuators: A sensor is responsible for sensing the haptic
information exerted by the user on a certain object and sending these force
readings to the haptic rendering module. The actuator will read the haptic
data sent by the haptic rendering module and transform this information into
a form perceivable by human beings.
Tele-haptics: the science of transmitting haptic sensations from a remote
explored object/environment, using a network such as the Internet, to a
human operator.
Tele-presence: the situation of sensing sufficient information about the
remote task environment and communicating this to the human operator in
a sufficiently natural way.
Concepts and Terminology -4




Virtual Reality (VR): the computer simulation of a real or virtual world
where users can interact with it in real time and change its state to
increase realism.
Virtual environment (VE): is an immersive virtual reality that is
simulated by a computer and primarily involves audio-visual
experiences. A virtual environment is mainly concerned with defining
interactive and virtual image displays.
Collaborative Virtual Environments (CVE): is one of the most
challenging fields in VR where the simulation is distributed among
geographically dispersed computers.
Simulation Engine: is responsible for computing the virtual
environment behavior over time.
Collaborative Haptic Audio Visual Environment (C-HAVE): C-HAVE
allows multiple users, each with his/her own haptic interface, to
collaboratively and/or remotely manipulate shared objects in a
virtual or real environment.
Path 1 - Human Haptics

The study of human sensing and manipulation
through tactile and kinesthetic sensations

Includes understanding



The human sensory system
The human perception and cognition system
The human motor system
Path 2 - Machine Haptics



Involves designing, constructing, and developing
mechanical devices that replace or augment human
touch
Measure the poses (positions and/or orientations) and/or
contact forces of any part of the human body, and
display the computed reaction touch
Divided into two categories: force feedback devices and
tactile devices.


Force feedback devices display force and/or torque and enable
users to feel resistive force, friction, roughness, etc.
Tactile devices present vibration, temperature, pressure, etc. on
the human skin
Path 3 - Computer Haptics

Design and development of algorithms and software that computes
interaction forces and simulates physical properties of touched
objects

Haptic rendering:


Haptic rendering refers to the set of algorithms and techniques that are
used to compute and generate forces and torques in response to
interaction between the haptic interface avatar inside the virtual
environment and the virtual objects populating the environment
Two types of algorithms:
 collision detection
 force computation algorithms
The Roundabout - Multimedia Haptics


Considers Haptics as a new media channel in a
complete multimedia system
Defined as:


the acquisition of spatial, temporal, and physical knowledge of
the environment through the human touch sensory system and
the integration/coordination of this knowledge with other sensory
displays (such as audio, video, and text) in a multimedia system
Comprises

Haptic media creation, storage, compression, dissemination,
representation, communication (transporation) and security
(biometrics)
Haptic-Audio-Visual Multimedia
System
Case Study: HugMe interpersonal
Communication System
1. The child is wearing a haptic suit (haptic jacket) that is capable of
simulating nurturing physical stimuli.
2. The parent uses a haptic device to communicate his/her feelings to
his child.
3. A 2.5 dimensional (2.5D) camera is used to capture the image and
depth information of the child and send it to the parent
Case Study: HugMe interpersonal
Communication System
Human in the Middle!
When
you are
away,
Your child
is crying
cannot
reachhim
your hugging
hands to hug
andtickling
tickle
YouYou
can
soothe
and
What can we do for this poor child???
Haptic Teleconferencing HugMe
With a haptic device
you can touch him
With a haptic jacket
he can feel touch
How it works?
Capture 3D(2.5D) video

Using a
depth
camera
Depth image
Color image
Reconstructed 3D scene
How it works? Demo Video
Fiducial marker
for tracking

Haptic jacket
Controller with Bluetooth and 9V battery
Pager motor
Inner fabric to
protect the skin
Further Reading about Haptics

In recent years, there has been extensive research literature on all
aspects of haptic systems. Some journals, such as IEEE
Transactions on Haptics, IEEE Transactions on Instrumentation and
Measurement, ACM Transactions on Graphics, ACM Transactions
on Applied Perception, ACM Transactions on Multimedia
Computing, Communications and Applications, MIT Press
Presence, Springer Multimedia Tools and Applications, Springer
Multimedia Systems Journal, and the Electronic Journal of Haptics
Research “Haptics-e” frequently publish haptics-based research
results. Many other journals, such as Journal of Robotics and
Mechatronics, have special issues on this subject.
谢谢！
ありがとう！
‫متشکرم‬