Robotic Assistance
A Guidebot for the Blind
• Providing assistance for
the Blind
– What do we mean by
“Blind?”
• Stereotypical blindness
• Visually impaired
• What assistance is
necessary?
– Many things we would
consider simple tasks
become very difficult
The PROBLEM
The PROBLEM – Types of Challenges
• Environmental
– These are the ones you think of first, getting around
– Don’t move those tables!
– In a new place or a busy city, more likely to have
either sighted assistance or guide dog
– These issues more typical of severe blindness,
though those with low vision have similar problems
The PROBLEM – Types of Challenges
• Social
– Frequent (sometimes subconscious) prejudice
against the blind and visually impaired
– Imagine taking classes here if you were suddenly
blinded
– Hard to find/keep job
– Hard to play sports/other social activities
– Can lead to lowered self esteem
The PROBLEM – Types of Challenges
• Technological
– Those with severe
blindness cannot use
computers, smartphones
in the same way we do
– Visually impaired people
need modified tech,
especially bigger screens
or text-to-speech
The PROBLEM – So What Can We Do?
• The blind face many problems beyond
navigation and movement
– We can only provide so much assistance with
turtlebots
– Limited time
– Extremely limited resources
• We chose to focus on assistance in the
navigational sense
NAVIGATION
Navigation
• The main function of our project
• Pre-mapping
– Not ideal, almost useless in real life, but without it,
too much trial and error
• GPS, Autonomous discovery of space
– Would be more useful, but limited time and budget
• Hybridize these!
Navigation
• Hybrid—Best choice for now
– Instead of following blundering
bot, told to wait
– Once robot finds path, leads
person through it
– This obstacle avoidance can be
something we code from
scratch without use of other
APIs
SPEECH RECOGNITION AND
TEXT TO SPEECH SOFTWARE
Speech Recognition and Text To Speech
• Initially wanted to use HARK
– Problems with “flowdesign,” we emailed the
support team but no idea how long it could be
• Sphinx was another option
– Is part of ROS already
– Unfortunately, tutorial is out of date
– Voice Commands to Turtlebot—Video
• First step is now to use sound_play package
Speech Recognition and Text To Speech
• Sound_Play provides node that
translates commands into sounds
– Node supports built-in sounds,
.OGG and .WAV files, and speech
synthesis
• Basically, take in text, say that text.
Speech Recognition and Text To Speech
• Three steps
– Figure out robot speech with sound_play
• This can also play music and tell time, useful for
technological aspect of blind difficulties
– Have robot accept voice commands, translate to text
• Cannot be done without external software (HARK,
Sphinx)
• If we have problems with this, instead use PS3 controller
– Connect input command and output audio
LOCALIZATION INTERFACE
Localization Interface
• Important part of the puzzle—need an interface
that doesn’t rely on vision
– First thought? Use a leash!
• Not as good an option as we thought—turtlebot is touchy
– What now?
• Want robot to slow down or even backtrack to user
– Hands-Free, Low Strength Radio Transmitter
• Omni-directional, can go through/around objects
Localization Interface – Implementation
• Have user wear ‘beacon,’ attach transmitter to
robot through DB-25 connector
• Read analog inputs and use them to estimate
distance
– These inputs aren’t actually distance values…
– Intensity values from 0 to 4095
• To find actual distance, we would have to play around a
little with the system, see what numbers occur at what
distances
THE INITIAL DESIGN
The Initial Design
Navigation
Localization
First, we figure
out the obstacle
avoidance code,
and make sure it
works.
Text-To-Speech
We must then
program the
localization
interface, so the
person is not left
behind.
Speech Software
Next, we must
attempt to have
the robot speak,
and tell directions
for the user.
Finally, if we have
time and the
proper software,
we program
certain commands
that can be used
by the person.
The Initial Design