AROSOL
(Autonomous Roving Sound Locator)
Sean Lyons
4/21/14
University of Florida
Department of Electrical and Computer Engineering
EEL 4665C – IMDL – Formal Report
Instructors: Dr. A. Antonio Arroyo, Dr. Eric M. Schwartz
TAs: Andy Gray, Josh Weaver, Nick Cox, and Daniel Frank
Contents
Abstract .............................................................................................................................. 2
Executive Summary ........................................................................................................... 2
Introduction ........................................................................................................................ 2
Integrated System ............................................................................................................... 3
Mobile Platform ................................................................................................................. 4
Actuation ............................................................................................................................ 4
Sensors ............................................................................................................................... 4
Behaviors ........................................................................................................................... 7
Experimental Layout and Results .................................................................................... .8
Conclusion ........................................................................................................................10
Documentation ..................................................................................................................10
Appendices ........................................................................................................................10
1
Abstract
Two senses that dominate the human experience are sight and sound. In order for
a robot to interact naturally with humans, it must be able to interact in via sight and
sound. There is a rich suite of applications available for robots that can perform sound
localization, including defense, speech recognition, virtual conferencing, and other
humanoid robotics [1], [2].
Executive Summary
AROSOL was born out of the dream of autonomously tracking sound using a
simple microcontroller. Vision was another side interest that was added in order to aide in
localizing the sound using color detection.
Low level sensing is provided by three front facing short-range IR sensors. These
allow AROSOL to avoid objects that cross its path. For objects that are missed by the IR
sensors, there are front and rear bump sensors that are implemented using four push
button switches.
Actuation is carried out via two Pololu motor driver carriers, controlled by the
microcontroller. The motor drivers drive two 34:1 gear motors. The motors come with
hall-effect shaft encoders that provide feedback to the microcontroller. This feedback
allows the microcontroller to deliver proportional control as well as PID control.
A custom-built band pass filter/amplifier circuit achieves audio sensing. The
filter/amp combo isolates a narrow high frequency band (𝑓𝑐 = 8.7𝑘𝐻𝑧) and amplifies and
level shifts to make the audio signal useable by the microcontroller.
Power is provided via a 7.4V two cell Li-Po battery, it is distributed with a
custom power PCB. Battery eliminator circuits power the microcontroller and the high
level ARM processor.
There are four states that AROSOL operates in: Sound, Obstacle, Camera, and
Mission Complete. AROSOL begins in sound state where it tracks a sound, it then
transitions into Camera state which uses color detection to hone in on sound source. The
obstacle state can interrupt any state in the even that an obstacle is in the way. Mission
complete is a state where the robot shows off that it has completed the mission.
Introduction
AROSOL (Autonomous Roving Sound Locator) is a high-frequency pitch
following robot. Similar to a dog, the bot will have “hearing” that is sensitive to high
frequency pitches, and will be able to locate the source of the sound, much like a dog
following a dog whistle. The problem to be investigated is machine intelligence via
sound. Sound offers new challenges and a lot of opportunities to grow because it
naturally has lots of interference and noise. Sound is an important application to study
because it bridges the gap between humans and robots. It lends itself to creating
humanoid robots (specifically speech processing), but also to the vastly opposing field of
defense. AROSOL will synthesize the digital world of microcontrollers and embedded
computers and the analog world of audio to investigate methods of how sound can make
intelligent machines.
2
Integrated System
Intelligence will be achieved by AROSOL in 2 two ways: a level-low level
environment sensing/reaction, and high-level stimulus processing (vision). Fig 1 is a
graphic describing each of the functional systems within AROSOL.
IR Sensors
Raspberry Pi
• Camera
• Wi-fi Dongle
Battery
Monitor
Serial
Comms
Link
Battery
Power
Microcontroller
Board
(AtxMega128A1U)
Filter/AMP
board
Microphones
Bump
Sensors
Motor
Drivers
Push button
switches
Motors
• Wheels
• hubs
• tires
Fig 1
3
Mobile Platform
The platform is comprised of two levels. The top level holds the microphones,
Raspberry Pi, and the camera, this isolates the microphones from noise created the
motors and drivers and allows the camera unobstructed vision. The bottom level contains
the bump switches, IR sensors, and microcontroller board. This minimizes cable routing.
The simplistic platform design was chose to maximize the area of each layer, thus
spreading the microphones apart for between processing. The mobile platform is shown
below in Fig 2
.
Fig 2
Actuation
AROSOL is differential drive robot with two drive wheels and a castor. Since the
robot is only intended for indoor use on flat flooring the motors are not required to have a
lot of torque or need to spin at extremely high speeds. AROSOL is a low speed robot
with lots of control over its motion. Two Pololu 25mm DC motors motors with a 34:1
gear ratio will power the two drive wheels. These motors offer high pulling capacity for a
light robot at low speeds. The lower torque/RPM requirements allow for lower power
consumption of each motor. Shaft encoders are included on each motor as a form of
feedback to a software PID controller for very smooth driving capabilities.
Sensors
IR

Short range: https://www.sparkfun.com/products/8959
o Power: 3.1-5.5 DC
o Range: 3 – 30 cm (1.18 – 11.8 in)
o Output Voltage: .3 – 3.1 V
4
o Type: Analog
Sound
 Electret Microphones: https://www.sparkfun.com/products/8635
o Power: 5V max
o Sensitivity: -44dB
o Resistance: 680 Ohm
o Size: 9.77mm diameter
 Audio Op-Amp: LT1632
Bump Sensors
 Contact Switch: https://www.sparkfun.com/products/97
o Will be attached to mechanical arm
Motors/Drivers
 25mm D 34:1 Metal DC Gearmotor: http://www.pololu.com/product/2284/specs
o Power: 6V
o RPM: 165
o Free running currnent: 80mA
o Stall Current: 2A
o Torque: 40oz-in (2.9 kg-cm)
 MC33926 Motor Driver Carrier: http://www.pololu.com/product/1212
o Channels: 1
o Power(min): 5V
o Power(max): 28V
o Continuous Current: 2.5A
o Peak Current: 5A
o Current Sensing: .525 V/A
o Max PWM Frequency: 20kHz
o Min logic voltage: 2.5V
o Max logic voltage: 5.5V
o Reverse voltage protection
Boards
 Mattair Tech MT-X1S Dev Board:
http://www.mattairtech.com/index.php/development-boards/mt-x1satxmega128a1-u-usb-development-board.html
o ATxmega128a1u based
o 3.3V LDO (1A)
o USB – Serial Bridge
 LUFA
o 32.768kHz RTC
o 32MHz internal Clk
o many more features
 Raspberry Pi Model A: http://www.raspberrypi.org/faqs
o Power: 5V (1.2A)
o Clk: 700MHz
o GPU: openGL ES 2.0 compatible, OpenVG compatible
5
o Raspberry Pi Camera attached
Special Sensor
The special sensor is a 2nd order two channel analog audio filter. The filter is used
pick out a “whistle (f = 8.7k) from the ambient noise of the room. The circuit schematic
is shown below in Fig 3
Fig 3
The frequency response is shown in Fig 4.
Fig 4
Notice that the center frequency is around 8.7kHz with a 40dB gain in the pass band. A
graph of Vo vs Vin is shown in Fig 5.
6
Fig 5
Notice the small amount of clipping in the negative half cycle of the output wave,
this is due to the input signal attempting to drive the op-amp below the negative supply
rail of 0V so clipping occurs. Clipping is a form of distortion and is usually undesirable
in a audio system, but since the wave isn’t be reconstructed after processing, it should be
allowable for basic binaural detection.
Experimental test results showed that the circuit had a bandwidth of roughly
1kHz, a gain of around 50V/V, and a center frequency of 8.7kHz.
Due to the component tolerances used, the left channel had an observed center
frequency of 8.7kHz and the right center frequency was 8kHz.
Behaviors
There are four main states AROSOL operates in:
1. Wait for whistle (roam and obstacle avoid)
2. Track whistle location (proportional control from microphone data)
3. Hone whistle location (using camera/ color detection)
When the robot boots up it drives around avoiding obstacles and waiting for the
sound of the whistle. Once it has heard the whistle sound, it continues to obstacle
avoid while looking for the location of the whistle. Once the whistle location is
found and the bot is reasonably close, it turns on the camera and searches for the
master. The master can be identified as the sound source with a purple rectangle
around it. Once the master is found, AROSOL will continue its approach. Once
the bot is sufficiently close to the sound source. It will stop and go into the
mission complete state, signifying that the first run of sound localization is
7
complete. After the mission complete stage runs for a few seconds, the robot
enters sound state and restarts the process. A state diagram is shown below in Fig
6. Note: Obstacle avoidance occurs in all states.
Fig 6
Experimental Layout/Results
The first challenge was making use of the sound data. There were several
challenges along this road. The first was timing. The microcontroller needed to process
data from motor encoders, IR sensors, bump sensors, microphones and serial data all at
the same time. However, the priority this information is varied. An interrupt system was
implemented to achieve this goal. The PID algorithm was tuned and implemented using
motor encoder information that is sampled ten times per second. The audio samples were
polled the quickest of all at around 40kHz which satisfies the rate required by the
Shannon sampling theorem. To gather data, the ADC was used in differential mode, with
a 1.25V precision reference provided on board. The envelope detection was performed
and the envelope of the two signals is used to determine the location of the sound.
Amplitude based testing such as this is not as precise as time delay testing, but still was
accurate enough for the application. Some experimental data is shown in Fig 7. Note that
SumL, SumR, and Difference are 16 bit unsigned integer values from the Xmega’s ADC.
Ear
Left
Left
Left
Left
Right
Right
Angle(deg) Distance SumL
0
2 ft
532
571
782
45
2 ft
246
219
219
60
2 ft
143
123
140
90
2ft
81
85
83
0
2 ft
25
25
27
45
2 ft
29
32
27
SumR
34
43
38
41
41
42
42
43
42
41
42
43
601
592
566
112
108
96
Difference
498
528
754
202
178
177
101
80
98
40
43
40
576
567
539
83
76
69
8
Right
60
2 ft
Right
90
2 ft
27
27
27
30
29
28
Fig 7
108
108
109
67
59
41
81
81
82
37
30
13
The vision data was gathered using openCV. I used a custom built openCV library
written in C that uses a modified MMAL calls for the raspberry Pi camera to improve
speed. No benchmarking was performed, but there is a significant performance boost
over Python. Images were processed in the HSV color space because it allows for more
independence on different angles of reflectance, etc. However, it was found that in order
for the vision to be precise, a constant light source would need to be added to the top of
the robot. With the light source added, there were very few false positives and almost no
missed detections. The area of the detected image was then calculated and movement was
based on the threshold lines shown below.
PID tuning was performed in a debug state in which the robot acts under RC
control to allow for stopping quickly if the controller “blows up”. The gains were tuned
in the following method:
1. Set Kp (proportional control term) such that the robot oscillates in a
straight line path
2. Set Kd (derivative term) such that the oscillations are nulled
3. Set Ki (integral term) such that the robot reaches steady state quickly and
in a stable manner
9
Conclusion
AROSOL fully explored the challenges faced by intelligent sound localization
performed by a simple microcontroller. While it is possible, the microcontroller often will
need an external intelligence source such as a DSP or a high level ARM board with a
camera. Using the current hardware setup, sound localization could be improved by using
the Raspberry Pi for sound processing with precision ADCs and the floating-point math
optimizations available to the Pi. These improvements will be added in the future.
The Intelligent Machine Design Lab course at the University of Florida teaches
the art of systems integration as it applies to machine intelligence. Multiple systems had
to be interfaced to achieve AROSOL’s goal. These systems include: microcontrollers,
motors, motor drivers, sensors, microphones, high-level processors, and cameras.
Controlling this varied hardware required a lot of advanced software that was developed
using C/C++.
Documentation
o http://www.jp.hondari.com/upload/document/entry/20111011/Nakamura_2011_847_5250.pdf
[1]
o http://ieeexplore.ieee.org/xpl/login.jsp?tp=&arnumber=6094558&url=http
%3A%2F%2Fieeexplore.ieee.org%2Fxpls%2Fabs_all.jsp%3Farnumber%
3D6094558[2]
o Atxmega128A1U: http://www.atmel.com/devices/atxmega128a1u.aspx
o Raspberry Pi: http://www.raspberrypi.org/
o Code used/ borrowed
o Nick Cox’s IMDL xmega Code:
 https://github.com/nc61/robot_xmega/blob/master/robot_x
mega.h
o OpenCV example code:
 http://opencv-srf.blogspot.com/2010/09/object-detectionusing-color-seperation.html
o Custom Built OpenCV library:
 https://robidouille.wordpress.com/2013/10/19/raspberry-picamera-with-opencv/
o WiringPi Library:
 https://projects.drogon.net/raspberry-pi/wiringpi/
Appendices
My Github (containing the project’s code)
 https://github.com/seanmlyons22/IMDL-AROSOL
10