Software Development Infrastructure
for Sensor Networks
• Operating systems (TinyOS)
– Resource (device) management
– Basic primitives
– Protocols (MAC, routing) {covered in many previous lectures}
• Programming languages and runtime environments
– Low level (C, nesC)
– Very high level abstractions (Regiment)
• Services needed
– Synchronization of clocks (RBS)
– Propagation of new code to all nodes (Trickle)
– Localization {not today}
Fine-Grained Network
Time Synchronization
Using Reference Broadcasts
Jeremy Elson, Lew Girod, and Deborah Estrin
University of California, Los Angeles
OSDI 2002 - Boston, MA
Used with permission of author
The bigger picture
• Isn’t this a solved problem by now???
– NTP, many other clock agreement algorithms, MACs
with sync built in (802.11), time broadcasts (GPS,
WWVB), high-stability oscillators (Rubidium, Cesium)
BUT...
• If this isn’t the Internet:
– Important assumptions no longer hold
• (fewer resources -- such as energy, good connectivity,
infrastructure, size, and cost -- are available …)
– Sensor apps have stronger requirements
• (…but we have to do better than the Internet anyway)
A palette of sync methods
Goal: make the set rich enough to minimize waste
Time Sync Parameter Space:
(max error, lifetime, scope, etc.)
Available Sync
Methods
Better
Application Requirement
Better
A palette of sync methods
Goal: make the set rich enough to minimize waste
Time Sync Parameter Space:
(max error, lifetime, scope, etc.)
Ideally, methods
should be tunable
Better
Application Requirement
Better
Time Definitions
• Clock stability – how well it maintains a
constant frequency
– Short term – temperature
– Long term – aging of oscillator
• Clock accuracy – how well its frequency and
time compare with standard
• Offset – time difference between 2 clocks
• Skew – frequency difference between 2
clocks
Traditional sync
Problem: Many sources of unknown, nondeterministic
latency between timestamp and its reception
Sender
Receiver
Send time
At the
tone: t=1
NIC
Receive Time
NIC
Access Time
Propagation Time
Physical Media
Reference Broadcasts
Sync 2 receivers with each other, NOT sender with receiver
Sender
Receiver
Receiver
Receive Time
NIC
Propagation Time
I saw it
at t=4
NIC
NIC
I saw it
at t=5
Physical Media
RBS reduces error by
removing much of it from the critical path
NIC
NIC
Sender
Sender
Receiver
Receiver 1
Critical Path
Receiver 2
Time
Critical Path
Traditional critical path:
From the time the sender
reads its clock, to when the
receiver reads its clock
RBS: Only sensitive to the
differences in receive time
and propagation delay
Receiver Determinism
1st testbed: Berkeley motes with
narrowband (19.2K) radios
Appears Guassian
Well-Behaved = Good
• Well behaved distributions are useful
– Error can be reduced statistically, by sending
multiple pulses over time and averaging
– Also, easier to model/simulate
Ignoring clock skew
• Server broadcasts m reference beacons
• Each of n receivers records local time
of each m refs
• Exchange:
m
Offset[i,j] = 1/m S (Tj,k – Ti,k)
k=1
Problem: Clock Skew
• It takes time to send multiple pulses
• By the time we do, clocks will have drifted
• So: don’t average; fit a line instead
Frequency & phase
of my clock wrt yours recovered from slope
and intercept
Time
Comparison to NTP
• Second implementation:
– Compaq IPAQs (small Linux machines)
– 11mbit 802.11 PCMCIA cards
• Ran NTP, RBS-Userspace, RBS-Kernel
– NTP synced to GPS clock every 16 secs
– NTP with phase correction, too; it did worse (!)
• In each case, asked 2 IPAQs to raise a GPIO
line high at the same time; differences
measured with logic analyzer
How NTP works
Peer 1
Filter 1
Peer 2
Filter 2
Peer 3
Filter 3
NTP Messages
Intersection
and
Clustering
Algorithms
Timestamps
Combining
Algorithm
Loop Filter
P/F-Lock Loop
VFO
• Multiple synchronization peers provide redundancy and diversity
• Clock filters select best from a window of eight clock offset samples
• Intersection and clustering algorithms pick best subset of servers
believed to be accurate and fault-free
• Combining algorithm computes weighted average of offsets for best
accuracy
• Phase/frequency-lock feedback loop disciplines local clock time and
frequency to maximize accuracy and stability
© Mills
Clock Resolution
Clock Resolution
RBS degraded slightly (6us to 8us); NTP degraded severely (51us to 1542us)
Multi-Hop RBS
• Some nodes broadcast RF synchronization pulses
• Receivers in a neighborhood are synced by using the pulse as
a time reference. (The pulse senders are not synced.)
• Nodes that hear both can relate the time bases to each other
“Red pulse 2 sec
after blue pulse!”
“Here 1 sec after blue pulse!”
“Here 0 sec after
blue pulse!”
“Here 3 sec after
red pulse!”
“Here 1 sec after
red pulse!”
Time Routing
Consider a physical topology consisting of
broadcasters (A, B, C..) and
receivers (1, 2, 3…)
1
2
5
A
3
B
4
6
7
C
8
9
D
10
11
(In reality, a node can play both roles)
Time Routing
The physical topology can be easily
converted to a logical topology; links
represent possible clock conversions
1
2
5
A
3
1
B
4
7
2
5
6
6
3
4
7
8
9
10
11
C
8
9
D
10
11
Use shortest path search to find a “time route”;
Edges can be weighted by error estimates
Multi-Hop RBS
Error (and std dev) over multiple hops, in usec
3.68 +/- 2.57
7
2.73 +/- 2.42
6
Error (usec)
2.73 +/- 1.91
5
1.85 +/- 1.28
4
Std Dev
Error
3
2
1
0
1 Hop
2 Hop
3 Hop
4 Hop
Summary
• RBS can improve accuracy by removing
sender from the critical path
• Multi-hop algorithm can extend RBS
property across broadcast domains, and to
external standards such as UTC
• Implemented on 4 different CPU/radio
platforms; no MAC tinkering required
• Facilitates post-facto sync (save energy by
only syncing after an event of interest) and
peer to peer sync (no global timescale)
Applications
• Acoustic Ranging
• Collaborative Signal Detection
• etc...
• Future work:
– Use higher precision clocks (e.g. Pentium TSC)
– Better outlier rejection, weighting
Software Development Infrastructure
for Sensor Networks
• Operating systems (TinyOS)
– Resource (device) management
– Basic primitives
– Protocols (MAC, routing) {covered in many previous lectures}
• Programming languages and runtime environments
– Low level (C, nesC)
– Very high level abstractions (Regiment)
• Services needed
– Synchronization of clocks (RBS)
– Propagation of new code to all nodes (Trickle)
– Localization {not today}