INTERNATIONAL JOURNAL OF NETWORK MANAGEMENT
Int. J. Network Mgmt 2016; 26:355–372
Published online 28 July 2016 in Wiley Online Library (wileyonlinelibrary.com). DOI: 10.1002/nem.1942
R EVERSE PTP: A clock synchronization scheme for
software-defined networks
Tal Mizrahi, and Yoram Moses
Department of Electrical Engineering, Technion — Israel Institute of Technology, Haifa, Israel
SUMMARY
We introduce R EVERSE PTP, a clock synchronization scheme for software-defined networks (SDNs).
R EVERSE PTP is based on the Precision Time Protocol (PTP), but is conceptually reversed; in
R EVERSE PTP, all nodes (switches) in the network distribute timing information to a single software-based
central node (the SDN controller), which tracks the state of all the clocks in the network. Hence, all computations and bookkeeping are performed by the central node, whereas the ‘dumb’ switches are only required
to periodically send it their current time. In accordance with the SDN paradigm, the ‘brain’ is implemented
in software, making R EVERSE PTP flexible and programmable from an SDN programmer’s perspective. We
present the R EVERSE PTP architecture and discuss how it can be used in typical SDN architectures. Our
experimental evaluation of a network with 34 R EVERSE PTP-enabled nodes demonstrates the effectiveness
and scalability of using R EVERSE PTP. Copyright © 2016 John Wiley & Sons, Ltd.
Received 31 August 2015; Revised 18 February 2016; Accepted 24 May 2016
1. INTRODUCTION
1.1. Background
Software-defined networking (SDN) is an architecture in which a network is managed by a centralized
controller. The controller provides an application programming interface that allows SDN programmers to manage the network using a high-level programming language. The SDN approach defines a
clear distinction between the data plane and the control plane; on the data plane, forwarding decisions
is taken locally by each switch in the network, while the control plane is managed by a centralized
entity called the controller, overcoming the need for complicated distributed control protocols and
providing the network operator with powerful and efficient tools to manage the data plane.
In recent years, as accurate time synchronization has become an accessible and affordable
tool, it is used in various different applications; mobile backhaul networks use clock synchronization protocols to synchronize mobile base stations [1]. Google’s Spanner [2] uses
synchronized clocks as a tool for synchronizing a distributed database. Industrial automation
systems [3] use synchronized clocks to allow deterministic response times of machines to external events and to enforce coordinated orchestration in factory product lines. The Time-sensitive
Networking technology [4] is used in automotive networks and in audio/video networks. The
well-known nuclear research labs at CERN use state-of-the-art synchronization [5], allowing
sub-nanosecond accuracy in response to management messages that control the experiments.
Correspondence to: Tal Mizrahi, Department of Electrical Engineering, Technion — Israel Institute of Technology,
Haifa, Israel.
E-mail: [email protected]
Yoram Moses is the Israel Pollak academic chair at Technion.
Copyright © 2016 John Wiley & Sons, Ltd.
356
T. MIZRAHI AND Y. MOSES
The rise of SDN raises two interesting use cases for accurate clock synchronization:
Distributing time over SDN-enabled networks. Accurate time is required in various environments, in which SDN is being considered (e.g., [6]). Hence, an SDN can be used as a vessel for
distributing accurate time between end points. Notably, accurate synchronization between end
points requires the SDN-enabled network equipment to take part in the synchronization protocol.
Timed network operations. A recently introduced approach [7–9] suggests the usage of accurate
time to coordinate network updates in an OpenFlow-enabled network in a simple and scalable
manner, reducing packet loss and anomalies during configuration or topology changes. Accurate
time can be a useful tool in various scenarios in the dynamic SDN setting, for example, coordinated topology changes, resource allocation updates, and accurate timestamping for event logs
and statistics collection. Our previous work in this context includes an extension [10] to the OpenFlow [11,12] protocol that enables timed operations using OpenFlow. This new feature has been
approved by the Open Networking Foundation and integrated into OpenFlow 1.5 [12].
A network that either distributes time between its end points or makes use of timed operations
requires a clock synchronization protocol. The Precision Time Protocol (PTP), defined in the IEEE
1588–2008 standard [13], is a natural candidate, as it can provide a very high degree of accuracy,
typically on the order of 1 microsecond (e.g., [14]) or less, and is widely supported in switch silicons.
The challenge of using a standard synchronization protocol such as PTP in an SDN environment lies
in the fundamental difference between these two technologies. A key property of SDN is its centralized
control plane, whereas PTP is a decentralized control protocol; a master clock is elected by the Best
Master Clock Algorithm (BMCA), and each of the slaves runs a complex clock servo algorithm that
continuously computes the accurate time based on the protocol messages received from the master
clock. Thus, if SDN switches function as PTP slaves, then in contrast to the SDN philosophy, they
are required to run complex algorithmic functionality and to exchange control messages with other
switches. Indeed, a hybrid [12] approach can be taken, where the SDN operates alongside traditional
control-plane protocols such as PTP. Our approach is to adapt PTP to the SDN philosophy by shifting
the core of its functionality to the controller.
1.2. R EVERSE PTP in a nutshell
In this paper, we introduce R EVERSE PTP, a novel approach that addresses the previous challenge;
in contrast to the conventional PTP paradigm, where a master clock distributes its time to multiple
slave clocks (Figure 1(a)), in R EVERSE PTP (Figure 1(b)), there is a single node (the SDN controller)
that runs multiple instances of a PTP slave and multiple masters (the switches). Every switch runs a
separate instance of PTP with the controller, where the switch is the master and the controller is the
Figure 1. Time distribution in PTP and R EVERSE PTP.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
357
slave. Hence, every switch distributes its time to the controller. The controller keeps track of the offset,
skew, and drift of each of the switches’ clocks with respect to the controller’s clock. This relieves the
switches of any complicated computations and does not require message exchange between switches,
as all protocol messages are exchanged with the controller.
The significant advantage of R EVERSE PTP is that the complex algorithmic functionality that
acquires the accurate time using the information received in the PTP messages is implemented in the
controller. This algorithmic logic can be modified or reprogrammed at the controller without upgrading
switches in the network or can be dynamically tuned and adapted to the topology and behavior of the
network based on a network-wide perspective. Moreover, an operator that uses switches from different vendors may experience inconsistent behavior when using conventional PTP, as different switches
may use different clock servo algorithms, while R EVERSE PTP guarantees uniform behavior as all the
algorithmic logic runs on the controller.
Notably, the main difference between PTP and R EVERSE PTP is the direction of time distribution;
in R EVERSE PTP, time is distributed in an all-to-one fashion in contrast to PTP’s one-to-all nature.
Hence, the accuracy of conventional PTP and R EVERSE PTP should be similar, given that all other
aspects of the network are the same.
R EVERSE PTP is defined as an IEEE 1588 profile.1 Because switches function as conventional PTP
masters, our approach is applicable to existing implementations of PTP-enabled switches. We note
that the R EVERSE PTP approach can be applied to other synchronization protocols, for example, the
Network Time Protocol [15].
In a precise sense, R EVERSE PTP is not a clock synchronization protocol. Instead, R EVERSE PTP
is a tool for coordinating synchronized actions in an SDN environment. All the timing information is
maintained in the ‘brain’ of the network, the controller, whereas the ‘dumb’ SDN switches do not need
to be synchronized.
1.3. Related work
In preliminary versions of this paper [16,17], we introduced R EVERSE PTP and its main principles. The
current paper describes R EVERSE PTP in detail and includes detailed experimental evaluation results.
A topic that has been thoroughly studied in the literature is software-based implementation of accurate clock synchronization [18,19], not to be confused with SDN, which is the network architecture
that the current paper focuses on. The PTIDES project [20,21] defines a programming model for
time-aware programs. The current paper presents R EVERSE PTP and focuses on how time-aware SDN
applications can use it. The programming model is beyond the scope of this paper.
In conventional synchronization protocols, multiple time sources are sometimes used to improve the
accuracy and security of the protocol [15], or for redundancy, allowing fast recovery when the primary
master fails [22]. Contrary to conventional synchronization protocols, R EVERSE PTP is not used for
clock synchronization, but for many-to-one time distribution, and for coordinating actions at different
sites in a timely manner.
1.4. Contributions
The main contributions of this paper are:
We introduce R EVERSE PTP. To the best of our knowledge, our work is the first to present a clock
synchronization scheme for SDN.
We show that R EVERSE PTP can be defined as a PTP profile, that is, a subset of the features of
PTP. Consequently, R EVERSE PTP can be implemented by existing PTP-enabled switches.
We show that R EVERSE PTP is applicable to two main scenarios: (i) an SDN that uses time to
coordinate configuration updates and notifications and (ii) an SDN that distributes accurate time
between its end points or attached networks.
We present experimental results that analyze and demonstrate the accuracy and scalability of
R EVERSE PTP and show that they are comparable with those of PTP.
1
A profile [13] is a specific selection of features and modes of PTP.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
358
T. MIZRAHI AND Y. MOSES
2. PRELIMINARIES
2.1. A brief overview of Precision Time Protocol
Precision Time Protocol is a protocol that allows the distribution of time and frequency over packetswitched networks. Three types of nodes are defined in PTP (Figure 2(a)); ordinary clocks (OCs),
transparent clocks (TCs), and boundary clocks (BCs). TCs and BCs are either switches or routers,
whereas OCs are typically end points. An OC can either be a master or a slave. A master distributes
information about its local time to slave clocks in the network using PTP messages.
The BMCA is used to elect the most accurate clock in the network as the master. A set of PTP
clocks that synchronize to a common master forms a PTP domain. A network may consist of several
domains, where each domain is dominated by a different master.
A master periodically sends Sync messages to its slaves, incorporating information about its current
time. Delay Request and Delay Response messages (Figure 2(b)) are used to measure the network delay
between the master and slave. At the end of this message exchange, the slave has four time values, T1 ,
T2 , T3 , and T4 , as illustrated in Figure 2(b), allowing it to compute the offset, o, between its clock and
the master’s clock as follows:
o D T2 T1 dM S
(1)
The parameter dM S is the delay between the master and the slave and is given by
.T4 T1 / .T3 T2 /
(2)
2
Sync and Delay messages are transmitted periodically, allowing slaves to synchronize to the master’s clock based on multiple measurements of the o and dM S values. Each slave typically uses a
servo algorithm, which filters and combines multiple measurements of equations (1) and (2), and
synchronizes the slave’s clock.
The usage of TCs and BCs in PTP is referred to as on-path support. A BC is a node that functions
as a slave on one of its ports and as a master on its other ports. TCs are simple intermediate nodes;
they are neither masters nor slaves. Their role is to relay PTP protocol messages between the master
and slaves and to compute a correction field for each message. The correction field represents the
aggregated internal delay of all the TCs on the path between the master and slave; when a TC relays
a protocol message, it measures the delay of the message from reception to transmission and adds the
value of this delay to the correction field. The slave can then use the correction field to eliminate the
non-deterministic delays that result from the TCs’ transmission queues. Because TCs only compute
the difference between the time of transmission and the time of reception, they are not required to
synchronize their clocks to the master’s time. A syntonized TC is a TC that is frequency-synchronized
to the master’s clock, allowing a more accurate correction field than when using a non-syntonized TC.
dM S D
Figure 2. The Precision Time Protocol (PTP).
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
359
Figure 3. A protocol for coordinated network updates
Precision Time Protocol supports two timestamping modes: one-step mode, where each Sync message includes its time of transmission, T1 , and two-step mode, where a Sync message is sent without a
timestamp, followed by a Follow-Up message that incorporates the timestamp T1 .
2.2. A model for using time in software-defined network
As is standard in the literature (e.g., [23,24]), we distinguish between real time, which is not observable
by nodes in our system, and clock time, as measured by the nodes. Real-time values are denoted in
lowercase, whereas clock time variables and constants in upper case. We assume that each node in our
system maintains a clock. As in [24,25], the value of a clock T .t/ at time t is assumed to be a quadratic
function of t, as follows:
T .t/ D t C o.t0 / C ¡.t0 / Œt t0 C 0:5 d Œt t0 2
(3)
Here, t0 is some previous reference time value, o.t0 / D T .t0 / t0 is the offset at time t0 , the clock
skew at t0 is denoted by ¡.t0 /, also known as the frequency error, and d is the drift, which is the first
derivative of the skew. As in [24,25], we assume that the drift, d , is constant.
Our system consists of n C 1 nodes: a controller c and a set S of n switches.2 We define two possible
timed operations:
Timed notification: We define a set B of notifications that a switch can send to the controller. A
notification sent to the controller indicates an event that occurred at the switch and is accompanied
by a timestamp, indicating when the event occurred.
A switch can send a notification message of the form Msc .i; ˇ; T i / to the controller, denoting
that the message includes a notification ˇ 2 B, and a timestamp T i in terms of i’s clock.
Timed command: We define a set A of possible commands that the controller can send
to switches. The controller can send a timed command message to switch i, denoted by
Mcs .i; ˛; T i /, implying that i should perform ˛ 2 A at time T i in terms of i’s clock.
A controller can use timed commands to invoke a coordinated action at multiple switches. T IME C ONF [7], formally defined in Figure 3, is a simple protocol for time-based updates, where the
controller defines a single execution time, Te , for all switches,3 and AM WD ¹˛1 ; ˛2 ; : : : ; ˛n º is a set
of n commands, such that the controller assigns the action ˛i to switch i for each i 2 S.
3. R EVERSE PTP: THEORY OF OPERATION
As described in Section 1.2, in R EVERSE PTP, a central node (the SDN controller) is the ‘brain’ of
the network and performs two main tasks: (i) running the PTP protocol as a slave separately with each
of the SDN switches (which function as PTP masters) and (ii) performing the necessary translation
between the masters’ times and the slave’s time. Each of these tasks is discussed in detail next.
Running the PTP protocol. The R EVERSE PTP slave is bound (concurrently and independently)
to n R EVERSE PTP masters. The slave exchanges PTP messages with each of the masters and, based
on these messages, maintains four parameters for each master i:
2
Throughout the paper, we assume that the controller is also the R EVERSE PTP slave.
The controller should take care to schedule an execution time that allows enough time for the update message to be sent
and propagated to all switches in the network.
3
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
360
T. MIZRAHI AND Y. MOSES
Table 1. R EVERSE PTP slave parameters
Tislast
The time at which the latest Sync message from master i was received. The superscript ‘s’
indicates that this timestamp is measured in terms of the slave’s clock.
The estimated offset between the clocks of master i and the slave at time Tislast .
The estimated skew between master i and the slave at Tislast .
The estimated drift between master i and the slave.
oO i
¡O i
dO i
The offset oO i , the skew ¡O i , and the drift dO i are computed by the slave based on the latest measurement of Tislast , as well as on previous measurements. Various well-known algorithms can be used for
computing these two parameters, for example, [19,25].
Time translations. The parameters of Table 1 allow the slave to translate any timestamp T s in terms
of the slave’s clock to a timestamp T i .T s / in terms of master i’s clock,4 or vice versa.
Notably, every timed operation that can be performed in a clock-synchronized network is also
possible in a R EVERSE PTP-enabled network:
Given a timed notification received from master i with a timestamp T i , the slave translates T i
to T s .T i / in terms of its local clock.
When the slave invokes a timed command that needs to take place at T s in terms of the slave’s
clock, the slave translates T s to T i .T s / and uses T i .T s / in the timed command message it sends
to master i.
The following theorem presents the translation of T i .T s / and T s .T i /, given the estimated
parameters of Table 1.
Theorem 1: Let tislast be the time at which the value of the slave’s clock is Tislast , and let oi tislast ,
¡i tislast , and di be the offset, skew, and drift at time tislast . If oi tislast D oO i , ¡i tislast D ¡O i , and
di D dO i , then
2
(i) T i .T s / D T s C oO i C ¡O i T s Tislast C 0:5 dO i T s Tislast and
s
i
(ii) T .T / D
Tislast
C
r
.1CO¡i /C .¡O i C1/2 C2dO i T i Tis
last
Ooi
dO i
Proof. We first prove (i). Because oi tislast D oO i , ¡i tislast D ¡O i , and di D dO i , by equation (3), we
obtain (i).
We now prove (ii). First, we observe that when the drift dO i is negligible, by (i), we have
limdO i !0 T i D T s C oO i C ¡O i T s Tislast . It follows that
lim T s D
dO i !0
T i oO i C ¡O i Tislast
(4)
1 C ¡O i
Now, based on (i), we have
T i D T s .1 C ¡O i / C oO i ¡O i Tislast C 0:5dO i T s 2 2T s Tislast C Tislast 2
4
We use the notation T i .T s /, referring to the value of master i’s clock at the time instant when the value of the slave
s’s clock is T s .
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
361
Reorganizing the terms, we obtain
0:5dO i T s 2 C 1 C ¡O i dO i Tislast T s C oO i ¡O i Tislast C 0:5 dO i Tislast 2 T i D 0
We isolate T s by solving the aforementioned quadratic equation and obtain
1 r
2
1 O
s
T D
di Ti last ¡O i 1 ˙
dO i Tislast ¡O i 1 2dO i oO i ¡O i Tislast C 0:5dO i Tislast 2 T i ;
dO i
dO i
s
and thus
r
1 O s
1
.O¡i C 1/2 C 2dO i T i Tislast oO i
T D
di Ti last ¡O i 1 ˙
dO i
dO i
s
(5)
The ‘˙’ implies that there are two possible solutions. However, only one of these solutions is valid.
We show this by observing the limit of the latter equation when dO i ! 0.
lim T s D lim
dO i !0
q
dO i Tislast ¡O i 1 ˙ .O¡i C 1/2 C 2dO i .T i Tislast oO i /
dO i
dO i !0
Now, by applying L’Hôpital’s rule to the latter, we obtain
r
0
Odi T s ¡O i 1 ˙ .O¡i C 1/2 C 2dO i T i T s oO i
i last
i last
lim T s D lim
0
dO i !0
dO i !0
dO i
0:5 2 T i Tislast oO i
D lim Tislast ˙ r
;
dO i !0
s
2
i
O
.O¡i C 1/ C 2di T Ti last oO i
and thus
lim T s D
dO i !0
Tislast .1 C ¡O i / ˙ T i Tislast oO i
1 C ¡O i
(6)
By comparing equations (6) and (4), we conclude that from the two ‘˙’ solutions, only the ‘C’
solution is valid, and thus, by equation (5), we have
T s .T i / D Tislast C
r
.1 C ¡O i / C .O¡i C 1/2 C 2dO i T i Tislast oO i
(7)
dO i
Intuitively, Theorem 1 states that if the slave’s estimated parameters are accurate, then T i .T s / is
given by (i), and T s .T i / is given by (ii). Notably, Corollaries 2 and 3, which follow directly from the
theorem, provide a practical method for the slave to compute master i’s clock value at a given time T s
or to compute the slave’s clock value for a given value T i of master i’s clock.
Corollary 2: For any time T s measured in terms of the R EVERSE PTP slave’s clock, the slave can
estimate the corresponding value of master i’s clock by
2
TO i .T s / D T s C oO i C ¡O i T s Tislast C 0:5 dO i T s Tislast
Copyright © 2016 John Wiley & Sons, Ltd.
(8)
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
362
T. MIZRAHI AND Y. MOSES
Table 2. R EVERSE PTP profile properties
Property
Behavior in R EVERSE PTP
Best Master Clock Algorithm
Path delay mechanism
Transport protocol
One-step vs. two-step
On-path support
Multicast vs. unicast
Domains
Management messages
none (not used)
End-to-end (E2E)
UDP over IPv4 or UDP over IPv6
Both one-step and two-step modes are supported
On-path support is optional, using transparent clocks
Sync: either unicast or multicast.
Delay_Req and Delay_Resp: always unicast.
Each master-slave pair forms a different domain.
The domain number is the same for all clocks, as in [22]
Management messages are optional
Corollary 3: For any time T i in terms of master i’s clock, the slave can estimate the corresponding
value of its clock by
r
.1 C ¡O i / C .O¡i C 1/2 C 2dO i T i Tislast oO i
(9)
TO s .T i / D Tislast C
dO i
A first-order approximation. Interestingly, it is possible to use the following first-order approxi
2
mation of equation (8), neglecting ¡O i T s T s
C 0:5 dO i T s T s
:
i last
i last
TO i .T s / D T s C oO i
(10)
This approximation is especially useful in cases where T s Tislast is negligible, that is, when the
timed operation occurs close to the time at which the last Sync message was received.
4. THE R EVERSE PTP PROFILE
In this section, we show that R EVERSE PTP can be defined as a PTP profile, that is, as a subset of the
features defined in the IEEE 1588 [13] standard. Significantly, because R EVERSE PTP is defined as a
subset of PTP, it follows that R EVERSE PTP can be implemented by existing PTP-enabled switches.
The R EVERSE PTP profile has two interesting properties with respect to SDN: (i) to the extent possible, it requires very simple functionality from the switches and (ii) all PTP messages are exchanged
between a controller and a switch, and thus, no messages are exchanged between switches.
As per IEEE 1588 [13], the selected modes of operation of the R EVERSE PTP profile are specified
in Table 2. We now briefly describe the set of attributes and modes that define this profile.
PTP domains. A set of PTP clocks that synchronize to a common master forms a PTP domain. In
R EVERSE PTP, every switch forms a master–slave pair with the controller, in which the switch serves
as master and the controller as slave; in particular, each master–slave pair forms a separate PTP domain
(Figure 4). When a slave receives a PTP message, it identifies the packet’s domain based on its source
address. Thus, as in [22], all domains can use the same domain number.5
On-path support. A PTP-enabled network may use TCs or BCs, which are intermediate switches
or routers that take part in the protocol, allowing high end-to-end accuracy. The usage of TCs or BCs
is referred to as on-path support. On-path support in R EVERSE PTP is optional. TCs may be used,
allowing to improve the accuracy of the protocol using the PTP correction field [13]. This implies that
a R EVERSE PTP-enabled switch may function both as a master, distributing its time to the slave, and
5
A less scalable solution to identify the packet’s domain is by using the domain number field in the PTP header. Because
this field is 8-bits long, this solution would limit the number of R EVERSE PTP masters to 256.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
363
Figure 4. R EVERSE PTP: each master determines a separate domain
as a TC, which relays PTP messages from other OCs (Figure 4(b)). For simplicity, BCs are not used in
R EVERSE PTP, footnoteNote that an SDN can function as a ‘big BC’, as described in Section 5.3, but
no BCs are used in R EVERSE PTP domains, because a BC must, by definition, keep a synchronized
clock and run a clock servo algorithm.
It is interesting to note that if TCs are used, they can be either syntonized [13] or non-syntonized. A
syntonized TC is a TC that is frequency-synchronized to the master’s clock, allowing a more accurate
correction field than a non-syntonized TC. Our centralized paradigm requires TCs to be as simple as
possible and hence non-syntonized; a non-syntonized TC is simply required to compute the residence
time of en route PTP messages and is thus not required to run a complex servo algorithm. Moreover, TCs are not required to exchange PTP messages; a TC only udpates the correction field of en
route messages, indicating the internal latency of each message as it is transmitted through the TC. In
Section 7.3, we briefly discuss how R EVERSE PTP can be extended to allow syntonized TCs.
BMCA. PTP uses the BMCA to choose a master. In contrast, R EVERSE PTP does not use the
BMCA; during the network initialization, all switches are configured as masters, and the controller
is configured as a slave. This approach is aligned with the SDN paradigm, where a bootstrapping procedure (e.g., [26]) is typically used for configuring basic attributes, such as the controllers’
IP addresses.
Unicast and multicast transmission. Sync messages in R EVERSE PTP can be sent either as unicast or as multicast. During network initialization, switches need to be configured with the controller’s
unicast address. If multiple SDN controllers are used, Sync messages are distributed to a multicast
address that represents the controllers. The operation of R EVERSE PTP in SDNs with multiple controllers is further discussed in Section 7.4. Delay Request and Delay Response messages are always
sent as unicast.
One-step vs. two-step. Both one-step and two-step modes can be used in R EVERSE PTP.
Peer delay mechanism. The delay measurement mechanism in PTP has two possible modes of
operation: end-to-end mode, where delay request and response messages are exchanged between the
master and slave, and peer-to-peer mode, where intermediate TCs perform delay measurement on a
one-hop basis. While the two modes can provide the same level of accuracy, peer-to-peer mode requires
PTP messages to be exchanged between TCs. Hence, R EVERSE PTP uses the end-to-end mode, as this
paradigm implies that all delay messages are exchanged between a controller and a switch, and no
messages need be exchanged between switches.
5. USING R EVERSE PTP IN SOFTWARE-DEFINED NETWORKS
5.1. The R EVERSE PTP architecture in software-defined network
A typical SDN architecture is illustrated in Figure 5(a). The network operating system is a logical entity
that manages the control plane of the network and communicates with switches using an SDN protocol
such as OpenFlow [12]. The controller may run one or more SDN Application, a module that performs
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
364
T. MIZRAHI AND Y. MOSES
Figure 5. The R EVERSE PTP architecture in SDNs: every switch runs a R EVERSE PTP master, and
the controller runs multiple R EVERSE PTP slave instances. In an SDN that runs conventional PTP, a
typical approach would be for the controller to run a PTP master and for each switch to be a PTP slave.
a network function such as routing or access control, using the SDN control plane. Every switch uses
one or more flow tables, which determine the switch’s data plane decisions, such as forwarding and
security decisions. A switch’s control plane agent is responsible for managing the flow tables based
on commands received from the controller.
We now describe an SDN architecture that uses R EVERSE PTP, as illustrated in Figure 5(b). The
logical blocks in the figure are described as follows:
Clocks. Every switch maintains a clock, which keeps the local wall-clock time and allows the switch
to perform time-triggered actions and to timestamp notifications. R EVERSE PTP does not require
the switches’ clocks to be synchronized or initialized to a common time reference. The controller
maintains a local clock. The controller’s clock is used as the reference for scheduling network-wide
coordinated updates and for measuring timestamped events. Thus, in some systems, the controller’s
clock may be required to be synchronized to an accurate external reference source such as a Global
Positioning System receiver.
R EVERSE PTP master. Each switch functions as a PTP master and periodically sends Sync messages to the controller (its PTP slave), containing a local timestamp. We emphasize that the PTP master
functionality is typically supported by existing implementations of PTP-enabled switches.
R EVERSE PTP slave. The controller maintains n R EVERSE PTP slave modules, where n is the
number of switches in the network. Each slave module periodically receives Sync messages from one
of the switches (its masters), i, and based on these messages, it maintains the offset, skew, and drift
of i w.r.t. the controller’s clock (Table 1).
Timestamp conversion. This module performs the required translation between the controller’s
clock time and the switches’ clock time, as described in Section 3; a timestamp T s in terms of the controller’s clock is translated into T i in terms of i’s clock using equation (8). Similarly, a notification from
switch i that contains a timestamp Ti can be converted to the controller’s timescale by equation (9).
It is a key observation that the timestamp conversion module allows SDN applications that run at
the controller to implement any time-based protocol that would require switches to be synchronized
using a conventional synchronization protocol. This interesting property applies generally to SDN
applications; coordination is not required directly between switches, but only through the controller.
We now describe two interesting examples of SDN applications that use R EVERSE PTP: the timebased update application and the time distribution application.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
365
Figure 6. Software-defined network as a boundary clock
5.2. Time-based updates using R EVERSE PTP
Time-based update application. This simple SDN application performs time-based network updates
using the T IME C ONF protocol (Figure 3). When the application sends a time-based update, denoted
by Mcs .i; ˛i ; Te /, on line 2 of Figure 3, the time conversion module translates Te to a time T i in the
domain of i’s clock.
Conceptually, the joint operation of the time-based update application and the timestamp conversion
block performs the following protocol:
As in equation (10), a first-order approximation of R EVERSE T IME C ONF where line 2 is replaced by
2
i
and 0:5 dO i Te T s
T
Te C oO i can be used when the higher order terms, ¡O i Te T s
,
i last
i last
are negligible. Switch scheduling. When switch i receives a scheduled message, Mcs .i; ˛; T i /, from
the controller, this module schedules the command ˛ to time T i in terms of the switch’s local clock.
5.3. Time distribution over software-defined networks using R EVERSE PTP
In some cases, time must be distributed between end stations or networks that are attached to an
SDN. For instance, an SDN-based mobile backhaul network must allow time distribution between
base station sites, enabling the base stations to be synchronized. In this section, we present an SDN
application, denoted by time dist. app in Figure 5(b), that allows time distribution over an SDN.
In conventional PTP-enabled networks, time is distributed over one or more PTP BCs [13], as shown
in Figure 6(a). A BC is a switch or a router that maintains an accurate clock based on Sync messages
that it receives from the PTP master and distributes its time to the PTP slaves. When a BC receives a
Sync message6 from the master (step 1 in Figure 6(a)), its ingress time is accurately measured. Based
on the Sync message and its ingress timestamp, the BC adjusts its clock. When the BC generates a Sync
message to one of the slaves, the message is accurately timestamped when it is transmitted through the
egress port (step 2 in Figure 6(a)).
Our approach is illustrated in Figure 6(b); R EVERSE PTP is used within the SDN, allowing the controller to maintain the time offset to each of the switches. An SDN is often viewed as a single ‘big
switch’. Similarly, in our approach, the SDN is a distributed BC that functions as a single logical
‘big BC’. When the master sends a Sync message, switch 1 accurately measures its ingress time, T 1
(step 1 in Figure 6(b)) and sends the packet and T 1 to the controller for further processing. The controller converts T 1 to T s using the timestamp conversion module, and the time dist. app (Figure 5(b))
adjusts the controller’s clock based on the incoming Sync message and T s . When the time dist. app
sends a Sync message through switch 2, it uses the packet’s correction field [13] to reflect the offset
6
The Sync message procedure is presented as an example. The procedure for other types of PTP messages is similar.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
366
T. MIZRAHI AND Y. MOSES
Figure 7. Network setup
o2 between switch 2 and the controller, and the packet is timestamped by switch 2 as it is transmitted
(step 2 in Figure 6(b)). This procedure can be implemented in OpenFlow [12], using Packet-in and
Packet-out messages between the controller and the switches. Note that there is currently no standard
means for the ingress port of switch 1 to convey T 1 to the controller. A similar problem has been
raised in the IEEE 1588 working group (e.g., [27]), and proposals that address it are currently under
discussion there.
The significant advantage of the ‘big BC’ approach compared with the conventional PTP approach
is that it enables the programmability of R EVERSE PTP, while presenting standard PTP behavior to
external non-SDN nodes.
6. EVALUATION
We have implemented a prototype of R EVERSE PTP, based on the open-source Precision Time
Protocol daemon (PTPd) [28], and evaluated its performance in a test bed with 34 nodes (Figure 7).
The goals of our experiments were (i) to evaluate the effectiveness of scheduling a simultaneous
event in the network using R EVERSE PTP, (ii) to verify that R EVERSE PTP and conventional PTP
provide a similar degree of accuracy under the same network conditions and, specifically, to verify
that R EVERSE PTP can schedule events with a high degree of accuracy using the approximation of
equation (10), and (iii) to analyze the scalability of R EVERSE PTP.
The experiments were conducted on the DeterLab test bed [29], where every test bed machine
(computer) served as a PTP clock running the software-based PTPd. Note that our experiments used
software-based PTP clocks in a network with up to two hops without on-path support, and we observed
that the achievable accuracy in this software-based environment was on the order of tens to hundreds
of microseconds. 7
Accuracy measurement method. The accuracy of a set of clocks is typically measured in a controlled environment by connecting identical fiber cables from each of the clocks to a measurement
device; each of the clocks transmits a 1 pulse per second signal to the measurement device, allowing
the device to measure the offset between the clocks. Such a setting was not possible in the DeterLab test bed. Therefore, we measured the accuracy by using two types of time-triggered events, as
described in the next subsection.
6.1. Time-triggered events
This experiment included two scenarios: performing a time-triggered coordinated event and recording
a timestamped event. Both scenarios were implemented using R EVERSE PTP, and we compare the
results with an implementation of the same scenarios using PTP.
7
Typical hardware-based PTP implementations allow an accuracy on the order of 1 microsecond.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
367
Figure 8. Accuracy measurements of a coordinated Ping. The timestamped event experiment (b)
provides a rough estimate of the clock accuracy.
R EVERSE PTP masters were implemented by running PTPd in master-only mode, with the BMCA
disabled. Our n-slave R EVERSE PTP node ran n instances of PTPd in slave-only mode, each at a
different PTP domain, using n PTP domain numbers. PTPd, when in slave mode, periodically provides the current Offset From Master. This value is typically used by the PTPd servo algorithm [19].
Our experiments used the Offset From Master output to perform the first order approximation
of equation (10).
Nodes 3 to 34 played the role of R EVERSE PTP masters, node 1 was the R EVERSE PTP slave, and
node 2 served as a measurement probe. The experiment consisted of two parts:
1. Coordinated event. We scheduled nodes 3 to 34 to send a Ping message to node 2 at the same
time. The scheduling was based on ReverseTimeConf using the offsets computed by node 1, the
ReversePTP slave, with the approximation of equation (10). To simplify the experiment, we did
not use a control protocol such as OpenFlow. Instead, we used a simple doorbell-based method
to distribute the scheduling to nodes 3 to 34; after the scheduling times were computed, they
were written to the Network File System, and each of the nodes 3 to 34 reads its scheduling time
from the Network File System. In node 2, we monitored the distribution of the Ping message
arrival times; the arrival time of each packet was captured by Wireshark [30] using the machine’s
Linux clock. Interestingly, this experiment is the message-based variant of a 1 pulse per second
signal sent from each of the 32 clocks to a single testing device.
We repeated the experiment with PTP instead of ReversePTP, using TimeConf (Figure 3).
The distribution of the arrival times of the 32 Ping messages at node 2 is illustrated in
Figure 8(a). The value ‘0’ on the time axis represents the median of the arrival times. As shown
in Figure 8(a), the arrival times were spread over a period of about 8 milliseconds.
In Section 3, we observed that the approximation of equation (10) is valid when T s Tislast
is negligible. In our experiments, the Ping messages were scheduled to be sent 15 seconds in the
future, and hence T s Tislast was on the order of 15 seconds. The results show that 15 seconds
is a sufficiently low value to allow equation (10) to produce an accurate approximation.
2. Timestamped event. We sent a broadcast Ping message from node 2 to nodes 3 to 34 and
measured its arrival times to each of these nodes using Wireshark. We then used equation (10)
to align the reception times to a common ReversePTP-based time reference. We repeated the
experiment using conventional PTP. The empirical probability density function of the arrival
times measured at nodes 3 to 34 is depicted in Figure 8(b). Notably, the distribution of the
arrival time in this experiment provides a rough estimate of the clock accuracy; because the
Broadcast Ping message is replicated and distributed by the test bed’s hardware switches, we
expect very low delay variation among the replicas. Thus, Figure 8(b) provides an estimate of the
clock accuracy.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
368
T. MIZRAHI AND Y. MOSES
Figure 9. R EVERSE PTP versus PTP: rate of PTP messages sent or received by each node.
We observed that in the coordinated event experiment, the time elapsed from when the Ping message
was scheduled to be transmitted until it was transmitted in practice varied at different nodes on the order
of a few milliseconds. Hence, in the coordinated event experiment, the accuracy of our measurement
was affected by the internal delay of the sending hosts’ operating systems, thus explaining the fact that
the arrival time in Figure 8(a) ranges over a period of about 8 milliseconds, a significantly wider range
than the one shown in Figure 8(b).
The experiment demonstrates how R EVERSE PTP can be effectively used to coordinate events or to
accurately measure the occurrence time of events. It shows that R EVERSE PTP provides the same level
of accuracy as the conventional PTP.
6.2. Scalability
In the second experiment, we studied the scalability aspects of R EVERSE PTP, with an emphasis on
the load placed on a R EVERSE PTP slave compared with a conventional PTP master.
Rate of protocol messages. Figure 9 compares the protocol message rate of R EVERSE PTP with
that of conventional PTP. The Sync rate in our experiments was 1 Sync message per second, with
a Delay_Req message rate of 1 per second as well. We used two-step mode, and thus, Follow_Up
messages were used as well. We ran the experiment for a duration of 100 seconds.
As depicted in Figure 9(b), the protocol message rates of a R EVERSE PTP master and a PTP slave
are similar, around 4 messages per second. Figure 9(a) illustrates the message rate at the R EVERSE PTP
slave and at the PTP master; the number of messages processed by the R EVERSE PTP slave is roughly
twice the number of the PTP master. The reason is that we ran PTP in hybrid mode, where Sync and
Follow_Up were multicast messages, whereas in R EVERSE PTP, all messages were unicast. The fact
that some of the messages were sent as multicast allowed the conventional PTP to use roughly half the
number of messages used by R EVERSE PTP.
CPU Utilization. A more important metric of scalability is CPU utilization. We analyzed the difference between R EVERSE PTP and PTP in terms of the percentage of the CPU that is utilized by the PTP
process. We repeated the experiment for two types of machines: type I, a Dual Core AMD Opteron,
running at 1.8 GHz, and type II, an Intel Xeon E3 LP, running at 2.4 GHz. Note that the two types we
chose are at the extremes of the performance scale; at the time of the experiment, type II was the most
powerful machine in the DeterLab test bed, while type I was the least powerful one.
The CPU utilization measured on R EVERSE PTP masters and PTP slaves (i.e., switches in an SDN
environment) was negligible, well below 0.1% on both machine types. A significantly higher utilization
was measured on PTP masters and R EVERSE PTP slaves (controllers in an SDN environment), as
illustrated in Figure 10(a,b). These two figures illustrate the CPU utilization of the PTPd process as
a function of the number of nodes, either R EVERSE PTP masters or PTP slaves. As expected, the
CPU utilization of the R EVERSE PTP slave (Figure 10(a)) is significantly higher than that of the PTP
master (Figure 10(b)), because the R EVERSE PTP slave runs n servo algorithm instances. Nevertheless,
R EVERSE PTP consumes only 5% of the resources of machine type I and significantly less than 1% of
the resources of the more powerful type II.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
369
Figure 10. CPU utilization in R EVERSE PTP and in PTP as a function of the number of nodes. The
figures are presented for two machines types: type I is a low performance machine, and type II is
high performance.
Figure 11. Coordinated updates using R EVERSE PTP
Figure 10(c) provides some insight into the effect of R EVERSE PTP on an SDN application such
as the T IME C ONF protocol. The figure illustrates the CPU utilization of an SDN controller running
R EVERSE T IME C ONF versus an SDN controller running T IME C ONF with the conventional PTP. In
the experiment, we used the time-enabled OpenFlow prototype of [8]; the controller sent timed update
messages to the switches at a rate of 1 message per second. The figure illustrates the CPU utilization as
a function of the number of SDN switches. Only the utilization of the SDN controller task was considered in Figure 10(c), without considering the PTP task. As shown in the figure, R EVERSE T IME C ONF
yields higher CPU load, as each timed message sent by the controller requires the time conversion of
line 2 of Figure 11.
These results indicate that although R EVERSE PTP incurs higher load on the controller’s CPU than
conventional PTP, the R EVERSE PTP scheme can easily scale to hundreds of nodes and can scale to
thousands of nodes when running a R EVERSE PTP slave on a powerful machine.
7. DISCUSSION
7.1. Accuracy
The clock accuracy in a network depends on a number of factors, including the accuracy of the timestamping mechanisms, the quality of the clock oscillators, and whether on-path support is used. As
discussed in the Introduction, given the same network characteristics, we expect R EVERSE PTP and
PTP to provide the same degree of accuracy. The experimental evaluation of Section 6 confirmed
that R EVERSE PTP, even with the approximation of equation (8), can provide an accuracy that is
comparable with that of conventional PTP.
As stated in Section 4, the R EVERSE PTP paradigm implies that TCs are non-syntonized. Although
the IEEE 1588 standard does not specify that TCs must be syntonized, syntonized TCs have been
shown [31] to provide a higher degree of accuracy, especially over a large number of hops. A possible
extension to R EVERSE PTP that mitigates this limitation is discussed in Section 7.3.
7.2. Scalability–Programmability trade-off
The experimental evaluation in Section 6.2 showed that R EVERSE PTP loads the SDN controller’s
resources more than traditional PTP. This emphasizes a trade-off that is at the heart of SDN; the
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
370
T. MIZRAHI AND Y. MOSES
controller takes on tasks that are traditionally performed by switches in non-SDN networks, allowing
high flexibility and programmability, at the cost of controller resources. The experiments show that
although R EVERSE PTP requires more resources than conventional PTP, the R EVERSE PTP scheme
can easily scale to networks with thousands of SDN switches.
7.3. Synchronizing clocks using R EVERSE PTP
The concept we presented does not require switches to be synchronized to a common wall-clock
time. However, R EVERSE PTP can be extended to allow switches to be time-synchronized. PTP allows
masters to query slaves about the master–slave offset using PTP management messages. Using these
messages in R EVERSE PTP, switches can synchronize their clocks with the controller’s clock. Note
that the offset only allows switches to get a first-order approximation, as per equation (10). It is possible to extend PTP to allow slaves to periodically send the three parameters Tislast , oO i , and ¡O i , allowing
R EVERSE PTP masters to maintain an accurately synchronized clock. Note that this extension allows
switches to be synchronized at the cost of additional complexity and message exchanges. The main
benefit of this approach compared with the conventional PTP is that the algorithmic logic is centralized
and programmable.
The latter extension can similarly be used to allow syntonized TCs; a non-syntonized TC may perform less accurate updates of the correction field because of its inaccurate frequency, whereas a TC
that periodically receives the aforementioned three parameters can use these parameters to accurately
compute the correction field, using well-known methods (e.g., [31]).
7.4. R EVERSE PTP in a software-defined network with multiple controllers
Software-defined networks often use multiple controllers in an active-standby mode to provide survivability. In other cases, an SDN is sliced into multiple virtual networks (e.g., [26]), each of which
is governed by a separate controller. Interestingly, the R EVERSE PTP architecture is well-suited for
multi-controller configurations; each of the switches (masters) distributes its time to all the controllers,
allowing each controller to monitor its own offset information regarding the switches. Thus, in sliced
networks, R EVERSE PTP allows each slice to be managed according to a different time reference, by
allowing each controller to be synchronized to a different reference source. Notably, this slicing property is exclusive to R EVERSE PTP and is not possible in conventional clock synchronization methods.
Time distribution to multiple controllers can be performed efficiently by using a multicast group that
consists of the controllers, thereby reducing the rate of PTP messages, because each switch sends its
Sync messages to the multicast group. Moreover, when the controllers act in an active-standby mode,
the switches only need to distribute their time to the currently active controller.
7.5. Security aspects
The potential security vulnerabilities of R EVERSE PTP are similar to those of conventional synchronization protocols [32,33]. In PTP, a successful attack results in one or more slaves not being accurately
synchronized to the correct time, whereas in R EVERSE PTP, a successful attack causes the controller
to have an inaccurate view of the offset to one or more of the switches. An application that requires
accurate time is similarly affected in both cases.
8. CONCLUSION
Clock synchronization protocols are not ‘one size fits all’, as different applications may have different
requirements and constraints. We introduced R EVERSE PTP, a clock synchronization scheme suitable
for SDN. While R EVERSE PTP is tailored for SDN, it can be valuable in many other centralized and
software-controlled architectures, such as industrial automation systems [3], power-grid networks [34],
and various network-managed applications [35,36]. R EVERSE PTP provides the same level of accuracy as conventional synchronization protocols, including PTP, while its novel architecture shifts the
complex functionality from the switches to the controller, facilitating the agility and programmability
that are of key importance in SDNs.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
REVERSEPTP: A CLOCK SYNCHRONIZATION SCHEME FOR SDN
371
ACKNOWLEDGEMENTS
The authors would like to thank Wojciech Owczarek for his dedicated help and support with PTPd.
We gratefully acknowledge the DeterLab project [29] for the opportunity to perform our experiments
on the DeterLab test bed. This work was supported in part by the ISF grant 1520/11.
REFERENCES
1. ITU-T G.8271/Y.1366. Time and phase synchronization aspects of packet networks, 2012.
2. Corbett JC, et al. Spanner: Google’s globally distributed database. ACM Transactions on Computer Systems (TOCS) 2013;
31(3): 8.
3. Harris K. An application of IEEE 1588 to industrial automation. In IEEE ISPCS, Ann Arbor, Michigan, USA, 2008.
4. IEEE. Time-sensitive networking task group, 2012. Available from: http://www.ieee802.org/1/pages/tsn.html.
5. Moreira P, Serrano J, Wlostowski T, Loschmidt P, Gaderer G. White rabbit: sub-nanosecond timing distribution over
ethernet. In IEEE ISPCS, Brescia, Italy, 2009.
6. Open Networking Foundation. OpenFlow-enabled mobile and wireless networks. ONF Solution Brief 2013.
7. Mizrahi T, Moses Y. Time-based updates in software defined networks. In ACM SIGCOMM Workshop on Hot Topics in
Software Defined Networks HotSDN), Hong Kong, 2013.
8. Mizrahi T, Moses Y. Software defined networks: it’s about time. In IEEE INFOCOM, San Francisco, California, USA,
2016.
9. Mizrahi T, Saat E, Moses Y. Timed consistent network updates in software defined networks. IEEE/ACM Transactions on
Networking (ToN) 2016.
10. Mizrahi T, Moses Y. Time-based updates in OpenFlow: a proposed extension to the OpenFlow Protocol, 2013. Available
from: http://tx.technion.ac.il/~dew/OFTimeTR.pdf.
11. McKeown N, et al. OpenFlow: enabling innovation in campus networks. SIGCOMM Computer Communication Review
2008; 38: 69–74.
12. Open Networking Foundation. OpenFlow switch specification, 2015. Version 1.5.0.
13. IEEE TC 9. 1588 IEEE Standard for a Precision Clock Synchronization Protocol for Networked Measurement and Control
Systems Version 2, 2008.
14. Li H. IEEE 1588 time synchronization deployment for mobile backhaul in China Mobile, IEEE ISPCS, 2014.
15. Mills D, Martin J, Burbank J, Kasch W. Network time protocol version 4: protocol and algorithms specification. RFC 5905,
IETF 2010.
16. Mizrahi T, Moses Y. Using R EVERSE PTP to distribute time in software defined networks. In IEEE ISPCS, Austin, Texas,
USA, 2014.
17. Mizrahi T, Moses Y. R EVERSE PTP: a software defined networking approach to clock synchronization. In ACM SIGCOMM
Workshop on Hot Topics in Software Defined Networks (HotSDN), Chicago, Illinois, USA, 2014.
18. Veitch D, Ridoux J, Korada SB. Robust synchronization of absolute and difference clocks over networks. IEEE/ACM
Transactions on Networking (TON) 2009; 17(2): 417–430.
19. Correll K, Barendt N, Branicky M. Design considerations for software only implementations of the IEEE 1588 precision
time protocol. In Conference on IEEE 1588, Winterthur, Switzerland, 2005; 10–12.
20. Zhao Y, Liu J, Lee E, et al. A programming model for time-synchronized distributed real-time systems. In IEEE Real Time
and Embedded Technology and Applications Symposium (RTAS), Bellevue, WA, USA, 2007.
21. Derler P, Eidson JC, Goose S, Lee EA, Matic S, Zimmer M. Using PTIDES and synchronized clocks to design distributed
systems with deterministic system wide timing. In IEEE ISPCS, Lemgo, Germany, 2013.
22. ITU-T G.8265.1/Y.1365.1. Precision time protocol telecom profile for frequency synchronization, 2010.
23. Halpern JY, Simons B, Strong R, Dolev D. Fault-tolerant clock synchronization. In PODC, ACM, 1984; 89–102.
24. Srikanth TK, Toueg S. Optimal clock synchronization. Journal of the ACM (JACM) 1987; 34(3): 626–645.
25. Mills DL. Improved algorithms for synchronizing computer network clocks. IEEE/ACM Transactions on Networking (TON)
1995; 3(3): 245–254.
26. Open Networking Foundation. OpenFlow management and configuration protocol (of-config 1.2), 2014.
27. Tse R, Ong C. Proposal for a standardized mechanism to transfer timing information from an ingress port to an egress port
of a PTP transparent clock. In ISPCS, Special Session on Proposed Revisions of IEEE, San Francisco, California, USA,
2012; 1588–2008.
28. PTPd. Precision Time Protocol daemon, version 2.3.0, 2013. Available from: http://ptpd.sourceforge.net/.
29. The DeterLab project, 2015. Available from: http://deter-project.org/about_deterlab.
30. Wireshark, 2014. Available from: http://www.wireshark.org/.
31. Garner GM. Effect of a frequency perturbation in a chain of syntonized transparent clocks, IEEE 802.1, 2007.
32. Mizrahi T. Time synchronization security using IPsec and MACsec. In IEEE ISPCS, Munich, Germany, 2011.
33. Mizrahi T. Security requirements of time protocols in packet switched networks. Technical Report RFC 7384, IETF, 2014.
34. Dickerson B. Time in the power industry: how and why we use it, Arbiter Systems, 2010. Available from: http://www.
arbiter.com/ftp/datasheets/TimeInThePowerIndustry.pdf.
35. Mizrahi T, Moses Y. OneClock to rule them all: using time in networked applications. In IEEE/IFIP Network Operations
and Management Symposium (NOMS) Mini-Conference, Istanbul, Turkey, 2016.
36. Mizrahi T, Moses Y. Time capability in NETCONF. Technical Report RFC 7758, IETF, 2016.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem
372
T. MIZRAHI AND Y. MOSES
AUTHORS’ BIOGRAPHIES
Tal Mizrahi is a PhD student at the Technion. He is also a switch architect at Marvell, with over 15 years of
experience in networking. Tal is an active participant in the Internet Engineering Task Force (IETF), and in the
IEEE 1588 working group. His research interests include network protocols, switch and router architecture, time
synchronization, and distributed systems.
Yoram Moses is the Israel Pollak academic chair and a professor of electrical engineering at the Technion. His
research focuses on distributed and multi-agent systems, with a focus on fault-tolerance and on applications of
knowledge and time in such systems. He is a co-author of the book Reasoning about Knowledge, recipient of the
Gödel prize in 1997 and the Dijkstra prize in 2009.
Copyright © 2016 John Wiley & Sons, Ltd.
Int. J. Network Mgmt 2016; 26:355–372
DOI: 10.1002/nem