Subject: Re: [netlmm] Issue: Timestamp vs Sequence Number based logic

From: Christian Vogt <[email-masked]>
Date: Tue, 04 Sep 2007 15:07:37 +0300

Alexandru Petrescu wrote:
> -use of NTP timestamp format is incompatible with ISO C 200X.
> -use of NTP protocol for syncing is incompatible with IEEE 802.16
>  requiring TIME instead of NTP.
> -use of timestamps to avoid sequencing issues upon handovers is new
>  software development (if starting from MIP6 and going to PMIP6),
>  whereas the problem could be solved with seqno tools from MIP6.
> -use of NTP timestamps obviously has problems of solving the sequencing
>  problem in year 2036.

Alex,

you have actually brought up two issues:  (1) Should we use time stamps at all?
 And if yes:  (2) Should we require NTP for clock synchronization?

Our consensus addressed issue (1), and my personal believe is that we should
stick to it.  Use of time stamps will be the default for reordering protection.
 But alternatives will be possible as future P-MIPv6 extensions.  (Of course,
such extensions will have to consider interoperability between P-MIPv6 domains
to allow roaming.)

Therefore, in-line with what Kilian suggested, /support/ (i.e., implementation)
of time stamps should be a "MUST", whereas /use/ of time stamps should be a
"MUST unless an alternative mechanism for reordering prevention is in place."

Regarding issue (2), I think your point is valid that the P-MIPv6 spec should
not require NTP, but leave it up to the operator how to synchronize clocks.  But
the text that Sri suggested already provides such flexibility.  The only
dependency on NTP currently is with the time stamp format for P-MIPv6 messages,
which will get fixed by adopting the format from SEND as suggested by Julien.

Given that issue #160 in our tracker equals the above issue (1), and given that
we reached consensus on that, I will close it down now.  Of course, I will
reopen it if that consensus gets challenged.

- Christian

_______________________________________________
netlmm mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/netlmm

---

From: Alexandru Petrescu <[email-masked]>
Date: Tue, 04 Sep 2007 15:07:06 +0200

Christian Vogt wrote:
>> -use of NTP timestamp format is incompatible with ISO C 200X. -use
>> of NTP protocol for syncing is incompatible with IEEE 802.16 
>> requiring TIME instead of NTP. -use of timestamps to avoid
>> sequencing issues upon handovers is new software development (if
>> starting from MIP6 and going to PMIP6), whereas the problem could
>> be solved with seqno tools from MIP6. -use of NTP timestamps
>> obviously has problems of solving the sequencing problem in year
>> 2036.
> 
> Alex,
> 
> you have actually brought up two issues:  (1) Should we use time
> stamps at all? And if yes:  (2) Should we require NTP for clock
> synchronization?

Right Christian, it's different issues.  I'd say (1) should we use
timestamps or seqnos? (2) should we require the NTP's way of encoding
timestamps or should we use ISO C200X way of encoding? and (3) should we
require the use of NTP to sync clocks or should we require the use of
GPS to sync clocks, or the use of TIME protocol?

> Our consensus addressed issue (1), and my personal believe is that we
> should stick to it.  Use of time stamps will be the default for
> reordering protection.

Ok, I see I can not convince you.

Why did I raise all this;

When trying to implement a system relying on timestamps as opposed to
relying on seqnos: my personal experience is that the latter is much
easier to make reliable.

> But alternatives will be possible as future P-MIPv6 extensions.  (Of
> course, such extensions will have to consider interoperability
> between P-MIPv6 domains to allow roaming.)

I think even the base timestamp logic as written today (reliance on NTP)
has to consider interoperability between P-MIPv6 domains to allow
roaming.  If a PMIPv6 synchs its clocks and another doesn't then
obviously is a risk of out-of-order P-BUs when roaming.

Also, there's a difference between keeping synchronous time locally,
within a system, and keeping a synchronous time to universal time.

Many systems just do the former.

Also, implementation-wise, issues about bad and incompatible time keep
come up regularly.  It is not long ago that a known software
manufacturer accepted that its way of representing and using timezones
between two different applications was wrong, with very important impact
on business (people's calendars got messed).

> Therefore, in-line with what Kilian suggested, /support/ (i.e.,
> implementation) of time stamps should be a "MUST", whereas /use/ of
> time stamps should be a "MUST unless an alternative mechanism for
> reordering prevention is in place."

I agree, I think the alternative is already in place, based on MIP6
seqnos.  It's surprising P-MIPv6 needs seqnos whereas MIP6 doesn't.

> Regarding issue (2), I think your point is valid that the P-MIPv6
> spec should not require NTP, but leave it up to the operator how to
> synchronize clocks.  But the text that Sri suggested already provides
> such flexibility.  The only dependency on NTP currently is with the
> time stamp format for P-MIPv6 messages, which will get fixed by
> adopting the format from SEND as suggested by Julien.

I agree, I think SeND format (48/16bit seconds since  1970) is better
than NTP format (32/32bit seconds since 1900) at least because more
compatible with some Unices and because delays the roll-over flag day to
much later than 2036.  But I think the roll-over issue is still there.
IMHO SeND could get rid  of the timestamp as well.

At minimal, IMHO we could write in the doc the roll-over P-MIPv6 year,
by the Gregorian calendar (anyone cares to compute the year?) when
P-MIPv6 will have issues and how it should seamlessly be dealt with it
without disrupting service to MN.

> Given that issue #160 in our tracker equals the above issue (1), and
> given that we reached consensus on that, I will close it down now.
> Of course, I will reopen it if that consensus gets challenged.

Thanks,

Alex

______________________________________________________________________
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
______________________________________________________________________

---

From: Julien Laganier <[email-masked]>
Date: Tue, 4 Sep 2007 17:27:49 +0200

On Tuesday 04 September 2007, Alexandru Petrescu wrote:
> compatible with some Unices and because delays the roll-over flag day
> to much later than 2036.  But I think the roll-over issue is still
> there. IMHO SeND could get rid  of the timestamp as well.
>
> At minimal, IMHO we could write in the doc the roll-over P-MIPv6
> year, by the Gregorian calendar (anyone cares to compute the year?)
> when P-MIPv6 will have issues and how it should seamlessly be dealt
> with it without disrupting service to MN.

Alex, as I wrote earlier with 48bits seconds we are safe for nearly 9 
millions years.

--julien

---

From: Alexandru Petrescu <[email-masked]>
Date: Tue, 04 Sep 2007 18:14:11 +0200

Julien Laganier wrote:
>> I agree, I think SeND format (48/16bit seconds since  1970) is 
>> better than NTP format (32/32bit seconds since 1900) at least 
>> because more compatible with some Unices and because delays the 
>> roll-over flag day to much later than 2036.  But I think the 
>> roll-over issue is still there. IMHO SeND could get rid  of the 
>> timestamp as well.
>> 
>>  when P-MIPv6 will have issues and how it should seamlessly be 
>> dealt with it without disrupting service to MN.
> 

Thanks, this is good to know (and sorry I have just seen now your
previous message).

We should then write down what happens at year 9million+1picosecond: (1)
how is the LMA making sure it survives the overflowing P-BU without
believing it's a very old P-BU; and (2) without disrupting service to MN.

Don't you think so?

Alex

---

From: Alexandru Petrescu <[email-masked]>
Date: Wed, 05 Sep 2007 11:20:59 +0200

Christian Vogt wrote:
[...]
> Of course, I will reopen it if that consensus gets challenged.

Hi Christian, first thank you for taking care thoroughly of this issue.

I wanted to send another note about precision of time synchronization.

I have just compared time kept by four different deployed systems: a
Windows PC with NTP, a GSM handset, a GPS device without EGNOS, and a 
wristwatch on the German DCF77 long wave time signal.

My observation is that the four systems display 4 different times, with
as much as 1second difference between each two systems.

How this relates to our P-MIPv6 issue of relying on time
synchronization?  Two:

C.V.:
> Use of time stamps will be the default for reordering protection. But
> alternatives will be possible as future P-MIPv6 extensions.  (Of

The current P-MIPv6 relying on entities being time synchronized may have 
an issue of interoperability between P-MIPv6 domains (eg one domain 
keeps time with NTP and the other with DCF77).  It is not only the 
future extensions that potentially have this issue.

The other issue is that _if_ P-MIPv6 protocol relies on time 
synchronization then it should recommend that all entities within the 
P-MIPv6 domain (all LMAs and MAGs) keep the time by using the same 
unique method (examples of time keeping methods are NTP, TIME, GSM time 
synchronization, GPS and DCF77 - feel free to suggest another).

Currently the document states:
>    o  All the mobility entities in a Proxy Mobile IPv6 domain,
>       exchanging signaling messages using Timestamp option must have
>       adequately synchronized time-of-day clocks.  These nodes SHOULD
>       synchronize their clocks to a common time source using Network
>       Time Protocol [RFC-1305].

I think the recommendation should be that MAGs and LMAs SHOULD keep time 
synchronized by a unique method.  We surely don't want MAG to use GPS 
synchronization and LMA to use NTP synchronization.

What do you think?

Alex

---

From: Christian Vogt <[email-masked]>
Date: Wed, 05 Sep 2007 15:07:39 +0300

Alex,

the reason why the WG considered timestamps superior to sequence numbers was
that was the lack of a mechanism to synchronize sequence numbers across MAGs.

OTOH, sufficiently precise timestamps would allow reordering prevention because
there will always be a link layer handover and some link-local signaling between
critical P-BU messages, and this will likely take much longer than the maximum
error in clock synchronization.

This is why we probably must accept some implementation and deployment
challenges related to timestamps and clock synchronization.

But, in another email, you described what P-MIPv6 domain operators should keep
in mind for clock synchronization.  I'd suggest that you write these
recommendations down for inclusion in the P-MIPv6 spec.  What do you and the
other folks think about this?

Best,
- Christian

---

From: Alexandru Petrescu <[email-masked]>
Date: Wed, 05 Sep 2007 15:06:33 +0200

> that was the lack of a mechanism to synchronize sequence numbers across MAGs.

In previous discussions there seemed to be some interest in making the 
seqno mechanism of 3775 to work for P-MIPv6 as well.  I think it's not 
difficult to link the seqno to a MN address, at MAG.  If there is 
interest I can contribute discussion to this.

> error in clock synchronization.

I'm not sure what example you have in mind... but if we consider some 
average wifi link-layer handovers to be around 500ms this is clearly 
below the 1s I've noticed as error between time keeping of different 
systems.  Most performant wifi handovers I've seen are around 5ms.

What do you consider as link-layer handover time?  What technology?

> challenges related to timestamps and clock synchronization.

I think this is artificial.

I don't think we have a clear idea about what the P-MIPv6 handover time is.

> other folks think about this?

If other people agree then yes, I can write recommendations about time 
synchronization...

(ideally I'd like to contribute to a mechanism for MAG to use seqnos
  linked to the MN address, instead; and not recommending time
  synchronization at all.)

Alex

---

From: Christian Vogt <[email-masked]>
Date: Wed, 05 Sep 2007 16:29:34 +0300

> I'm not sure what example you have in mind... but if we consider some average
> wifi link-layer handovers to be around 500ms this is clearly below the 1s
> I've noticed as error between time keeping of different systems.  Most
> performant wifi handovers I've seen are around 5ms.

Alex,

I fully agree with you that we should not focus on link layer handover delays
that are prevalent today, but that we should support much lower ones.

But at least within a single P-MIPv6 domain, it should well be possible to
reduce synchronization errors to below 1 ms because the round-trip times are
then sufficiently small.

As I had mentioned in an earlier email, roaming across P-MIPv6 domains requires
global clock synchronization, where errors might be higher.  This requires
additional thought on how to synchronize clocks.

- Christian

---

From: Alexandru Petrescu <[email-masked]>
Date: Wed, 05 Sep 2007 15:40:39 +0200

> reduce synchronization errors to below 1 ms because the round-trip times are

I agree, provided every MAG and the LMA within that domain use the same 
time synchronization mechanism.

I think, not sure, that when you say '1ms' you already assume that they 
use NTP, because  NTP is known to claim that '1ms' accuracy.

> additional thought on how to synchronize clocks.

I agree.

Alex

---

From: "Alper Yegin" <[email-masked]>
Date: Wed, 5 Sep 2007 23:14:21 +0300

Hello,

> average

There seems to be an assumption here with respect to the distance between
the MAG and LMA. Considering how PMIP6 will be deployed with various
architectures (e.g., WiMAX), I don't think we can make this assumption.

> > additional thought on how to synchronize clocks.

Are we talking about synchronizing the clocks among all MAGs and LMAs that
are interacting with each other? Each MAG and LMA may belong to a different
operator, located at a different corner of the Internet. Can NTP practically
deal with this? Just asking...

Alper

---

From: "Sri Gundavelli" <[email-masked]>
Date: Wed, 5 Sep 2007 13:32:46 -0700

Hi Alper,

> -----Original Message-----
> From: Alper Yegin [mailto:[email-masked]] 
> Sent: Wednesday, September 05, 2007 1:14 PM
> To: 'Alexandru Petrescu'; 'Christian Vogt'
> Cc: 'Kilian Weniger'; [email-masked]; 'Julien Laganier'
> Subject: RE: [netlmm] Issue: Timestamp vs Sequence Number based logic
> 
> Hello,
> 
> > Christian Vogt wrote:
> > >> wifi link-layer handovers to be around 500ms this is 
> clearly below the
> > 1s
> > > I fully agree with you that we should not focus on link 
> layer handover
> > delays
> > > But at least within a single P-MIPv6 domain, it should 
> well be possible
> > to
> > > reduce synchronization errors to below 1 ms because the 
> round-trip times
> > are
> > > then sufficiently small.
> 
> > > As I had mentioned in an earlier email, roaming across 
> P-MIPv6 domains
> > requires

It is important to highlight the fact that we need this entire 
synchronization logic to avoid one scenario, where the LMA
ends up processing a PBU that was sent by pMAG and the LMA
received that much later due to network out of order deliverly.
In my mind, this is a special case. Now the solutions:

The spec supports both Sequence Number based and Timestamp based
approaches. If a given architecture has some kind of interface
between MAG to MAG for context transfers, which I assume it is
true for WiMAX as well, then the sequence number based scheme will
just fine. 

Now, for completeness, in the absence of a context transfer
protocol, the base spec supports a Timestamp based approach. It
expects the LMA's and MAG's to sync to a common clock source. If
a deployment does not enable NTP kind of protocols and if the
LMA receives a packet with PBU from a MAG with a invalid timestamp,
the LMA can return its own timestamp and the MAG can use the same
in the requests to avoid this special race condition. So, it should
not be a concern that the LMA and MAG are in different domains and
syncing is an issue. Also, MIP4 uses timestamp based logic and
there are deployments based on that. 

Regards
Sri

---

From: "Alper Yegin" <[email-masked]>
Date: Fri, 7 Sep 2007 00:32:20 +0300

Thanks Sri, that makes sense to me.

Alper 

> From: Sri Gundavelli [mailto:[email-masked]]
> Sent: Wednesday, September 05, 2007 11:33 PM
> To: 'Alper Yegin'; 'Alexandru Petrescu'; 'Christian Vogt'

---

From: Julien Laganier <[email-masked]>
Date: Fri, 7 Sep 2007 14:17:27 +0200

On Tuesday 04 September 2007, Alexandru Petrescu wrote:
> >> year, by the Gregorian calendar (anyone cares to compute the
> >> year?) when P-MIPv6 will have issues and how it should seamlessly
> >> be dealt with it without disrupting service to MN.
> >
> > Alex, as I wrote earlier with 48bits seconds we are safe for nearly
> > 9 millions years.
>
> We should then write down what happens at year 9million+1picosecond:
> (1) how is the LMA making sure it survives the overflowing P-BU
> without believing it's a very old P-BU; and (2) without disrupting
> service to MN.
>
> Don't you think so?

Maybe I'm wrong, but I somehow feel a timestamp wrap around is not a 
problem when it is going to happen 9 millions years from now; that 
gives us ample time (9 million years I guess) to specify an extended 
timestamp option that works after 9 million years and takes precedence 
over the current timestamp option. Further, when the wrap around occurs 
we'll probably not be using IPv6 anymore...

If that is really a concern to you, would you be fine with a 128 bits 
timestamp, that would work until the end of the universe and time?

--julien

---

From: Alexandru Petrescu <[email-masked]>
Date: Fri, 07 Sep 2007 14:43:16 +0200

Julien Laganier wrote:
>>>>  because more compatible with some Unices and because delays
>>>> the roll-over flag day to much later than 2036.  But I think
>>>> the roll-over issue is still there. IMHO SeND could get rid  of
>>>> the timestamp as well.
>>>> 
>>>> At minimal, IMHO we could write in the doc the roll-over 
>>>> P-MIPv6 year, by the Gregorian calendar (anyone cares to 
>>>> compute the year?) when P-MIPv6 will have issues and how it 
>>>> should seamlessly be dealt with it without disrupting service 
>>>> to MN.
>>> Alex, as I wrote earlier with 48bits seconds we are safe for 
>>> nearly 9 millions years.
>> We should then write down what happens at year 
>> 9million+1picosecond: (1) how is the LMA making sure it survives 
>> the overflowing P-BU without believing it's a very old P-BU; and 
>> (2) without disrupting service to MN.
>> 
>> Don't you think so?
> 
> timestamp option that works after 9 million years and takes 
> precedence over the current timestamp option. Further, when the wrap 
> around occurs we'll probably not be using IPv6 anymore...
> 

Julien, thank you for asking.  After discussing face-to-face with
colleagues around this issue, I've been also told that in 9million years
we have enough time to conquer the galaxies and/or disappear, etc, etc.

I would like to say that you are right.

It's just that when trying to specify the usage of time the things are
very complex to specify and to implement too.

Think for example that if we use timestamps that expire in 9million
years (or 128bit at the end of universe) then the common use of time
within that timestamp is going to be very localized.  Ie much of the
bits in these timestamps don't change at all, they're the same for a
long time.  This means, grosso modo, that we use 64bit to waste bandwidth.

Alex
>> a 128 bits timestamp, that would work until the end of the universe
>>  and time
PS:128bits survives the time itself?

---

From: Julien Laganier <[email-masked]>
Date: Fri, 7 Sep 2007 14:51:58 +0200

On Friday 07 September 2007, Alexandru Petrescu wrote:
> >> a 128 bits timestamp, that would work until the end of the
> >> universe and time
>

Time is either a dimension of the universe, or an intellectual 
construction of human beings. In both case, if the universe is gone the 
time too :)

--julien