Subject: Spin bit - semantics - rtt and loss

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Mar 2018 00:12:48 +0000

Since we decided to put some work into spin bit, albeit in a separate document from V1, here are some thoughts.

It is nice that spin bit can be used to estimate rtt. I am sure that it is useful to many well-meaning people. It is not very useful to me, though.

When I debug network problems, they are much more likely to be about lost packets. And I like to be able to know packet loss frequency and pattern very accurately. (I am assuming an encrypted packet number, or the problem would be trivial.)

I would prefer, if we have a spin bit, for it to spin not every rtt but every, say, 10 packets that an endpoint sends. Client sends 10 packets with 0 and then 10 packets with 1, repeat. Sever does the same.

So maybe two spin bits -- one for rtt and one for loss? I would assume that loss bit would assist in rtt estimation in case of loss.

Thoughts?

- Igor

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 07:55:08 +0100

Hi Igor,

Indeed packet loss is important to track.

Your proposal is an instance of the "square sequence" proposed earlier by 
Kazuho. He suggested a much longer period (hundreds); you're exploring lower 
values. Each range has issues: too long turns into a large temporal imprecision 
for sparse flows (like ACKs); too short blurs the signal in case of tail drops 
(many contiguous losses).

Also, these methods only give upstream losses (between the sender and the 
midpoint). For this reason we propose a more ambitious 7-bit solution here: Spin 
Counters

  http://snaggletooth.akam.ai/IETF-101-HotRFC/10-Ferrieux.pdf

Would that help ?

On 23/03/2018 01:12, Lubashev, Igor wrote:
> It is nice that spin bit can be used to estimate rtt. I am sure that it is
> useful to many well-meaning people. It is not very useful to me, though.
>
> When I debug network problems, they are much more likely to be about lost
> packets. And I like to be able to know packet loss frequency and pattern very
> accurately. (I am assuming an encrypted packet number, or the problem would be
> trivial.)
>
> I would prefer, if we have a spin bit, for it to spin not every rtt but every,
> say, 10 packets that an endpoint sends. Client sends 10 packets with 0 and then
> 10 packets with 1, repeat. Sever does the same.
>
> So maybe two spin bits -- one for rtt and one for loss? I would assume that loss
> bit would assist in rtt estimation in case of loss.
>
> Thoughts?
>
> - Igor

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 07:55:08 +0100

Hi Igor,

Would that help ?

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Mar 2018 07:15:38 +0000

Thanks, Alexandre!

Yes, 7 bits would work 64 times better, but it is 7 times more bits.  That's a difference between adding a byte to short packets vs adding no extra bytes.  I could live with both.

The 10-packet square wave was just an initial suggestion (the exact size is minor details, and I actually expect that to be an endpoint configuration option).  It also seems that having an rtt spin bit alongside the square wave should help in detecting losses that exceed a "wavelength" (20 packets in this case), though someone more analytically talented than me would likely be able to tell exactly what benefits an rtt wave adds to the loss estimation.

- Igor

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, March 23, 2018 6:55 AM
To: Lubashev, Igor <ilubashe=[email-masked]>
Cc: [email-masked]
Subject: Re: Spin bit - semantics - rtt and loss

Hi Igor,

Your proposal is an instance of the "square sequence" proposed earlier by Kazuho. He suggested a much longer period (hundreds); you're exploring lower values. Each range has issues: too long turns into a large temporal imprecision for sparse flows (like ACKs); too short blurs the signal in case of tail drops (many contiguous losses).

Also, these methods only give upstream losses (between the sender and the midpoint). For this reason we propose a more ambitious 7-bit solution here: Spin Counters

Would that help ?

On 23/03/2018 01:12, Lubashev, Igor wrote:
> Since we decided to put some work into spin bit, albeit in a separate 
> document from V1, here are some thoughts.
>
> It is nice that spin bit can be used to estimate rtt. I am sure that 
> it is useful to many well-meaning people. It is not very useful to me, though.
>
> When I debug network problems, they are much more likely to be about 
> lost packets. And I like to be able to know packet loss frequency and 
> pattern very accurately. (I am assuming an encrypted packet number, or 
> the problem would be trivial.)
>
> I would prefer, if we have a spin bit, for it to spin not every rtt 
> but every, say, 10 packets that an endpoint sends. Client sends 10 
> packets with 0 and then
> So maybe two spin bits -- one for rtt and one for loss? I would assume 
> that loss bit would assist in rtt estimation in case of loss.
>
> Thoughts?
>
> - Igor

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration, Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law; they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 23 Mar 2018 07:15:38 +0000

Thanks, Alexandre!

- Igor

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] 
Sent: Friday, March 23, 2018 6:55 AM
To: Lubashev, Igor <ilubashe=[email-masked]>
Cc: [email-masked]
Subject: Re: Spin bit - semantics - rtt and loss

Hi Igor,

Would that help ?

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 23 Mar 2018 00:42:28 -0700

Just to summarize the square wave discussion:

It is possible to send many other patterns one bit at a time that deal with
both short and long range loss.
At some point I suggested emitting a 8 bit Gray code counter which is
emitted one bit at a time. Optionally repeated twice before changing in
order to better sync the signal. This repeats every 8 packets or every 16
packets.

It was designed for loss detection - not spin or RTT because that isn’t
encoded into signal. But it can be used to sync the instance of a separate
spin bit. This would use two bits.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 23 March 2018 at 01.13.01, Lubashev, Igor (
ilubashe=[email-masked]) wrote:

When I debug network problems, they are much more likely to be about lost
packets. And I like to be able to know packet loss frequency and pattern
very accurately. (I am assuming an encrypted packet number, or the problem
would be trivial.)

I would prefer, if we have a spin bit, for it to spin not every rtt but
every, say, 10 packets that an endpoint sends. Client sends 10 packets with
0 and then 10 packets with 1, repeat. Sever does the same.

So maybe two spin bits -- one for rtt and one for loss? I would assume that
loss bit would assist in rtt estimation in case of loss.

Thoughts?

- Igor

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 23 Mar 2018 00:42:28 -0700

Kind Regards,
Mikkel Fahnøe Jørgensen

Thoughts?

- Igor

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 23 Mar 2018 00:50:30 -0700

It should be noted that a Gray counter would not count linearly, but have
one bit changing for every power of 2 larger amount of data sent (details
may vary).

The spin bit could also be encoded by some phase of the pattern, for
example sending the same 8 bit count twice, but with one of the counts
inverted when spin is set. But that makes things more difficult to
implement and looses some precision when very few packets are sent.

Kind Regards,
Mikkel Fahnøe Jørgensen

On 23 March 2018 at 08.42.28, Mikkel Fahnøe Jørgensen ([email-masked])
wrote:

Kind Regards,
Mikkel Fahnøe Jørgensen

Thoughts?

- Igor

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Fri, 23 Mar 2018 00:50:30 -0700

Kind Regards,
Mikkel Fahnøe Jørgensen

Kind Regards,
Mikkel Fahnøe Jørgensen

Thoughts?

- Igor

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 09:35:12 +0100

Hi Mikkel,

One thing about patterned sequences as opposed to square, is their lack of 
robustness to mild loss or reordering. That's unfortunate, because passive 
measurement is primarily useful when such things occur ...

To me the bottom line is that properly addressing this need has a cost, and that 
this cost may exceed one single bit (above the 1 or 2 spin bit(s)).

The alternative then becomes

  (a) let's handle packet loss ; need k bits ; that may not fit in the currently 
"available" bits in the type byte ; let's frankly allocate them

  (b) packet loss handling deemed luxury

On 23/03/2018 08:50, Mikkel Fahnøe Jørgensen wrote:
> It should be noted that a Gray counter would not count linearly, but have one 
> bit changing for every power of 2 larger amount of data sent (details may vary).
>
> The spin bit could also be encoded by some phase of the pattern, for example 
> sending the same 8 bit count twice, but with one of the counts inverted when 
> spin is set. But that makes things more difficult to implement and looses some 
> precision when very few packets are sent.
>
> Kind Regards,
> Mikkel Fahnøe Jørgensen
>
>
> On 23 March 2018 at 08.42.28, Mikkel Fahnøe Jørgensen ([email-masked] 
> <mailto:[email-masked]>) wrote:
>
>> At some point I suggested emitting a 8 bit Gray code counter which is emitted 
>> one bit at a time. Optionally repeated twice before changing in order to 
>> better sync the signal. This repeats every 8 packets or every 16 packets.
>>
>> It was designed for loss detection - not spin or RTT because that isn't 
>> On 23 March 2018 at 01.13.01, Lubashev, Igor 
>> (ilubashe=[email-masked] 
>> <mailto:ilubashe=[email-masked]>) wrote:
>>

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 09:35:12 +0100

Hi Mikkel,

On 23/03/2018 08:50, Mikkel Fahnøe Jørgensen wrote:
> On 23 March 2018 at 08.42.28, Mikkel Fahnøe Jørgensen (mikkelfj@gmail..com

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 09:39:06 +0100

Note that the spin counters don't just use more bits ; they also add 
bidirectional feedback and allow to see loss both downstream and upstream in 
either direction. And their transmission is robust to any level of reordering 
and loss.

On 23/03/2018 08:15, Lubashev, Igor wrote:
> To: Lubashev, Igor<ilubashe=[email-masked]>
> Cc: [email-masked]

---

From: <[email-masked]>
Date: Fri, 23 Mar 2018 09:39:06 +0100

---

From: Fioccola Giuseppe <[email-masked]>
Date: Fri, 23 Mar 2018 09:51:28 +0000

Agree and, in general, spin square wave and spin counters can also allow intra-domain loss and delay measurements with the application of RFC8321.

Giuseppe

-----Messaggio originale-----
Da: QUIC [mailto:[email-masked]] Per conto di [email-masked]
Inviato: venerdì 23 marzo 2018 09:39
A: Lubashev, Igor
Cc: Lubashev, Igor; [email-masked]
Oggetto: Re: Spin bit - semantics - rtt and loss

Note that the spin counters don't just use more bits ; they also add bidirectional feedback and allow to see loss both downstream and upstream in either direction. And their transmission is robust to any level of reordering and loss.

On 23/03/2018 08:15, Lubashev, Igor wrote:
> Also, these methods only give upstream losses (between the sender and 
> the midpoint). For this reason we propose a more ambitious 7-bit 
> solution here: Spin Counters
>
>>  Since we decided to put some work into spin bit, albeit in a 
>> separate  document from V1, here are some thoughts.
>>
>> pattern very accurately. (I am assuming an encrypted packet number, 
>> or  the problem would be trivial.)
>>
>>  So maybe two spin bits -- one for rtt and one for loss? I would 
>> assume  that loss bit would assist in rtt estimation in case of loss.
>>
>>  Thoughts?
>>
>>  - Igor
>
>
> ______________________________________________________________________
> ___________________________________________________
>

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie. 

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 23 Mar 2018 09:53:58 +0000

Hi,

> Agree and, in general, spin square wave and spin counters can also allow intra-domain loss and delay measurements with the application of RFC8321.

I note that RFC8321 has an IPR disclosure filed against it. At some point, it would be good to understand if and to what degree said IPR extends to spinning bits in QUIC.

Lars

---

From: Fioccola Giuseppe <[email-masked]>
Date: Fri, 23 Mar 2018 10:29:56 +0000

Hi Lars,
It is difficult to give a definitive answer now because it will depend by the implementation.
The application of RFC8321 measurements is certainly a factor.

Giuseppe

-----Messaggio originale-----
Da: Eggert, Lars [mailto:[email-masked]] 
Inviato: venerdì 23 marzo 2018 10:54
A: Fioccola Giuseppe
Cc: [email-masked]; Lubashev, Igor; [email-masked]
Oggetto: Re: Spin bit - semantics - rtt and loss

Hi,

Lars

---

From: "Eggert, Lars" <[email-masked]>
Date: Fri, 23 Mar 2018 10:51:42 +0000

Hi,

On 2018-3-23, at 11:29, Fioccola Giuseppe <[email-masked]> wrote:
> It is difficult to give a definitive answer now because it will depend by the implementation. The application of RFC8321 measurements is certainly a factor.

understood.

Given that draft-trammell-quic-spin-01 is what's currently been written up, understanding whether any such IPR applies to it would be useful.

Lars

---

From: Fioccola Giuseppe <[email-masked]>
Date: Mon, 26 Mar 2018 13:44:19 +0000

Ok, I will investigate and find out if any IPR applies.

Giuseppe

-----Messaggio originale-----
Da: Eggert, Lars [mailto:[email-masked]] 
Inviato: venerdì 23 marzo 2018 11:52
A: Fioccola Giuseppe
Cc: [email-masked]; Lubashev, Igor; [email-masked]
Oggetto: Re: Spin bit - semantics - rtt and loss

Hi,

understood.

Lars