Subject: [Detnet] DP solution (Sequence Numbering)

From: Balázs Varga A <[email-masked]>
Date: Mon, 15 Jan 2018 13:05:44 +0000

Hi,

Some thoughts to start discussion on open points of the DP solution draft.

This mail(-thread) is on sequence numbering.

Problem in case of MPLS and DetNet Control Word:

- S/N skips zero problem: Ethernet PW definition states that S/N skips zero.

It uses zero to mean no S/N in use. (rfc4448)

Possible problem resolutions:

#1: Using a "flag" in the reserved bits part of the CW:

      0                   1                   2                   3

      0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

     |0 0 0 0|D|reserved - set to 0  |   16 bit Sequence Number      |

     +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

D-bit: set to "1" means DetNet specific Seq. Number used,

Bit5-15: reserved.

RFC4448 states that reserved bits "MUST be ignored upon receipt", so backward

compatibility is ensured. Such a solution would allow also to use longer seq.num.

formats (if needed). Such a format is also inline with RFC4385.

#2: Making "0" as a valid Seq.Num.

DetNet flows are recognized and forwarded special within the DetNet

domain. S/N is used within the DetNet domain by DetNet Service functions

(i.e. PREF). S/N used only on DetNet end-nodes and DetNet relay-nodes. No

other nodes care about the value of S/N, therefore DetNet can specify its

own sequence number processing algorithm. So "0" can be a valid number

and not the indication that no Seq.Num. is used. This is inline with RFC4385.

#3: Other ... ?!?

Comments are welcome.

Cheers

Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: 2018. január 10. 19:01
To: DetNet WG <[email-masked]>
Subject: [Detnet] recording for today's interim

is available for download at:

https://ietf.webex.com/ietf/ldr.php?RCID=e5116bc7db254c24a7bb3217bf202301

raw notes are available at

http://etherpad.tools.ietf.org:9000/p/notes-ietf-interim-2018-detnet-01

Please feel free to update the etherpad based on the recording.

Lou (and co-chairs)

_______________________________________________

detnet mailing list

[email-masked]<mailto:[email-masked]>

https://www.ietf.org/mailman/listinfo/detnet

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 15 Jan 2018 14:14:19 +0000

On 15/01/2018 13:05, Balázs Varga A wrote:
> D-bit: set to “1” means DetNet specific Seq. Number used,
>
> Bit5-15: reserved.
>
> RFC4448 states that reserved bits “MUST be ignored upon receipt”, so 
> backward
>

Why do we need any compatibility with RFC4448 in this regard? The
only reason that I can see is if we think there are a lot of existing
T-PEs out there that support RFC4448 and s/n processing. I am ready
to be corrected, but I really don't think there is any significant 
deployment.

I therefore think that it would be much better to define a new
Detnet Ethernet encapsulation type with the exact properties we need.

Now related to that, do we need to support any non-ethernet
encapsulated Detnet payloads, i.e. do we have to support
a native connectivity type other than Ethernet? If so
we need to make sure that we leave the bits in place to
length recovery.

Also will we ever need to have this be self discovering (something
that PWs were no designed to be)?

If the answer is no to both of these, we could for example make the
CW look as follows:

0                   1                   2                   3

|0 0 0 0|                                  Sequence Number |

Then we can let the control plane specify the s/n length at whatever
make sense between 0 and 28 bits which should cover most applications.

What we have lost here is fragmentation at the PW layer, and the
ability to send non-Ethernet frames, but if those are not needed
we have a compact PW that serves our needs.

Note that this is shown in RFC4385  Fig1.

> #2: Making “0” as a valid Seq.Num.
>
> own sequence number processing algorithm. So “0” can be a valid number
>

Exactly. Let's just define the Detnet encapsulation(s) to meet the needs
of Detnet rather than try to force fit it into RFC4448.

Best Regards

Stewart

> #3: Other … ?!?
> [email-masked] <mailto:[email-masked]>
>
> [email-masked]

---

From: "Andrew G. Malis" <[email-masked]>
Date: Mon, 15 Jan 2018 16:45:35 -0500

Stewart has it right, we’ve both made this point at the last couple of
DETNET sessions. Determine the exact functionality that we need. such as
the sequence number length (I made the point during the recent virtual
meeting that 16 bits is probably not enough to prevent wrap-around) and
whether we need fragmentation, and just follow the guidelines for RFC 4385
figure 1 (first nibble needs to be zero, but after that, we’re free to do
what we want). It’s easy for IANA to allocate another PW type, DETNET in
this case.

Cheers,
Andy

On Mon, Jan 15, 2018 at 9:14 AM, Stewart Bryant <[email-masked]>
wrote:

> From: detnet [mailto:[email-masked] <[email-masked]>]
> On Behalf Of Lou Berger
> To: DetNet WG <[email-masked]> <[email-masked]>
> detnet mailing [email-masked]://www.ietf.org/mailman/listinfo/detnet
>
>
>

---

From: Mach Chen <[email-masked]>
Date: Tue, 16 Jan 2018 01:14:29 +0000

Hi Balazs,

#2 is a more nature solution, the DetNet CW is dedicated for DetNet, seems there is no compatibility issue.

In addition, I think we need to define a way on how to differentiate a DetNet flow from a non-DetNet flow, then the network processor can process the CW as expected.  In TSN, it uses the priority to determine whether a flow is TSN or non-TSN flow. For DetNet, should we also use the same way (e.g., TC, DSCP)? Or the DetNet service Label itself implies the situation, but it seems that this only works when the DetNet service label is the top label.

Any other thoughts?

Best regards,
Mach

From: detnet [mailto:[email-masked]] On Behalf Of Balázs Varga A
Sent: Monday, January 15, 2018 9:06 PM
To: DetNet WG <[email-masked]>
Subject: [Detnet] DP solution (Sequence Numbering)

Hi,

      0                   1                   2                   3

Bit5-15: reserved.

#3: Other ... ?!?

Comments are welcome.

Cheers

Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: 2018. január 10. 19:01
To: DetNet WG <[email-masked]<mailto:[email-masked]>>
Subject: [Detnet] recording for today's interim

Lou (and co-chairs)

detnet mailing list

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 16 Jan 2018 10:46:34 +0000

If we need to discriminate at the PSN layer rather than the DN layer 
then we really are forced to create a new FEC type for MPLS, or 
synthesize one through RSVP or SR. IP has no native support for this, 
but we could introduce the concept, for example by making this the 
property of the IP address or some way of MPLSing IP.

However there is a deeper question we need to answer. Is our expectation 
that each hop in the data plane DN aware, or is our expectation that at 
lest in some networks the data plane is generic with some DN nodes 
strategically placed?

I can see than in some greenfield specialist cases the routers will all 
be DN aware, but in order to get deployment in most networks we need to 
design for the case where only strategic nodes are DN aware.

- Stewart

On 16/01/2018 01:14, Mach Chen wrote:
> In addition, I think we need to define a way on how to differentiate a 
> DetNet flow from a non-DetNet flow, then the network processor can 
> process the CW as expected.  In TSN, it uses the priority to determine 
> whether a flow is TSN or non-TSN flow. For DetNet, should we also use 
> the same way (e.g., TC, DSCP)? Or the DetNet service Label itself 
> implies the situation, but it seems that this only works when the 
> DetNet service label is the top label.
>
> Any other thoughts?
>
> Best regards,
>
> Mach
>
> *From:* detnet [mailto:[email-masked]] *On Behalf Of *Balázs 
> Varga A
> *Sent:* Monday, January 15, 2018 9:06 PM
> *To:* DetNet WG <[email-masked]>
> *Subject:* [Detnet] DP solution (Sequence Numbering)
>
> Hi,
>
> To: DetNet WG <[email-masked] <mailto:[email-masked]>>

---

From: "Andrew G. Malis" <[email-masked]>
Date: Tue, 16 Jan 2018 10:20:40 -0500

Stewart,

Like with PWs (since DETNET for MPLS would just be another PW type),
targeted LDP and provisioning (via a central controller, NMS, etc.) are
alternatives to RSVP and SR to provisioning/allocating the DETNET PW label.

You and Mach are both right that in the IP (non-MPLS) case, we need a way
for the NP to detect DETNET flows. We could have a new IP protocol type for
DETNET, and a DETNET header directly follows the IP header. In that case,
the DETNET header would also need a next protocol identifier, unless the
next protocol is always another IP header.

Of course, for MPLS, the DETNET PW label is the identifier.

Cheers,
Andy

On Tue, Jan 16, 2018 at 5:46 AM, Stewart Bryant <[email-masked]>
wrote:

> If we need to discriminate at the PSN layer rather than the DN layer then
> we really are forced to create a new FEC type for MPLS, or synthesize one
> through RSVP or SR. IP has no native support for this, but we could
> introduce the concept, for example by making this the property of the IP
> address or some way of MPLSing IP.
>
> I can see than in some greenfield specialist cases the routers will all be
> DN aware, but in order to get deployment in most networks we need to design
> for the case where only strategic nodes are DN aware.
>
> - Stewart
>
> DetNet flow from a non-DetNet flow, then the network processor can process
> the CW as expected.  In TSN, it uses the priority to determine whether a
> flow is TSN or non-TSN flow. For DetNet, should we also use the same way
> (e.g., TC, DSCP)? Or the DetNet service Label itself implies the situation,
> but it seems that this only works when the DetNet service label is the top
> label.
>
>
>
> Any other thoughts?
>
>
>
> Best regards,
>
> Mach
>
>
>
> *From:* detnet [mailto:[email-masked] <[email-masked]>] *On
> Behalf Of *Balázs Varga A
> *To:* DetNet WG <[email-masked]> <[email-masked]>

---

From: Stewart Bryant <[email-masked]>
Date: Tue, 16 Jan 2018 16:38:22 +0000

Andy

On 16/01/2018 15:20, Andrew G. Malis wrote:
> targeted LDP and provisioning (via a central controller, NMS, etc.) 
> are alternatives to RSVP and SR to provisioning/allocating the DETNET 
> PW label.

.. and I suppose strictly BGP which is also used for this purpose in VPNs

>
> You and Mach are both right that in the IP (non-MPLS) case, we need a 
> way for the NP to detect DETNET flows. We could have a new IP protocol 
> type for DETNET, and a DETNET header directly follows the IP header. 
> In that case, the DETNET header would also need a next protocol 
> identifier, unless the next protocol is always another IP header.

Last I saw, the thinking was that there would be an IP extension header, 
although I am not a fan of those since (a) I am not confident what a 
router will do one it does not understand, and (b) I am concerned about 
the need to support IPv4 which I suspect will have a long tail in 
enterprise deployment.

I prefer a UDP type for this purpose since it has universal support 
through any middle box. firewall etc, and all routers need to look a the 
UDP DP so checking if it is one they need to work on is just one compare 
and conditional branch.

> Of course, for MPLS, the DETNET PW label is the identifier.

Only at the PW layer. Note my question about whether this is sufficient.

- Stewart

>
> Cheers,
> Andy
>
>
> On Tue, Jan 16, 2018 at 5:46 AM, Stewart Bryant 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     If we need to discriminate at the PSN layer rather than the DN
>     layer then we really are forced to create a new FEC type for MPLS,
>     or synthesize one through RSVP or SR. IP has no native support for
>     this, but we could introduce the concept, for example by making
>     this the property of the IP address or some way of MPLSing IP.
>
>     However there is a deeper question we need to answer. Is our
>     expectation that each hop in the data plane DN aware, or is our
>     expectation that at lest in some networks the data plane is
>     generic with some DN nodes strategically placed?
>
>     I can see than in some greenfield specialist cases the routers
>     will all be DN aware, but in order to get deployment in most
>     networks we need to design for the case where only strategic nodes
>     are DN aware.
>
>     - Stewart
>
>
>>     #2 is a more nature solution, the DetNet CW is dedicated for
>>     DetNet, seems there is no compatibility issue.
>>
>>     In addition, I think we need to define a way on how to
>>     differentiate a DetNet flow from a non-DetNet flow, then the
>>     network processor can process the CW as expected.  In TSN, it
>>     uses the priority to determine whether a flow is TSN or non-TSN
>>     flow. For DetNet, should we also use the same way (e.g., TC,
>>     DSCP)? Or the DetNet service Label itself implies the situation,
>>     but it seems that this only works when the DetNet service label
>>     is the top label.
>>
>>     Any other thoughts?
>>
>>     Best regards,
>>
>>     Mach
>>
>>     *From:* detnet [mailto:[email-masked]
>>     <mailto:[email-masked]>] *On Behalf Of *Balázs Varga A
>>     *To:* DetNet WG <[email-masked]> <mailto:[email-masked]>
>>     RFC4448 states that reserved bits “MUST be ignored upon receipt”,
>>     so backward
>>
>>     From: detnet [mailto:[email-masked]
>>     <mailto:[email-masked]>] On Behalf Of Lou Berger
>>     <https://ietf.webex.com/ietf/ldr.php?RCID=e5116bc7db254c24a7bb3217bf202301>
>>
>>     <http://etherpad.tools.ietf.org:9000/p/notes-ietf-interim-2018-detnet-01>
>>
>>     <https://www.ietf.org/mailman/listinfo/detnet>
>>
>>
>>
>>     <https://www.ietf.org/mailman/listinfo/detnet>
>
>
>     <https://www.ietf.org/mailman/listinfo/detnet>
>
>

---

From: "Grossman, Ethan A." <[email-masked]>
Date: Tue, 16 Jan 2018 18:03:54 +0000

Just a couple of my thoughts:

1)      Regarding whether every node in the network is required to be DetNet aware, I was assuming that it would be like AVB/TSN, in which you can only make a stream reservation through a path of nodes that are TSN aware, and traffic that had to cross non-aware nodes would have to be treated as non-TSN traffic, i.e. some packets could get dropped and there would not be timing guarantees, but at least they wouldn’t be dropped wholesale due to massive protocol incompatibility.  Is that the expectation here or is there a significant difference?

2)      I agree that we need to have some kind of strategy for supporting IPv4, for example providing at least DetNet-type delay and jitter bounds, even if it had some limitations compared to MPLS/IPv6 (e.g. no PREF and thus limited reliability).

Ethan.

From: detnet [mailto:[email-masked]] On Behalf Of Stewart Bryant
Sent: Tuesday, January 16, 2018 8:38 AM
To: Andrew G. Malis <[email-masked]>
Cc: DetNet WG <[email-masked]>; Mach Chen <[email-masked]>; Balázs Varga A <[email-masked]>
Subject: Re: [Detnet] DP solution (Sequence Numbering)

Andy

Like with PWs (since DETNET for MPLS would just be another PW type), targeted LDP and provisioning (via a central controller, NMS, etc.) are alternatives to RSVP and SR to provisioning/allocating the DETNET PW label.

You and Mach are both right that in the IP (non-MPLS) case, we need a way for the NP to detect DETNET flows. We could have a new IP protocol type for DETNET, and a DETNET header directly follows the IP header. In that case, the DETNET header would also need a next protocol identifier, unless the next protocol is always another IP header.

Last I saw, the thinking was that there would be an IP extension header, although I am not a fan of those since (a) I am not confident what a router will do one it does not understand, and (b) I am concerned about the need to support IPv4 which I suspect will have a long tail in enterprise deployment.

I prefer a UDP type for this purpose since it has universal support through any middle box. firewall etc, and all routers need to look a the UDP DP so checking if it is one they need to work on is just one compare and conditional branch.

- Stewart

Cheers,
Andy

On Tue, Jan 16, 2018 at 5:46 AM, Stewart Bryant <[email-masked]<mailto:[email-masked]>> wrote:

If we need to discriminate at the PSN layer rather than the DN layer then we really are forced to create a new FEC type for MPLS, or synthesize one through RSVP or SR. IP has no native support for this, but we could introduce the concept, for example by making this the property of the IP address or some way of MPLSing IP.

However there is a deeper question we need to answer. Is our expectation that each hop in the data plane DN aware, or is our expectation that at lest in some networks the data plane is generic with some DN nodes strategically placed?

I can see than in some greenfield specialist cases the routers will all be DN aware, but in order to get deployment in most networks we need to design for the case where only strategic nodes are DN aware.

- Stewart

Any other thoughts?

Best regards,
Mach

From: detnet [mailto:[email-masked]] On Behalf Of Balázs Varga A
Sent: Monday, January 15, 2018 9:06 PM
To: DetNet WG <[email-masked]><mailto:[email-masked]>
Subject: [Detnet] DP solution (Sequence Numbering)

Hi,

      0                   1                   2                   3

Bit5-15: reserved.

RFC4448 states that reserved bits “MUST be ignored upon receipt”, so backward

#3: Other … ?!?

Comments are welcome.

Cheers

Bala'zs

-----Original Message-----
From: detnet [mailto:[email-masked]] On Behalf Of Lou Berger
Sent: 2018. január 10. 19:01
To: DetNet WG <[email-masked]<mailto:[email-masked]>>
Subject: [Detnet] recording for today's interim

https://ietf.webex.com/ietf/ldr.php?RCID=e5116bc7db254c24a7bb3217bf202301<https://urldefense.proofpoint.com/v2/url?u=https-3A__ietf.webex.com_ietf_ldr.php-3FRCID-3De5116bc7db254c24a7bb3217bf202301&d=DwMDaQ&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=1iXUtDOrCnkeGR2oliK0f2OS-pxHY8s2kn4ANa3aBdY&s=q1XpxAraOgLHA2XkFCtkOzf7sfs66FA5jlrpEuPouLo&e=>

http://etherpad.tools.ietf.org:9000/p/notes-ietf-interim-2018-detnet-01<https://urldefense.proofpoint.com/v2/url?u=http-3A__etherpad.tools.ietf.org-3A9000_p_notes-2Dietf-2Dinterim-2D2018-2Ddetnet-2D01&d=DwMDaQ&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=1iXUtDOrCnkeGR2oliK0f2OS-pxHY8s2kn4ANa3aBdY&s=ahg40Qs9nTFT_kpU9_ucD-7yBKcjHeZYdvQzh_h0UT0&e=>

Lou (and co-chairs)

detnet mailing list

https://www.ietf.org/mailman/listinfo/detnet<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_detnet&d=DwMDaQ&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=1iXUtDOrCnkeGR2oliK0f2OS-pxHY8s2kn4ANa3aBdY&s=dp30SUzobcB1YGtwty21KV_efynNObbcHkmAWhnbw9U&e=>

detnet mailing list

https://www.ietf.org/mailman/listinfo/detnet<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_detnet&d=DwMDaQ&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=1iXUtDOrCnkeGR2oliK0f2OS-pxHY8s2kn4ANa3aBdY&s=dp30SUzobcB1YGtwty21KV_efynNObbcHkmAWhnbw9U&e=>

_______________________________________________
detnet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/detnet<https://urldefense.proofpoint.com/v2/url?u=https-3A__www.ietf.org_mailman_listinfo_detnet&d=DwMDaQ&c=lI8Zb6TzM3d1tX4iEu7bpg&r=ZcHC6wX_gDwPDcfMaFNZiQ&m=1iXUtDOrCnkeGR2oliK0f2OS-pxHY8s2kn4ANa3aBdY&s=dp30SUzobcB1YGtwty21KV_efynNObbcHkmAWhnbw9U&e=>

---

From: Jouni <[email-masked]>
Date: Wed, 17 Jan 2018 01:05:34 +0200

> On 16 Jan 2018, at 20:03, Grossman, Ethan A. <[email-masked]> wrote:
> 

This is (still) my understanding of the scope. My expectation is that there is no difference for DetNEt traffic.

- Jouni

> On Tue, Jan 16, 2018 at 5:46 AM, Stewart Bryant <[email-masked]> wrote:

---

From: Mach Chen <[email-masked]>
Date: Wed, 17 Jan 2018 02:28:32 +0000

Hi Stewart,

Some responses inline...

> From: Mach Chen
> Sent: Wednesday, January 17, 2018 9:51 AM
> To: Mach Chen <[email-masked]>
> Subject: FW: [Detnet] DP solution (Sequence Numbering)
> 
> 
> 
> From: Stewart Bryant [mailto:[email-masked]]
> Sent: Tuesday, January 16, 2018 6:47 PM
> To: Mach Chen <[email-masked]>; Balázs Varga A
> <[email-masked]>; DetNet WG <[email-masked]>
> If we need to discriminate at the PSN layer rather than the DN layer then we

PSN layer may not be needed for some cases, for example, the latency guarantee case, all nodes along a path have to be DN aware, for each segment between two adjacent nodes, PSN layer is not necessary. 

> really are forced to create a new FEC type for MPLS, or synthesize one through
> RSVP or SR. IP has no native support for this, but we could introduce the
> concept, for example by making this the property of the IP address or some
> way of MPLSing IP.
> However there is a deeper question we need to answer. Is our expectation that
> each hop in the data plane DN aware, or is our expectation that at lest in some
> networks the data plane is generic with some DN nodes strategically placed?

IMHO, for latency guarantee case, all nodes along a path have to be DN aware, for reliability only case,  only the Replication/Elimination nodes have to be DN aware.

> I can see than in some greenfield specialist cases the routers will all be DN
> aware, but in order to get deployment in most networks we need to design for
> the case where only strategic nodes are DN aware.

Yes, but it depends the requirements,  strategic nodes deployment is possible for "reliability only" case, but may not be for latency case. 

Best regards,
Mach

> In addition, I think we need to define a way on how to differentiate a DetNet
> flow from a non-DetNet flow, then the network processor can process the CW
> as expected.  In TSN, it uses the priority to determine whether a flow is TSN or
> non-TSN flow. For DetNet, should we also use the same way (e.g., TC, DSCP)?
> Or the DetNet service Label itself implies the situation, but it seems that this
> only works when the DetNet service label is the top label.
> 
> Any other thoughts?
> 
> Best regards,
> Mach
> 
> RFC4448 states that reserved bits "MUST be ignored upon receipt", so
> backward compatibility is ensured. Such a solution would allow also to use
> longer seq.num.
> DetNet flows are recognized and forwarded special within the DetNet domain.
> S/N is used within the DetNet domain by DetNet Service functions (i.e. PREF).
> S/N used only on DetNet end-nodes and DetNet relay-nodes. No other nodes
> care about the value of S/N, therefore DetNet can specify its own sequence
> number processing algorithm. So "0" can be a valid number and not the
> indication that no Seq.Num. is used. This is inline with RFC4385.
> 
> #3: Other . ?!?
> 
> Comments are welcome.
> 
> Cheers
> Bala'zs
>

---

From: Mach Chen <[email-masked]>
Date: Wed, 17 Jan 2018 03:36:52 +0000

Hi Ethan,

> From: Grossman, Ethan A. [mailto:[email-masked]]
> Sent: Wednesday, January 17, 2018 2:04 AM
> To: Stewart Bryant <[email-masked]>; Andrew G. Malis
> <[email-masked]>
> Cc: DetNet WG <[email-masked]>; Mach Chen <[email-masked]>;
> Balázs Varga A <[email-masked]>
> Subject: RE: [Detnet] DP solution (Sequence Numbering)
> 
> 1) Regarding whether every node in the network is required to be DetNet
> aware, I was assuming that it would be like AVB/TSN, in which you can only
> make a stream reservation through a path of nodes that are TSN aware, and
> traffic that had to cross non-aware nodes would have to be treated as non-TSN
> traffic, i.e. some packets could get dropped and there would not be timing
> guarantees, but at least they wouldn’t be dropped wholesale due to massive
> protocol incompatibility.  Is that the expectation here or is there a significant
> difference?

I have the same understanding here, for non-DN aware node, there should be no difference between DetNet traffic and non-DetNet traffic.

> 2) I agree that we need to have some kind of strategy for supporting IPv4, for
> example providing at least DetNet-type delay and jitter bounds, even if it had
> some limitations compared to MPLS/IPv6 (e.g. no PREF and thus limited
> reliability).

The proposed DN layer is mainly for PREF, if just for delay/jitter bounds, it should be no problem to support IPv4, IMHO.

Best regards,
Mach
> Cc: DetNet WG <[email-masked]>; Mach Chen <[email-masked]>;
> Balázs Varga A <[email-masked]>
> You and Mach are both right that in the IP (non-MPLS) case, we need a way for
> the NP to detect DETNET flows. We could have a new IP protocol type for
> DETNET, and a DETNET header directly follows the IP header. In that case, the
> DETNET header would also need a next protocol identifier, unless the next
> protocol is always another IP header.
> 
> although I am not a fan of those since (a) I am not confident what a router will
> do one it does not understand, and (b) I am concerned about the need to
> support IPv4 which I suspect will have a long tail in enterprise deployment.
> 
> I prefer a UDP type for this purpose since it has universal support through any
> middle box. firewall etc, and all routers need to look a the UDP DP so checking
> if it is one they need to work on is just one compare and conditional branch.
> 
> 
> number processing algorithm. So “0” can be a valid number and not the

---

From: Stewart Bryant <[email-masked]>
Date: Wed, 17 Jan 2018 08:40:45 +0000

> This is (still) my understanding of the scope. My expectation is that there is no difference for DetNEt traffic.

The consequence of that is that where we are just worried about reliable 
delivery and not transit time we are unable to make use of existing 
infrastructure by strategic upgrades to existing networks.

By making this statement we say that we expect all nodes to be able to 
support all features and all DN paths to be traffic engineered even over 
the WAN. Is that correct?

- Stewart

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Wed, 17 Jan 2018 09:47:53 +0000

Hello Stewart

I’m with you on this. Unless every node is DetNet aware including all the stack at the end points there will be a time where a non DetNet node feeds a DetNet node.

This may happen at the Interface between a legacy host or within the network if non-DetNet routers are deployed as you point out. If we do nothing a tight ingress filtering will drop packets and render the service unusable.

Bottom line is that the ingress of the DetNet pipe after that node must accommodate for flows that behave outside of the tight boundaries of the flow. Maybe more queuing capacity before offending packets are dropped and maybe a larger end to end guaranteed worst case latency. 

This problem can probably be tackled outside the DP work since it impacts mostly the UNI and the guarantees. This is why I started the « transport «  draft. Would you be interested in contributing ?

Regards,

Pascal

> Le 17 janv. 2018 à 09:41, Stewart Bryant <[email-masked]> a écrit :
> 
> 
> 
> The consequence of that is that where we are just worried about reliable delivery and not transit time we are unable to make use of existing infrastructure by strategic upgrades to existing networks.
> 
> By making this statement we say that we expect all nodes to be able to support all features and all DN paths to be traffic engineered even over the WAN. Is that correct?
> 
> - Stewart
> 
>> 
>> - Jouni
>> 
>>

---

From: Loa Andersson <[email-masked]>
Date: Fri, 19 Jan 2018 13:10:37 +0800

Folks,

Thanks for the discussion on sequenced number, where they are used and
what size they need to be.

There seems to a few ideas floating around.

This mail have #1 has a CW of 32 bits and a sequence number 16 bits.

#2 has a CW of 32 bits (?) and a sequence number of 27 bits if the D-bit
is thare (?), right?

The draft-ietf-detnet-dp-sol has a CW of 32 bits and a sequence number 
of 28 bits.

Other ideas I've heard in discussion is to make the sequence number
configurable.

Questions:

It has been said that that if anyone can give a use case where we need a
sequence number bigger than 28 bits, we'll have to accommodate this. Is
the jury still out on this, or can we say that 28 bits is enough?? Would
27 bit be enough? 16 bits?? Or might we even need a two byte CW?

There is not much idea to start drawing figures of the CW unless we have
an answer to that question.

Honestly I don't have to much idea on ho to go about to get that answer.
Any ideas? I think that the dt had some math on this in an early stage.
Anyone that have that available?

On the D-bit. Is it enough to have the D-bit set to determine if it is
a DetNet flow? What if the D-bit is not set?

/Loa

-- 

Loa Andersson                        email: [email-masked]
Senior MPLS Expert
Huawei Technologies (consultant)     phone: +46 739 81 21 64

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 19 Jan 2018 07:25:32 -0500

Loa,

We really don’t need a “D” bit, as we can allocate a PW type to indicate
that this is a DetNet PW, plus the context of the label can identify its
use for a DetNet PW/flow.

We DO need the first nibble to be zero. But beyond that, we are free to use
all 28 bits as a sequence number, if there’s no need for either flags or
PW-layer fragmentation, which would require two bits.

The key to determine the required number of bits in the sequence number is
to determine, for the maximum expected rate of a DetNet flow, how long will
it take to wrap around?

For example, for a 10Gbps flow, that would reasonably equate to 833K pps,
assuming 1500-byte packets. This would wrap around in a little over one
second with 20 bits. A 100 Gbps flow with a 27-bit sequence number would
wrap around in about 1 second.

All that said, a second is a very long holding time for flows with
constrained latency, so we could probably take two bits off those numbers
and still be OK (wrap around in 250 ms).

So 25 bits is probably sufficient, at least for today. 28 bits gives us
room for further BW growth in the future, approaching 1Tbps.

Cheers,
Andy

On Fri, Jan 19, 2018 at 12:10 AM, Loa Andersson <[email-masked]> wrote:

> The draft-ietf-detnet-dp-sol has a CW of 32 bits and a sequence number of
> 28 bits.
>

---

From: Loa Andersson <[email-masked]>
Date: Sat, 20 Jan 2018 00:29:55 +0800

Andy,

> use for a DetNet PW/flow.

Fine, however we have at least diverging opinions, that both say that we 
should not use PWs
- first we have people that believe that we should use EVPN instead
- second there are people that say that we should define what is
   necessary for DetNet and then specify that independent of EVPN and PWs

We need a consensus on that before we can start the design according to
one or the other of those paradigms.
> 
> We DO need the first nibble to be zero. But beyond that, we are free to 
> use all 28 bits as a sequence number, if there’s no need for either 
> flags or PW-layer fragmentation, which would require two bits.
> 
> The key to determine the required number of bits in the sequence number 
> is to determine, for the maximum expected rate of a DetNet flow, how 
> long will it take to wrap around?

Isn't that only part of the truth? We are not concerned only by the fact 
that the source might sue the same sequence number again, but that a 
struggling packet might show up at a point in the network where a wrap
around just happened.
> 
> For example, for a 10Gbps flow, that would reasonably equate to 833K 
> pps, assuming 1500-byte packets. This would wrap around in a little over 
> one second with 20 bits. A 100 Gbps flow with a 27-bit sequence number 
> would wrap around in about 1 second.
> 
> constrained latency, so we could probably take two bits off those 
> numbers and still be OK (wrap around in 250 ms).
> 

I think that we need to design for Tbps (or worse), but that is another
thing we need consensus on before starting to specify.

/Loa
> 
> Cheers,
> Andy
> 
> 
> 
> On Fri, Jan 19, 2018 at 12:10 AM, Loa Andersson <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     Folks,
> 
>     The draft-ietf-detnet-dp-sol has a CW of 32 bits and a sequence
>     number of 28 bits.
> 
>                 0                   1                  
>         2                   3
>         [email-masked] <mailto:[email-masked]> <mailto:[email-masked]
>         <mailto:[email-masked]>>
> 
>     Loa Andersson                        email: [email-masked] <mailto:[email-masked]>
>     Senior MPLS Expert
>     <tel:%2B46%20739%2081%2021%2064>
> 
> 

--

---

From: "Andrew G. Malis" <[email-masked]>
Date: Fri, 19 Jan 2018 13:36:43 -0500

Loa,

You’re right, we do need to think about the maximum lifetime of an errant
packet in the network as well, thanks for that.

You’re also correct that we need to define what’s optimal for DETNET,
however we do have a charter constraint. To quote Lou’s recent email, "We
are not expected to come up with new encapsulations, but rather how DetNet
uses what exists today. “. So we can’t be TOO creative ...

I’m really interested to hear what other people have to say about the upper
limit of flow BW that DetNet needs to support, and I also made an
assumption in my math that an average DetNet packet (at least for a very
large flow) would be 1500 bytes. Do we have consensus on that as well? That
also has implications for the sequence number length.

Cheers,
Andy

On Fri, Jan 19, 2018 at 11:29 AM, Loa Andersson <[email-masked]> wrote:

>> use all 28 bits as a sequence number, if there’s no need for either flags
>> or PW-layer fragmentation, which would require two bits.
>>
>> is to determine, for the maximum expected rate of a DetNet flow, how long
>> will it take to wrap around?
>>
>

---

From: Loa Andersson <[email-masked]>
Date: Sun, 21 Jan 2018 14:28:34 +0800

Andy, all,

(I thought I sent a response to this, but it never did show up, so 
re-sending. Apologize if there are duplicates).

So we all agree that we need the sequence number for elimination
purposes.

We still need to agree on the size and format of the sequence number

What will impact the calculation of the sequence number size are:

- the maximum sustained DetNet flow BW
- the minimum average packet size of a DetNet flow
- the maximum lifetime of an errant packet

My guesstimate is that we need to allow for a sustained BW of 1 Tbs, 
that the average packet size is less that 1500 bytes and that the life
time of an errant packet is at least 500 ms.

But we obviously need better answers than that to be able to calculate 
the sequence number size we med.

Any takers??

/Loa

On 2018-01-20 02:36, Andrew G. Malis wrote:
> Loa,
> 
> You’re right, we do need to think about the maximum lifetime of an 
> errant packet in the network as well, thanks for that.
> 
> however we do have a charter constraint. To quote Lou’s recent email, 
> "We are not expected to come up with new encapsulations, but rather how 
> DetNet uses what exists today. “. So we can’t be TOO creative ...
> 
> I’m really interested to hear what other people have to say about the 
> upper limit of flow BW that DetNet needs to support, and I also made an 
> large flow) would be 1500 bytes. Do we have consensus on that as well? 
> That also has implications for the sequence number length.
> 
> Cheers,
> Andy
> 
> On Fri, Jan 19, 2018 at 11:29 AM, Loa Andersson <[email-masked] 
>         We really don’t need a “D” bit, as we can allocate a PW type to
>         indicate that this is a DetNet PW, plus the context of the label
>         can identify its use for a DetNet PW/flow.
> 
> 
>     Fine, however we have at least diverging opinions, that both say
>     that we should not use PWs
>         We DO need the first nibble to be zero. But beyond that, we are
>         free to use all 28 bits as a sequence number, if there’s no need
>         for either flags or PW-layer fragmentation, which would require
>         two bits.
> 
>         The key to determine the required number of bits in the sequence
>         number is to determine, for the maximum expected rate of a
>         DetNet flow, how long will it take to wrap around?
> 
> 
>     Isn't that only part of the truth? We are not concerned only by the
>     fact that the source might sue the same sequence number again, but
>     that a struggling packet might show up at a point in the network
>     where a wrap
>     around just happened.
> 
> 
>         For example, for a 10Gbps flow, that would reasonably equate to
>         833K pps, assuming 1500-byte packets. This would wrap around in
>         a little over one second with 20 bits. A 100 Gbps flow with a
>         27-bit sequence number would wrap around in about 1 second.
> 
>         All that said, a second is a very long holding time for flows
>         with constrained latency, so we could probably take two bits off
>         those numbers and still be OK (wrap around in 250 ms).
> 
>         So 25 bits is probably sufficient, at least for today. 28 bits
>         gives us room for further BW growth in the future, approaching
>         1Tbps.
> 
> 

--

---

From: Jouni <[email-masked]>
Date: Sun, 21 Jan 2018 11:43:42 +0200

Quick and dirty calculation with 1500 octet packet (+ ethernet overhead of 42 octets) at 400Gb/s link a 25 bit SN is able to hold more than 1 sec of traffic. With 256 octet packet 1 sec would need 28 bit SN.
1Tb/s with 512 octet packet (+ethernet overhead) for 1 sec would need 28 bit SN. This is based on the calculation how many packets you are able to inject within a given time constraint.  I do not consider here yet how you would find out a late packet based on the SN i.e., whether you need to reserve half of the SN space for “in window” packets or some other amount like 10%. Anyway, it is still _early_ in Sunday morning so tolerate errors..

To me 28 bits seems to be plenty.. and a single on-wire format is simple (assuming the actual SN bits are defined by the CP or config).

- Jouni

> On 21 Jan 2018, at 08:28, Loa Andersson <[email-masked]> wrote:
> 
> Andy, all,
> 
> (I thought I sent a response to this, but it never did show up, so re-sending. Apologize if there are duplicates).
> 
> My guesstimate is that we need to allow for a sustained BW of 1 Tbs, that the average packet size is less that 1500 bytes and that the life
> But we obviously need better answers than that to be able to calculate the sequence number size we med.
> 
> Any takers??
> 
> /Loa
> 
> 
> 
>> You’re right, we do need to think about the maximum lifetime of an errant packet in the network as well, thanks for that.
>> You’re also correct that we need to define what’s optimal for DETNET, however we do have a charter constraint. To quote Lou’s recent email, "We are not expected to come up with new encapsulations, but rather how DetNet uses what exists today. “. So we can’t be TOO creative ...
>> I’m really interested to hear what other people have to say about the upper limit of flow BW that DetNet needs to support, and I also made an assumption in my math that an average DetNet packet (at least for a very large flow) would be 1500 bytes. Do we have consensus on that as well? That also has implications for the sequence number length.
>> Cheers,
>> Andy
>> On Fri, Jan 19, 2018 at 11:29 AM, Loa Andersson <[email-masked] <mailto:[email-masked]>> wrote:
>>    Andy,

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 22 Jan 2018 11:01:14 +0000

We can of course do the s/n triage in two phases. Approximate in the 
middle of the middle of the n/w and exact at the T-PE.

- Stewart

On 19/01/2018 16:29, Loa Andersson wrote:
>> indicate that this is a DetNet PW, plus the context of the label can 
>> identify its use for a DetNet PW/flow.
>
> Fine, however we have at least diverging opinions, that both say that 
> we should not use PWs
>> We DO need the first nibble to be zero. But beyond that, we are free 
>> to use all 28 bits as a sequence number, if there’s no need for 
>> either flags or PW-layer fragmentation, which would require two bits.
>>
>> number is to determine, for the maximum expected rate of a DetNet 
>> flow, how long will it take to wrap around?
>
> that a struggling packet might show up at a point in the network where 
> a wrap
> around just happened.
>>
>> pps, assuming 1500-byte packets. This would wrap around in a little 
>> over one second with 20 bits. A 100 Gbps flow with a 27-bit sequence 
>> number would wrap around in about 1 second.
>>
>> So 25 bits is probably sufficient, at least for today. 28 bits gives 
>> us room for further BW growth in the future, approaching 1Tbps.
>
>> <mailto:[email-masked]>

---

From: Lou Berger <[email-masked]>
Date: Mon, 22 Jan 2018 09:16:30 -0500

Jouni,

     I may be off but isn't sequence number independent per flow? So 
it's not the max link speed that's critical, but rather the max flow 
size, right?

Certainly link speed provides an upper bound on a single flow, and there 
are certainly some use cases where a single flow will consume even the 
biggest link, but my experience is that such flows *always* use large 
packets.  In any case, I (personally, no hat) don't see this as the 
common case nor *necessarily* the required *initial* detnet solution.

Also, I ran sum numbers for bits needed for 1sec at different frame 
sizes.  25 bits looks sufficient for almost all flows, and is only 
insufficient for 4 smaller frame sizes at 400G and 1T. So in either case 
(flows or links)  28 bits  more than sufficient.

Lou

Hopefully this looks okay...

bits needed 	64 BPkt	128 BPkt	256 BPkt	512 BPkt	1514 BPkt	4096 BPkt	9216 
BPkt
10M 	14 	13 	12 	11 	10 	8 	7
1G 	21 	20 	19 	18 	16 	15 	14
100G 	27 	26 	25 	24 	23 	22 	20
400G 	29 	28 	27 	26 	25 	24 	22
1T 	30 	30 	29 	28 	26 	25 	24

---

From: Balázs Varga A <[email-masked]>
Date: Mon, 22 Jan 2018 14:40:36 +0000

Hi,

I would like to highlight two aspects to S/N size calculations:

1, Path delay differences.
We use the S/N to identify duplicates at “path merging points” (where Duplicate
Elimination functions resides in the network).

So what really matters is the delay difference of the paths. And the “window” size
we use at the Elimination point (see next point). So if the path difference is e.g.,
10 ms than we need a window size to be able to cope with packets received
during that time period.

I think path differences should be well below 1 sec. In many use-cases even 1 ms
is too much.

2, Window size (used for S/N comparison)
The other constraint we should consider more is how big window size the chips
are able to deal with. It is a story e.g., about memory …

Cheers
Bala’zs

From: Lou Berger [mailto:[email-masked]]
Sent: 2018. január 22. 15:17
To: Jouni <[email-masked]>; Loa Andersson <[email-masked]>
Cc: DetNet WG <[email-masked]>; Andrew G. Malis <[email-masked]>; Balázs Varga A <[email-masked]>
Subject: Re: [Detnet] DP solution (Sequence Numbering)

Jouni,

    I may be off but isn't sequence number independent per flow?  So it's not the max link speed that's critical, but rather the max flow size, right?

Certainly link speed provides an upper bound on a single flow, and there are certainly some use cases where a single flow will consume even the biggest link, but my experience is that such flows *always* use large packets.  In any case, I (personally, no hat) don't see this as the common case nor *necessarily* the required *initial* detnet solution.
Also, I ran sum numbers for bits needed for 1sec at different frame sizes.  25 bits looks sufficient for almost all flows, and is only insufficient for 4 smaller frame sizes at 400G and 1T. So in either case (flows or links)  28 bits  more than sufficient.

Lou

bits needed

64 BPkt

 128 BPkt

 256 BPkt

 512 BPkt

 1514 BPkt

 4096 BPkt

 9216 BPkt

10M

14

13

12

11

10

8

7

1G

21

20

19

18

16

15

14

100G

27

26

25

24

23

22

20

400G

29

28

27

26

25

24

22

1T

30

30

29

28

26

25

24

- Jouni

On 21 Jan 2018, at 08:28, Loa Andersson <[email-masked]><mailto:[email-masked]> wrote:

Andy, all,

purposes.

Any takers??

/Loa

Loa,

Cheers,

Andy

On Fri, Jan 19, 2018 at 11:29 AM, Loa Andersson <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]><mailto:[email-masked]>> wrote:

   Andy,

       Loa,

      necessary for DetNet and then specify that independent of EVPN

   and PWs

       two bits.

   where a wrap

   around just happened.

       1Tbps.

   /Loa

       Cheers,

       Andy

detnet mailing list

--

Loa Andersson                        email: [email-masked]<mailto:[email-masked]>

Senior MPLS Expert

detnet mailing list

detnet mailing list

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 22 Jan 2018 15:26:07 +0000

> are able to deal with. It is a story e.g., about memory …

Well partially a matter of memory and partially a matter of calculation.

If you have a small window in a big s/n you can run a number of tricks 
such as a rotating bit mask and a single full function counter to 
identify say the start.

I think we need to first understand what we would like, then see if 
building is is plausible.

- Stewart

---

From: "Jouni" <[email-masked]>
Date: Mon, 22 Jan 2018 17:35:33 +0200

Hi,

I was just doing the worst case since 1Tb links got into discussion i.e. one flow takes over the entire capacity.

-          Jouni

Sent: Monday, January 22, 2018 16:17 PM

Jouni,

Lou

bits needed

64 BPkt 

 128 BPkt 

 256 BPkt 

 512 BPkt 

 1514 BPkt 

 4096 BPkt 

 9216 BPkt 

10M

14

13

12

11

10

8

7

1G

21

20

19

18

16

15

14

100G

27

26

25

24

23

22

20

400G

29

28

27

26

25

24

22

1T

30

30

29

28

26

25

24

 
 
- Jouni

On 21 Jan 2018, at 08:28, Loa Andersson  <mailto:[email-masked]> <[email-masked]> wrote:
 
Andy, all,

 

 

Any takers??
 
/Loa

Loa,
On Fri, Jan 19, 2018 at 11:29 AM, Loa Andersson <[email-masked] <mailto:[email-masked]>   <mailto:[email-masked]> <mailto:[email-masked]>> wrote:

--

---

From: "Jouni" <[email-masked]>
Date: Mon, 22 Jan 2018 17:43:05 +0200

The point 2 is exactly what I wanted to stress during the interim call.  E.g., the bitmaps to keep track for packet history window quickly become an issue in a multiport chips. These are high speed lookups so off-chip ram is not typically an option. Anyway, the window size is an implementation/cost matter..

-          Jouni

From: Balázs Varga A [mailto:[email-masked]] 
Sent: Monday, January 22, 2018 16:41 PM
To: Lou Berger <[email-masked]>; Jouni <[email-masked]>; Loa Andersson <[email-masked]>
Cc: DetNet WG <[email-masked]>; Andrew G. Malis <[email-masked]>
Subject: RE: [Detnet] DP solution (Sequence Numbering)

Hi,

during that time period.

is too much.

Cheers

Bala’zs

From: Lou Berger [mailto:[email-masked]] 
Sent: 2018. január 22. 15:17
To: Jouni <[email-masked] <mailto:[email-masked]> >; Loa Andersson <[email-masked] <mailto:[email-masked]> >
Cc: DetNet WG <[email-masked] <mailto:[email-masked]> >; Andrew G. Malis <[email-masked] <mailto:[email-masked]> >; Balázs Varga A <[email-masked] <mailto:[email-masked]> >
Subject: Re: [Detnet] DP solution (Sequence Numbering)

Jouni,

Lou

bits needed

64 BPkt 

 128 BPkt 

 256 BPkt 

 512 BPkt 

 1514 BPkt 

 4096 BPkt 

 9216 BPkt 

10M

14

13

12

11

10

8

7

1G

21

20

19

18

16

15

14

100G

27

26

25

24

23

22

20

400G

29

28

27

26

25

24

22

1T

30

30

29

28

26

25

24

 
 
- Jouni

Andy, all,

 

 

Any takers??
 
/Loa

Loa,

--

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 22 Jan 2018 16:52:39 +0000

Which is why I keep saying that checking is a system egress function.

- Stewart

On 22/01/2018 15:43, Jouni wrote:
>
> The point 2 is exactly what I wanted to stress during the interim 
> call. E.g., the bitmaps to keep track for packet history window 
> quickly become an issue in a multiport chips. These are high speed 
> lookups so off-chip ram is not typically an option. Anyway, the window 
> size is an implementation/cost matter..
>
> -Jouni
>
> *From:*Balázs Varga A [mailto:[email-masked]]
> *Sent:* Monday, January 22, 2018 16:41 PM
> *To:* Lou Berger <[email-masked]>; Jouni <[email-masked]>; 
> Loa Andersson <[email-masked]>
> *Cc:* DetNet WG <[email-masked]>; Andrew G. Malis <[email-masked]>
> *Subject:* RE: [Detnet] DP solution (Sequence Numbering)
>
> Hi,
>
> *From:*Lou Berger [mailto:[email-masked]]
> *Sent:* 2018. január 22. 15:17
> *To:* Jouni <[email-masked] <mailto:[email-masked]>>; 
> Loa Andersson <[email-masked] <mailto:[email-masked]>>
> *Cc:* DetNet WG <[email-masked] <mailto:[email-masked]>>; Andrew G. 
> Malis <[email-masked] <mailto:[email-masked]>>; Balázs Varga A 
> <[email-masked] <mailto:[email-masked]>>
> *Subject:* Re: [Detnet] DP solution (Sequence Numbering)
>
> Jouni,
>
> Certainly link speed provides an upper bound on a single flow, and 
> there are certainly some use cases where a single flow will consume 
> even the biggest link, but my experience is that such flows *always* 
> use large packets.  In any case, I (personally, no hat) don't see this 
> as the common case nor *necessarily* the required *initial* detnet 
> solution.
>
> insufficient for 4 smaller frame sizes at 400G and 1T. So in either 
> case (flows or links)  28 bits  more than sufficient.
>
> Lou
>
>         On 21 Jan 2018, at 08:28, Loa Andersson<[email-masked]> <mailto:[email-masked]>  wrote:
>
>         Andy, all,
>
>         Loa Andersson                        email:[email-masked] <mailto:[email-masked]>
>
>         Senior MPLS Expert
>

---

From: "Jouni" <[email-masked]>
Date: Mon, 22 Jan 2018 19:52:36 +0200

The current architecture does not prohibit deployment models where PREF takes place at the ingress/egress only. We need a solution for rings and ladders  where every node basically can participate PREF.

-          Jouni

From: Stewart Bryant [mailto:[email-masked]] 
Sent: Monday, January 22, 2018 18:53 PM
To: Jouni <[email-masked]>; 'Balázs Varga A' <[email-masked]>; 'Lou Berger' <[email-masked]>; 'Loa Andersson' <[email-masked]>
Cc: 'DetNet WG' <[email-masked]>; 'Andrew G. Malis' <[email-masked]>
Subject: Re: [Detnet] DP solution (Sequence Numbering)

- Stewart

-          Jouni

From: Balázs Varga A [mailto:[email-masked]] 
Sent: Monday, January 22, 2018 16:41 PM
To: Lou Berger  <mailto:[email-masked]> <[email-masked]>; Jouni  <mailto:[email-masked]> <[email-masked]>; Loa Andersson  <mailto:[email-masked]> <[email-masked]>
Cc: DetNet WG  <mailto:[email-masked]> <[email-masked]>; Andrew G. Malis  <mailto:[email-masked]> <[email-masked]>
Subject: RE: [Detnet] DP solution (Sequence Numbering)

Hi,

during that time period.

is too much.

Cheers

Bala’zs

Jouni,

Lou

bits needed

64 BPkt 

 128 BPkt 

 256 BPkt 

 512 BPkt 

 1514 BPkt 

 4096 BPkt 

 9216 BPkt 

10M

14

13

12

11

10

8

7

1G

21

20

19

18

16

15

14

100G

27

26

25

24

23

22

20

400G

29

28

27

26

25

24

22

1T

30

30

29

28

26

25

24

 
 
- Jouni

Andy, all,

 

 

Any takers??
 
/Loa

Loa,

--

---

From: Stewart Bryant <[email-masked]>
Date: Mon, 22 Jan 2018 18:14:00 +0000

No, I meant that in as far as we have them, the equipment model should 
be ingress rep, egress elim.

- Stewart

On 22/01/2018 17:52, Jouni wrote:
>
> The current architecture does not prohibit deployment models where 
> PREF takes place at the ingress/egress only. We need a solution for 
> rings and ladders  where every node basically can participate PREF.
>
> -Jouni
>
> *From:*Stewart Bryant [mailto:[email-masked]]
> *Sent:* Monday, January 22, 2018 18:53 PM
> *To:* Jouni <[email-masked]>; 'Balázs Varga A' 
> <[email-masked]>; 'Lou Berger' <[email-masked]>; 'Loa 
> Andersson' <[email-masked]>
> *Cc:* 'DetNet WG' <[email-masked]>; 'Andrew G. Malis' <[email-masked]>
>     lookups so off-chip ram is not typically an option. Anyway, the
>     window size is an implementation/cost matter..
>
>     -Jouni
>
>     *To:* Lou Berger <[email-masked]> <mailto:[email-masked]>;
>     Jouni <[email-masked]> <mailto:[email-masked]>;
>     Loa Andersson <[email-masked]> <mailto:[email-masked]>
>     *Cc:* DetNet WG <[email-masked]> <mailto:[email-masked]>; Andrew
>     G. Malis <[email-masked]> <mailto:[email-masked]>
>     *To:* Jouni <[email-masked]
>     <mailto:[email-masked]>>; Loa Andersson <[email-masked]
>     <mailto:[email-masked]>>
>     *Cc:* DetNet WG <[email-masked] <mailto:[email-masked]>>; Andrew
>     G. Malis <[email-masked] <mailto:[email-masked]>>; Balázs
>     Varga A <[email-masked]
>     <mailto:[email-masked]>>
>         I may be off but isn't sequence number independent per flow? 
>     So it's not the max link speed that's critical, but rather the max
>     flow size, right?
>
>     there are certainly some use cases where a single flow will
>     consume even the biggest link, but my experience is that such
>     flows *always* use large packets.  In any case, I (personally, no
>     hat) don't see this as the common case nor *necessarily* the
>     required *initial* detnet solution.
>
>     Also, I ran sum numbers for bits needed for 1sec at different
>     frame sizes.  25 bits looks sufficient for almost all flows, and
>     is only insufficient for 4 smaller frame sizes at 400G and 1T. So
>     in either case (flows or links)  28 bits  more than sufficient.
>
>     Lou
>

---

From: Jouni <[email-masked]>
Date: Mon, 22 Jan 2018 22:52:42 +0200

Ah, right. Sorry for my confusion.

- Jouni

> On 22 Jan 2018, at 20:14, Stewart Bryant <[email-masked]> wrote:
> 
> No, I meant that in as far as we have them, the equipment model should be ingress rep, egress elim.
> 
> - Stewart
> 
>

---

From: Loa Andersson <[email-masked]>
Date: Tue, 23 Jan 2018 15:03:49 +0800

Lou,

tnx, very useful! I've been looking around for a case that stresses the
28 bit sequence number, but can't really find one.

5G seems to require (aggregate) flows of something close to 400 Gbbs,
but a 28 bit sequence number can support that.

The only issue I have now is what Andy called "errant" packet, i.e. a
packet that gets duplicated, where one or more copies takes fast paths
and arrive to the elimination point well before wrap around, but where
one or a few packets gets delayed such that the wrap around happens
before they arrive to the elimination point.

Anyone that cares to address this?

/Loa

On 2018-01-22 22:16, Lou Berger wrote:
>>> On 21 Jan 2018, at 08:28, Loa Andersson<[email-masked]>  wrote:
>>>
>>> Andy, all,
>>>
>>> Loa Andersson                        email:[email-masked]
>>> Senior MPLS Expert

--