Subject: Re: [manet] DLEP Credit Windowing

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 10:42:53 +0000

Hello Stan,

We have submitted a new draft today to describe the PFC flow control mechanism. Sorry, this was long due on me; but better late than never!

https://datatracker.ietf.org/doc/draft-dalela-dlep-flow-control/

Here is the quick rationale underlying this draft. The current DLEP credit window scheme requires credits exchanged over a TCP session. This is harder to achieve in many scenarios because many data path implementations are incapable of handling TCP windowing. So we are suggesting an alternative approach that utilizes IEEE 802.1Qbb (a.k.a. PFC).

Request comments.

Thanks
-Ashish

From: Stan Ratliff [mailto:[email-masked]]
Sent: Friday, October 16, 2015 2:34 AM
To: Ashish Dalela (adalela) <[email-masked]>
Cc: Henning Rogge <[email-masked]>; [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Ashish,

If you'd like to write a proposed DLEP extension using PFC, I encourage you to do so. As Henning mentioned, more than 1 flow control mechanism is likely for DLEP.

Stan

On Thu, Oct 15, 2015 at 10:49 AM, Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>> wrote:
Yes it will and that's why I suggested the use of PFC. The control and data planes will have different priorities. The radio never pauses the control channel because it is never sent over the air. If desired,  the data packets themselves could be given different priorities allowing some flows to be selectively paused.

Sent from Samsung Mobile

-------- Original message --------
From: Henning Rogge
Date:15/10/2015 18:36 (GMT+05:30)
To: "Ashish Dalela (adalela)"
Cc: [email-masked]<mailto:[email-masked]>
Subject: Re: [manet] DLEP Credit Windowing
On Thu, Oct 15, 2015 at 2:25 PM, Ashish Dalela (adalela)
<[email-masked]<mailto:[email-masked]>> wrote:
> The ideal mechanism for Ethernet flow control is the use of 802.3 PAUSE
> frames, which has further been enhanced by the PFC (802.1Qbb) mechanism. In
> essence, we don’t send credits before packets are forwarded. Rather we pause
> the traffic when the traffic cannot be forwarded and the buffers are full.
> PFC is widely used in a number of Ethernet flow-control scenarios today.

Would a 802.3 PAUSE Frame not also block the DLEP control channel?

Henning Rogge

_______________________________________________
manet mailing list
[email-masked]<mailto:[email-masked]>
https://www.ietf.org/mailman/listinfo/manet

---

From: Rick Taylor <[email-masked]>
Date: Fri, 4 Mar 2016 11:47:12 +0000

Hi Ashish,

Thank you for this, it looks extremely interesting.

Obviously I haven't had a chance to completely digest the draft, but one 
thing that leaps out to me is the use of VLAN tagging, specifically 
section 7.  Are you suggesting that the DLEP session runs over a VLAN 
trunk between router and modem (somehow) and the VLANs then fan out at 
the modem in some way?

Given the 'transparent layer-2' requirement of DLEP, how do you see this 
working with your model of a VLAN per destination (for flow control), or 
have I misunderstood what you are proposing?

Regards

Rick

On 04/03/16 10:43, Ashish Dalela (adalela) wrote:
> Hello Stan,
>
> We have submitted a new draft today to describe the PFC flow control
> mechanism. Sorry, this was long due on me; but better late than never!
>
> Here is the quick rationale underlying this draft. The current DLEP
> credit window scheme requires credits exchanged over a TCP session. This
> is harder to achieve in many scenarios because many data path
> implementations are incapable of handling TCP windowing. So we are
> suggesting an alternative approach that utilizes IEEE 802.1Qbb (a.k.a.
> PFC).
>
> Request comments.
>
> Thanks
>
> -Ashish
>
> *From:*Stan Ratliff [mailto:[email-masked]]
> *Sent:* Friday, October 16, 2015 2:34 AM
> *To:* Ashish Dalela (adalela) <[email-masked]>
> *Cc:* Henning Rogge <[email-masked]>; [email-masked]
> *Subject:* Re: [manet] DLEP Credit Windowing
>
> Ashish,
>
> If you'd like to write a proposed DLEP extension using PFC, I encourage
> you to do so. As Henning mentioned, more than 1 flow control mechanism
> is likely for DLEP.
>
> Stan
>
> On Thu, Oct 15, 2015 at 10:49 AM, Ashish Dalela (adalela)
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     Yes it will and that's why I suggested the use of PFC. The control
>     and data planes will have different priorities. The radio never
>     pauses the control channel because it is never sent over the air. If
>     desired,  the data packets themselves could be given different
>     priorities allowing some flows to be selectively paused.
>
>     Sent from Samsung Mobile
>
>
>
>     Cc: [email-masked] <mailto:[email-masked]>
>     <[email-masked] <mailto:[email-masked]>> wrote:
>     [email-masked] <mailto:[email-masked]>

---

From: Henning Rogge <[email-masked]>
Date: Fri, 4 Mar 2016 12:59:22 +0100

Hi,

I am a bit worried about the text at the beginning of chapter 6.1:

"IEEE 802.1Qbb standards PAUSE frame is re-interpreted to support this
scheme using both Class of Service and MAC."

Does this mean you are using 802.1Qbb in a different way than
suggested in the standard?

Doing so could mean that you need custom ethernet drivers on both
ends... and maybe even a custom switch.

Henning Rogge

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 12:26:11 +0000

Hi Rick,

Yes, one of the proposals is to use a VLAN per neighbor. The default VLAN 1 will be used for the control plane. That allows us to have 4K neighbors, each distinguished by a unique VLAN ID.

Thanks

Ashish

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Friday, March 04, 2016 5:17 PM
To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Ashish,

Obviously I haven't had a chance to completely digest the draft, but one thing that leaps out to me is the use of VLAN tagging, specifically section 7.  Are you suggesting that the DLEP session runs over a VLAN trunk between router and modem (somehow) and the VLANs then fan out at the modem in some way?

Given the 'transparent layer-2' requirement of DLEP, how do you see this working with your model of a VLAN per destination (for flow control), or have I misunderstood what you are proposing?

Regards

Rick

On 04/03/16 10:43, Ashish Dalela (adalela) wrote:
> credit window scheme requires credits exchanged over a TCP session. 
> This is harder to achieve in many scenarios because many data path 
> If you'd like to write a proposed DLEP extension using PFC, I 
> encourage you to do so. As Henning mentioned, more than 1 flow control 
> mechanism is likely for DLEP.
>
> Stan
>
>     > essence, we don't send credits before packets are forwarded. Rather we pause

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 12:31:55 +0000

Hi Henning,

802.1Qbb supports only 8 CoS queues, and in cases where there are no more than 8 neighbors and 1 traffic class per neighbor, the default Qbb mechanism is proposed.  

When we have to go beyond 8 neighbors and/or multiple traffic classes per neighbor, two other schemes are proposed - (1) identifying the neighbor by a unique SMAC, and (2) identifying the neighbor by a unique VLAN.

It is correct that Qbb does not specify the use of SMAC and VLAN as distinguishing factors for traffic classes, and only supports 8 CoS. However, since we don't change the packet format, we consider this "allowed behavior" although it may not be implemented. In some cases, this *might* involve update of Ethernet firmware, or the implementer might choose to update the driver. 

Thanks

Ashish

-----Original Message-----
From: Henning Rogge [mailto:[email-masked]] 
Sent: Friday, March 04, 2016 5:29 PM
To: Ashish Dalela (adalela) <[email-masked]>
Cc: Stan Ratliff <[email-masked]>; [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi,

"IEEE 802.1Qbb standards PAUSE frame is re-interpreted to support this scheme using both Class of Service and MAC."

Does this mean you are using 802.1Qbb in a different way than suggested in the standard?

Doing so could mean that you need custom ethernet drivers on both ends... and maybe even a custom switch.

Henning Rogge

---

From: Rick Taylor <[email-masked]>
Date: Fri, 4 Mar 2016 13:58:27 +0000

Hi Ashish,

Introducing a VLAN per neighbour is actually quite a big change from 
core DLEP.  Destinations (neighbours) in DLEP are distinguished by MACs 
that are MUST be addressable/reachable on the same (V)LAN segment as the 
DLEP peers.

What you are proposing with VLAN-per-destination looks a lot more like 
the draft extension I put forward for layer-3 DLEP 
(https://tools.ietf.org/html/draft-taylor-manet-l3-dlep-00) where I 
proposed the workaround of stating that the MAC address is no longer a 
MAC, just an identifying unique sequence of octets.

In your case the additional VLAN-Id data item paired with the MAC 
becomes the unique destination identifier.

Might I suggest that you actually have 2 extensions in this one draft: 
Per-VLAN destinations, and PFC based flow control?  Maybe splitting the 
document into two might be clearer for the readers?

Cheers,

Rick

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 14:20:49 +0000

Hi Rick,

The usage of the VLAN is limited to that between the router and the modem. It doesn't stretch to neighbors. In other words, the VLAN is not seen on the radio network. It is added by the modem and the router when forwarding packets to each other, and stripped when then packet exits the router (towards another router) or the modem (towards another modem). This clarification can be added to the next version if that would help.

PFC based flow-control only gives us 8 neighbors with one traffic class each. And usage of VLAN alone doesn't imply the use of PFC per VLAN. That's why it is important to combine them. 

Currently VLANs can be used without PFC, and PFC can be used without VLAN. So, separating them will not achieve the key purpose of flow control. 

Thanks
Ashish

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Friday, March 04, 2016 7:28 PM
To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Ashish,

Introducing a VLAN per neighbour is actually quite a big change from core DLEP.  Destinations (neighbours) in DLEP are distinguished by MACs that are MUST be addressable/reachable on the same (V)LAN segment as the DLEP peers.

What you are proposing with VLAN-per-destination looks a lot more like the draft extension I put forward for layer-3 DLEP
(https://tools.ietf.org/html/draft-taylor-manet-l3-dlep-00) where I proposed the workaround of stating that the MAC address is no longer a MAC, just an identifying unique sequence of octets.

In your case the additional VLAN-Id data item paired with the MAC becomes the unique destination identifier.

Might I suggest that you actually have 2 extensions in this one draft: 
Per-VLAN destinations, and PFC based flow control?  Maybe splitting the document into two might be clearer for the readers?

Cheers,

Rick

On 04/03/16 12:26, Ashish Dalela (adalela) wrote:
> To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff 
> <[email-masked]>; Henning Rogge <[email-masked]>
> Cc: [email-masked]
>> encourage you to do so. As Henning mentioned, more than 1 flow 
>> control mechanism is likely for DLEP.
>>
>> Stan
>>

---

From: Rick Taylor <[email-masked]>
Date: Fri, 4 Mar 2016 15:01:39 +0000

Hi Ashish,

Okay, I understand that the VLAN's don't extend over the 'air'.  My 
issue with them is between router and modem - as core DLEP stands, your 
method is not allowed.

However, it can be allowed in the way you describe via an extension, as 
you have already done.  My concern is not "VLANs can be used without 
PFC, and PFC can be used without VLAN", it is that core DLEP and VLANs 
doesn't work.

I would much rather have an extension entitled "VLANs and DLEP" that 
covers the VLAN topic in isolation.  And an updated version of your 
draft covering PFC-based flow control, referring to the VLAN extension 
if VLANs are in use.

Sorry, I don't mean to be negative, the PFC part of your draft seems 
excellent.

Cheers,

Rick

---

From: Henning Rogge <[email-masked]>
Date: Fri, 4 Mar 2016 16:13:08 +0100

> Sorry, I don't mean to be negative, the PFC part of your draft seems

Using VLAN as a "tunnel" to the radio could invite quite a bit of trouble...

There is the question about VLAN-in-VLAN (multiple DLEP radios
attached over VLAN-capable switch to router)...

or even VLAN-in-VLAN-in-VLAN (someone is sending VLAN tagged traffic
over a DLEP-Radio which is attached over a VLAN-capable switch.

Henning Rogge

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 15:51:26 +0000

Hi Rick,

Sure, that can be done. This draft is to seek the inputs and feedback on what is the best path for us. Any and all suggestions are welcome, including "this is a very bad idea" :-)

Thanks

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Friday, March 04, 2016 8:32 PM
To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Ashish,

Okay, I understand that the VLAN's don't extend over the 'air'.  My issue with them is between router and modem - as core DLEP stands, your method is not allowed.

However, it can be allowed in the way you describe via an extension, as you have already done.  My concern is not "VLANs can be used without PFC, and PFC can be used without VLAN", it is that core DLEP and VLANs doesn't work.

I would much rather have an extension entitled "VLANs and DLEP" that covers the VLAN topic in isolation.  And an updated version of your draft covering PFC-based flow control, referring to the VLAN extension if VLANs are in use.

Sorry, I don't mean to be negative, the PFC part of your draft seems excellent.

Cheers,

Rick

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Fri, 4 Mar 2016 16:01:34 +0000

Hi Henning,

So based on the earlier comment by Rick, the use of VLAN is itself incompatible with the current core DLEP philosophy. By implication, VLAN-in-VLAN or VLAN-in-VLAN-VLAN would also be incompatible. 

As Rick suggested, should we do a draft "DLEP and VLANs" that captures these encapsulation scenarios? What type of encapsulation is being used would have to be negotiated during "Neighbor UP". 

The PFC draft (the present one) can then be modified to refer to the above draft. All these will be optional extensions (again as per Rick's suggestion).

I would like to hear from everyone if this is an acceptable approach.

Thanks

-----Original Message-----
From: Henning Rogge [mailto:[email-masked]] 
Sent: Friday, March 04, 2016 8:43 PM
To: Rick Taylor <[email-masked]>
Cc: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

On Fri, Mar 4, 2016 at 4:01 PM, Rick Taylor <[email-masked]> wrote:
> issue with them is between router and modem - as core DLEP stands, 
> your method is not allowed.
>
> However, it can be allowed in the way you describe via an extension, 
> as you have already done.  My concern is not "VLANs can be used 
> without PFC, and PFC can be used without VLAN", it is that core DLEP 
> and VLANs doesn't work.
>

There is the question about VLAN-in-VLAN (multiple DLEP radios attached over VLAN-capable switch to router)...

or even VLAN-in-VLAN-in-VLAN (someone is sending VLAN tagged traffic over a DLEP-Radio which is attached over a VLAN-capable switch.

Henning Rogge

---

From: Lou Berger <[email-masked]>
Date: Fri, 4 Mar 2016 15:31:40 -0500

I'm a fan of both pause and pfc, and could see how it could play a role
here, but I do worry about non-standard usage of 802.1 "stuff".  Have
you considered 802.1 Qau (CN/QCN)?

On 3/4/2016 11:01 AM, Ashish Dalela (adalela) wrote:
> [email-masked]

---

From: "Arun Parashar (arparash)" <[email-masked]>
Date: Tue, 15 Mar 2016 15:40:17 +0000

Hi Rick, Stan,

Per comments, VLAN topic is dealt in separate draft and flow control draft is updated accordingly.

1. VLANs and DLEP draft - https://tools.ietf.org/html/draft-dalela-vlans-and-dlep-01
2. Flow Control draft - https://tools.ietf.org/html/draft-dalela-dlep-flow-control-02

Request comments.

-Arun

-----Original Message-----
From: manet [mailto:[email-masked]] On Behalf Of Ashish Dalela (adalela)
Sent: Friday, March 04, 2016 9:21 PM
To: Rick Taylor <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Rick,

Thanks

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]]
Sent: Friday, March 04, 2016 8:32 PM
To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Ashish,

Cheers,

Rick

---

From: "Arun Parashar (arparash)" <[email-masked]>
Date: Mon, 28 Mar 2016 05:50:09 +0000

Hi All,

Has everyone had chance to read and submit the review comments, we would like to hear comments/concerns before we take these two documents further in the chain.

-Arun

-----Original Message-----
From: Arun Parashar (arparash) 
Sent: Tuesday, March 15, 2016 9:10 PM
To: Rick Taylor <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]; Ashish Dalela (adalela) <[email-masked]>
Subject: RE: [manet] DLEP Credit Windowing

Hi Rick, Stan,

Request comments.

-Arun

-----Original Message-----
From: manet [mailto:[email-masked]] On Behalf Of Ashish Dalela (adalela)
Sent: Friday, March 04, 2016 9:21 PM
To: Rick Taylor <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Rick,

Thanks

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]]
Sent: Friday, March 04, 2016 8:32 PM
To: Ashish Dalela (adalela) <[email-masked]>; Stan Ratliff <[email-masked]>; Henning Rogge <[email-masked]>
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Ashish,

Cheers,

Rick

---

From: Rick Taylor <[email-masked]>
Date: Tue, 29 Mar 2016 12:27:28 +0000

Hi Arun,

I have two concerns regarding your drafts:

1) DLEP isn't finished yet, and so I don't think any extensions will 
proceed until then.

2) I still have worries about the use of VLANs per neighbour.  Given the 
single layer-2 requirement of DLEP, I don't see how this will work.

Will you be in Argentina?  If so, I would be happy to talk over my 
concerns face to face.

Cheers,

Rick

---

From: "Ashish Dalela (adalela)" <[email-masked]>
Date: Tue, 29 Mar 2016 15:12:39 +0000

Hello Rick,

Arun and I are co-authors. So, let me try to respond to you. The proposal for VLAN came about because we wanted to implement PFC as the flow-control mechanism instead of the credit-windowing scheme currently proposed. The reason is that if we separate control and data planes, then holding a TCP window in the data plane is very hard to achieve. HW based data planes, don't support TCP windows. Moreover, PFC is the standard way of dealing with flow-control over Ethernet. 

Now, the issue is that PFC allows only 8 CoS queues, so that would limit us to only 8 neighbors and/or flows. That was perceived as a limitation. Hence the use of VLANs to identify neighbors.

We understand that in current DLEP a single L2 network is assumed between the radios and the radio/router. So, VLANs can be used only between the radio and router, in case there are no VLANs between the radios. These VLANs will help us identify multiple neighbors, and PFC + VLAN will help us pause a particular neighbor. If, however, someone feels the need to use multiple VLANs between the radios, that would be achieved by VLAN-in-VLAN, and so forth. 

Bottom line, the VLAN proposal is driven by the need for flow-control, using a standard Ethernet based scheme, because in the control-data plane separation scenario, we don't have the alternative to use the current credit-window scheme.

Thanks

-----Original Message-----
From: manet [mailto:[email-masked]] On Behalf Of Rick Taylor
Sent: Tuesday, March 29, 2016 5:57 PM
To: Arun Parashar (arparash) <[email-masked]>; [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Hi Arun,

1) DLEP isn't finished yet, and so I don't think any extensions will proceed until then.

2) I still have worries about the use of VLANs per neighbour.  Given the single layer-2 requirement of DLEP, I don't see how this will work.

Will you be in Argentina?  If so, I would be happy to talk over my concerns face to face.

Cheers,

Rick

On 28/03/16 06:50, Arun Parashar (arparash) wrote:
> 1. VLANs and DLEP draft - 
> https://tools.ietf.org/html/draft-dalela-vlans-and-dlep-01
> 2. Flow Control draft - 
> https://tools.ietf.org/html/draft-dalela-dlep-flow-control-02
>
> -Arun
>
> To: Rick Taylor <[email-masked]>; Stan Ratliff 
> 1. VLANs and DLEP draft - 
> https://tools.ietf.org/html/draft-dalela-vlans-and-dlep-01
> 2. Flow Control draft - 
> https://tools.ietf.org/html/draft-dalela-dlep-flow-control-02
>
> Request comments.
>
> -Arun
>
> To: Rick Taylor <[email-masked]>; Stan Ratliff 
> Sure, that can be done. This draft is to seek the inputs and feedback 
> on what is the best path for us. Any and all suggestions are welcome, 
> including "this is a very bad idea" :-)
>
> Thanks
>
>> What you are proposing with VLAN-per-destination looks a lot more 
>> like the draft extension I put forward for layer-3 DLEP
>>>> We have submitted a new draft today to describe the PFC flow 
>>>> control mechanism. Sorry, this was long due on me; but better late than never!
>>>>

---

From: Stan Ratliff <[email-masked]>
Date: Tue, 29 Mar 2016 15:02:06 -0400

Ashish,

On Tue, Mar 29, 2016 at 11:12 AM, Ashish Dalela (adalela) <[email-masked]
> wrote:

> Hello Rick,
>
> Arun and I are co-authors. So, let me try to respond to you. The proposal
> for VLAN came about because we wanted to implement PFC as the flow-control
> mechanism instead of the credit-windowing scheme currently proposed. The
> reason is that if we separate control and data planes, then holding a TCP
> window in the data plane is very hard to achieve.

I'm afraid I must have missed something - the original Credit Windowing
draft has no requirement to hold a TCP window in the data plane. I guess
there would be an issue if the DLEP code itself was in the control plane,
which would necessitate control plane/data plane chatter to keep counts
consistent, but you'll have that problem irrespective of the approach,
wouldn't you?

One other point I've neglected to mention - the current Credit Windowing
draft allows a partner to flow-control a single MAC address, while keeping
traffic to other destinations (via the same radio). I don't believe PFC
allows for that level of granularity.

Regards,
Stan

> HW based data planes, don't support TCP windows. Moreover, PFC is the
> standard way of dealing with flow-control over Ethernet.
>
> Now, the issue is that PFC allows only 8 CoS queues, so that would limit
> us to only 8 neighbors and/or flows. That was perceived as a limitation.
> Hence the use of VLANs to identify neighbors.
>
> We understand that in current DLEP a single L2 network is assumed between
> the radios and the radio/router. So, VLANs can be used only between the
> radio and router, in case there are no VLANs between the radios. These
> VLANs will help us identify multiple neighbors, and PFC + VLAN will help us
> pause a particular neighbor. If, however, someone feels the need to use
> multiple VLANs between the radios, that would be achieved by VLAN-in-VLAN,
> and so forth.
>
> Bottom line, the VLAN proposal is driven by the need for flow-control,
> using a standard Ethernet based scheme, because in the control-data plane
> separation scenario, we don't have the alternative to use the current
> credit-window scheme.
>
> Thanks
>
> > Has everyone had chance to read and submit the review comments, we would
> like to hear comments/concerns before we take these two documents further
> in the chain.
> >
> you have already done.  My concern is not "VLANs can be used without PFC,
> and PFC can be used without VLAN", it is that core DLEP and VLANs doesn't
> work.
> >
> covers the VLAN topic in isolation.  And an updated version of your draft
> covering PFC-based flow control, referring to the VLAN extension if VLANs
> are in use.
> >
> >> The usage of the VLAN is limited to that between the router and the
> modem. It doesn't stretch to neighbors. In other words, the VLAN is not
> seen on the radio network. It is added by the modem and the router when
> forwarding packets to each other, and stripped when then packet exits the
> router (towards another router) or the modem (towards another modem). This
> clarification can be added to the next version if that would help.
> >>
> >> PFC based flow-control only gives us 8 neighbors with one traffic class
> each. And usage of VLAN alone doesn't imply the use of PFC per VLAN. That's
> why it is important to combine them.
> >>
> >> Currently VLANs can be used without PFC, and PFC can be used without
> VLAN. So, separating them will not achieve the key purpose of flow control.
> >>
> >> Thanks
> >> Ashish
> >>
> proposed the workaround of stating that the MAC address is no longer a MAC,
> just an identifying unique sequence of octets.
> >>
> >>> Yes, one of the proposals is to use a VLAN per neighbor. The default
> VLAN 1 will be used for the control plane. That allows us to have 4K
> neighbors, each distinguished by a unique VLAN ID.
> >>>
> >>> Thanks
> >>>
> >>> Ashish
> >>>
> >>> Obviously I haven't had a chance to completely digest the draft, but
> one thing that leaps out to me is the use of VLAN tagging, specifically
> section 7.  Are you suggesting that the DLEP session runs over a VLAN trunk
> between router and modem (somehow) and the VLANs then fan out at the modem
> in some way?
> >>>
> >>> Given the 'transparent layer-2' requirement of DLEP, how do you see
> this working with your model of a VLAN per destination (for flow control),
> or have I misunderstood what you are proposing?
> >>>
> >>> Regards
> >>>
> >>> Rick
> >>>

---

From: "Arun Parashar (arparash)" <[email-masked]>
Date: Wed, 30 Mar 2016 04:33:32 +0000

Hi Rick,

>>I'm afraid I must have missed something - the original Credit Windowing draft has no requirement to hold a TCP window in the data plane. I guess there would be an issue if the DLEP code itself was in the control plane, which would necessitate control plane/data plane chatter to keep counts consistent, but you'll have that problem irrespective of the approach, wouldn't you?

Let’s take the case of data plane is handled in asic (which is normally the case), how does asic handle dlep credit windowing exchange credits? This problem requires control plane involvement and thereby the chatter between control/data plane to calculate and sync credits, which is why we are proposing to use what is now the standard for ethernet flow control (802.1Qbb aka PFC). PFC based schemes don’t have any control plane involvement, PFC pause/resume packets are handled at L2 layer itself (a wide support of it is already there).

>> One other point I've neglected to mention - the current Credit Windowing draft allows a partner to flow-control a single MAC address, while >> keeping traffic to other destinations (via the same radio). I don't believe PFC allows for that level of granularity.

I think you are mixing 802.3x flow control and PFC, what we are proposing is use of IEEE 802.1Qbb aka PFC. The PFC based schemes proposed in the draft does allow per radio granularity in fact it provides even better granularity “per traffic class per radio”. Please see the attached picture which describes the scenario with PFC, neighbor can be identified using MAC or VLAN or CoS based on the schemes listed in the draft.

In the attached figure, the impact matrix would look as follows -

1.      Traffic Class ‘X’ destined to R2 - Paused

2.      Other Traffic Classes destined to R2 – Not Impacted

3.      Traffic destined to R3 – Not Impacted

Hope it helps to clarify the query.

-Arun

From: Stan Ratliff [mailto:[email-masked]]
Sent: Wednesday, March 30, 2016 12:32 AM
To: Ashish Dalela (adalela) <[email-masked]>
Cc: Rick Taylor <[email-masked]>; Arun Parashar (arparash) <[email-masked]>; [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

Ashish,

On Tue, Mar 29, 2016 at 11:12 AM, Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>> wrote:
Hello Rick,

Arun and I are co-authors. So, let me try to respond to you. The proposal for VLAN came about because we wanted to implement PFC as the flow-control mechanism instead of the credit-windowing scheme currently proposed. The reason is that if we separate control and data planes, then holding a TCP window in the data plane is very hard to achieve.

I'm afraid I must have missed something - the original Credit Windowing draft has no requirement to hold a TCP window in the data plane. I guess there would be an issue if the DLEP code itself was in the control plane, which would necessitate control plane/data plane chatter to keep counts consistent, but you'll have that problem irrespective of the approach, wouldn't you?

One other point I've neglected to mention - the current Credit Windowing draft allows a partner to flow-control a single MAC address, while keeping traffic to other destinations (via the same radio). I don't believe PFC allows for that level of granularity.

Regards,
Stan

HW based data planes, don't support TCP windows. Moreover, PFC is the standard way of dealing with flow-control over Ethernet.

Thanks

-----Original Message-----
From: manet [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Rick Taylor
Sent: Tuesday, March 29, 2016 5:57 PM
To: Arun Parashar (arparash) <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
Subject: Re: [manet] DLEP Credit Windowing

Hi Arun,

Cheers,

Rick

On 28/03/16 06:50, Arun Parashar (arparash) wrote:
> To: Rick Taylor <[email-masked]<mailto:[email-masked]>>; Stan Ratliff
> <[email-masked]<mailto:[email-masked]>>; Henning Rogge <[email-masked]<mailto:[email-masked]>>
> Cc: [email-masked]<mailto:[email-masked]>; Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>>
> From: manet [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Ashish Dalela
> (adalela)
> To: Rick Taylor <[email-masked]<mailto:[email-masked]>>; Stan Ratliff
> <[email-masked]<mailto:[email-masked]>>; Henning Rogge <[email-masked]<mailto:[email-masked]>>
> From: Rick Taylor [mailto:[email-masked]<mailto:[email-masked]>]
> To: Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>>; Stan Ratliff
> <[email-masked]<mailto:[email-masked]>>; Henning Rogge <[email-masked]<mailto:[email-masked]>>
>> From: Rick Taylor [mailto:[email-masked]<mailto:[email-masked]>]
>> To: Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>>; Stan Ratliff
>> <[email-masked]<mailto:[email-masked]>>; Henning Rogge <[email-masked]<mailto:[email-masked]>>
>>> From: Rick Taylor [mailto:[email-masked]<mailto:[email-masked]>]
>>> To: Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>>; Stan Ratliff
>>> <[email-masked]<mailto:[email-masked]>>; Henning Rogge <[email-masked]<mailto:[email-masked]>>
>>>> *From:*Stan Ratliff [mailto:[email-masked]<mailto:[email-masked]>]
>>>> *To:* Ashish Dalela (adalela) <[email-masked]<mailto:[email-masked]>>
>>>> *Cc:* Henning Rogge <[email-masked]<mailto:[email-masked]>>; [email-masked]<mailto:[email-masked]>
>>>> <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>>>>
>>>>        Cc: [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>
>>>>        <[email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>> wrote:
>>>>        [email-masked]<mailto:[email-masked]> <mailto:[email-masked]<mailto:[email-masked]>>

---

From: Henning Rogge <[email-masked]>
Date: Wed, 30 Mar 2016 07:54:57 +0200

On Wed, Mar 30, 2016 at 6:33 AM, Arun Parashar (arparash)
<[email-masked]> wrote:
> Hi Rick,
> Let’s take the case of data plane is handled in asic (which is normally the
> case), how does asic handle dlep credit windowing exchange credits? This
> problem requires control plane involvement and thereby the chatter between
> control/data plane to calculate and sync credits, which is why we are
> proposing to use what is now the standard for ethernet flow control
> (802.1Qbb aka PFC). PFC based schemes don’t have any control plane
> involvement, PFC pause/resume packets are handled at L2 layer itself (a wide
> support of it is already there).

If I remember the draft correctly it does use the PFC frames a bit
different than normal... what guarantee is there that an ASIC can
handle these frames?

Henning Rogge

---

From: "Ananth Laxminarasimhan (alaxmina)" <[email-masked]>
Date: Wed, 30 Mar 2016 06:41:25 +0000

Hi Henning,
>what guarantee is there that an ASIC can handle these frames?

If 802.1qbb, i.e. PFC is used, then no change would be required in the implementation or handing of those frames.
But, if we need to support more than 8 neighbours, then PFC would be used in conjunction with either a unique MAC, or a unique VLAN-ID to identify the neighbour.
In the latter 2 scenarios, however, the frame format does not change and the ASIC should be able to handle these frames; though, as Ashish has indicated in an earlier mail, it might involve change in the Ethernet firmware or the driver.

-Ananth

-----Original Message-----
From: manet [mailto:[email-masked]] On Behalf Of Henning Rogge
Sent: 30 March 2016 11:25
To: Arun Parashar (arparash)
Cc: [email-masked]
Subject: Re: [manet] DLEP Credit Windowing

On Wed, Mar 30, 2016 at 6:33 AM, Arun Parashar (arparash) <[email-masked]> wrote:
> Hi Rick,
> Let’s take the case of data plane is handled in asic (which is 
> normally the case), how does asic handle dlep credit windowing 
> exchange credits? This problem requires control plane involvement and 
> thereby the chatter between control/data plane to calculate and sync 
> credits, which is why we are proposing to use what is now the standard 
> for ethernet flow control (802.1Qbb aka PFC). PFC based schemes don’t 
> have any control plane involvement, PFC pause/resume packets are 
> handled at L2 layer itself (a wide support of it is already there).

If I remember the draft correctly it does use the PFC frames a bit different than normal... what guarantee is there that an ASIC can handle these frames?

Henning Rogge