Subject: Re: [manet] next version of pause extension

From: Henning Rogge <[email-masked]>
Date: Wed, 8 Nov 2017 08:23:54 +0100

Rick, Stan and me had also a discussion about this when talking about
the "link ID" extension...

I proposed the idea that we use "link IDs" to define multiple
"versions" of a connection to the same target MAC address. This would
also mean we could easily add a "this needs VLAN TOS x" or "this needs
DSCP y" flag to a neighbor, which means we don't need to introduce
"sub-TLVs".

Henning Rogge

On Mon, Oct 23, 2017 at 3:15 PM, Lou Berger <[email-masked]> wrote:
> Rick (all),
>
> When we last talked (ietf99, see
> http://etherpad.tools.ietf.org:9000/p/notes-ietf-99-manet). I think we
> agreed that draft-ietf-manet-dlep-pause-extension-01 captured your
> current thinking on "containers".  Are you still happy with the draft,
> or do you think there is some change needed?  If the latter, obviously
> let us know what change you'd like to see.
>
> Thanks,
>
> Lou
>
>
>
> _______________________________________________
> manet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/manet

---

From: Lou Berger <[email-masked]>
Date: Wed, 8 Nov 2017 09:01:07 -0500

Henning,

see below.

> the "link ID" extension...

I followed the thread (I think) -- I was hoping we'd get some good time
in Singapore to discuss this - will you (and Rick+ Stan) be there?

> DSCP y" flag to a neighbor, which means we don't need to introduce

I'm torn, the lid approach means a lot of added messaging to essentially
provide queue index semantics - but I also like the generic concepts of
LIDS (although I personally think LID length should be fixed per
session, not per spec.)

Given that lids are so immature, I don't think we should hold up pause.
Let's get it done and out there, and *if* we come up with something
better in a year or two we can deprecate pause.

Lou

---

From: Rick Taylor <[email-masked]>
Date: Wed, 8 Nov 2017 14:34:31 +0000

Lou, Henning,

Top posting to summarize...

I just missed the submission deadline for LIDs, but I have a version-02 
ready to go when the window reopens on the 11th.

Meanwhile, I think the direction Lou is going with the QueueIds is
pretty close to what we were discussing about LinkIds - orthogonal in
meaning, but similar in approach.

I'm with Lou on not holding up Pause while waiting for LIDs, as they
are unrelated.

However, I do worry that the Pause extension and the DiffServ CW
extension seem to do very much the same thing, in subtly different
ways:  Both announce the queues in use for the session and assign Ids
and some kind of flow marking information (+1), and both then use those
QueueIds to identify queues for operations (again +1).

However, I know various of us involved in DLEP over the years have all
known that 'flows' and 'queues' need to be addressed at some point, and
I wonder if this is the moment.  Is it time to pull the 'Queue/Flow
Identifier' text out of both extensions and consolidate it into a
single draft, which CW and Pause (and others) can then reference?  We
can then all have a good argument about packing DSCP code points into
'sub-tlvs' once for one document, and follow it up with mechanism
extensions for different use-cases of different types of 'flow'.

Thoughts?

Rick

---

From: Lou Berger <[email-masked]>
Date: Wed, 8 Nov 2017 09:56:29 -0500

Rick,

> QueueIds to identify queues for operations (again +1).

I really view Pause and Credit as having different use cases.  The first
is fairly simple to implement but is likely to be inefficient, i.e.,
result in periods of time when the link is idle.  The second is far more
powerful and complex to implement, but is likely to be more efficient
*and* allow a smart router to make more sophisticated decisions based on
window sizes.

> single draft, which CW and Pause (and others) can then reference?

The defined mechanisms in the Credit draft are currently structured
(intentionally) to allow of future reuse.  In particular the same credit
control can 100% be applied *without modification* to different forms of
traffic identification and classification, e.g. 802.1 PCPs.

I thought about breaking credit window control and traffic
classification into their own documents, but then decided that future TC
definitions/extensions could just reference the DIs defined in this
document - so there was no substantive value in doing this, but it
certainly is a valid options.  Of course, we can break into two if the
WG prefers.

Thanks,
Lou
>  We

---

From: Henning Rogge <[email-masked]>
Date: Thu, 9 Nov 2017 13:02:55 +0100

> *and* allow a smart router to make more sophisticated decisions based on

Unfortunately Pause (at least with Linux) is not that great to use...
it works with some cards, it doesn't works with different cards and
sometimes it behaves strange.

We experimented with it but have yet to test it combined with VLAN TOS IDs.

Unfortunately I have yet to find a good way to implement something
like the credit extension without extending the Linux Kernel, so this
is even worse to implement.

(Tips and Hints welcome!)

Henning Rogge

---

From: Henning Rogge <[email-masked]>
Date: Thu, 9 Nov 2017 13:06:58 +0100

> ready to go when the window reopens on the 11th.

I think the last version we discussed had both the ability to open
"new namespaces" for links that don't have real MAC addresses and
allow multiple "versions/flows/types" of neighbors to the same MAC
address, e.g. to announce two different QoS classes to the same
neighbor.

Henning

---

From: Rick Taylor <[email-masked]>
Date: Thu, 9 Nov 2017 16:10:26 +0000

Comments inline...

> -----Original Message-----
> From: Henning Rogge [mailto:[email-masked]]
> Sent: 09 November 2017 12:03
> To: Lou Berger
> Cc: Rick Taylor; [email-masked]
> Subject: Re: [manet] next version of pause extension
> 
> > I really view Pause and Credit as having different use cases.  The
> > first is fairly simple to implement but is likely to be inefficient,
> > i.e., result in periods of time when the link is idle.  The second is
> > far more powerful and complex to implement, but is likely to be more
> > efficient
> > *and* allow a smart router to make more sophisticated decisions based
> > on window sizes.
> 
> Unfortunately I have yet to find a good way to implement something like the
> credit extension without extending the Linux Kernel, so this is even worse to
> implement.
> 

I would suggest the answer lives somewhere in the ethtool API, but it's a weird backwater of the kernel that doesn't see much love.  I don't think one can avoid kernel patches.

Rick

---

From: Rick Taylor <[email-masked]>
Date: Thu, 9 Nov 2017 16:24:43 +0000

Comments inline...

> Sent: 09 November 2017 12:07
> To: Rick Taylor
> Cc: [email-masked]; [email-masked]
> > I just missed the submission deadline for LIDs, but I have a
> > version-02 ready to go when the window reopens on the 11th.
> 
> I think the last version we discussed had both the ability to open "new
> namespaces" for links that don't have real MAC addresses and allow multiple
> "versions/flows/types" of neighbors to the same MAC address, e.g. to
> announce two different QoS classes to the same neighbor.
>

Yes, the new version of LIDs is greatly simplified: One just adds a 'Destination beyond this MAC' identifier.  I've been lucky that LIDs fits very easily alongside 8175 as all its supporting data items and messages already exist, I just need to add one extra bit of information.

Queue and Flows however are an altogether more complicated problem.  Yes, the answer is probably a QueueId dataitem to discriminate between the egress queues on the local modem, but there still needs to be a "These are my queues" announcement (complex), and even then I'm not sure that covers all the potential use-cases!

Are we discussing:

* The local modem classifies egress traffic into more than one egress queue, *no matter* what destination it goes to?  In which case the metrics are {per-Destination,per-Queue}, and the control is per-Queue

* The local modem maintains one egress queue for each destination? In which case we have metrics per-Queue and control per-Queue, but some kind of Queue-to-Destination mapping.

* Both of the above?  In which case I'm going to suggest we don't have the building blocks in place yet to address this!

One thing that is starting to crystalize in my mind is that it's all about transmit queues, any receive queueing that occurs should be transparent to the DLEP session for simplicity's sake.

Lots to discuss, but I think we need to try to get a handle on the use-cases before we talk at cross-purposes.

Lou, can you detail what your use-case is for Pause and DiffServ CW?

Rick

---

From: Henning Rogge <[email-masked]>
Date: Wed, 15 Nov 2017 12:58:38 +0100

> Queue and Flows however are an altogether more complicated problem.  Yes, the answer is probably a QueueId dataitem to discriminate between the egress queues on the local modem, but there still nee…

lets assume we have an additional TLV called "queue id" which carriers
a numeric identifier similar to the current LID.

> * The local modem classifies egress traffic into more than one egress queue, *no matter* what destination it goes to?  In which case the metrics are {per-Destination,per-Queue}, and the control is…

Each neighbor would announce the same queue-id per "traffic class".
Flow control would refer to this flow-ids.

e.g.:

neighbor 1, traffic class 1: queue-id 1
neighbor 1, traffic class 2: queue-id 2
neighbor 2, traffic class 1: queue-id 1
neighbor 2, traffic class 2: queue-id 2

flow control could be done by defining parameters for queue-id 1 or queue-id 2.

> * The local modem maintains one egress queue for each destination? In which case we have metrics per-Queue and control per-Queue, but some kind of Queue-to-Destination mapping.

Each neighbor would announce the a different queue-id per "traffic
class". Flow control would refer to this flow-ids.

e.g.:

neighbor 1, traffic class 1: queue-id 1
neighbor 1, traffic class 2: queue-id 2
neighbor 2, traffic class 1: queue-id 3
neighbor 2, traffic class 2: queue-id 4

flow control could be done by defining parameters for flow-id 1 to 4.

The "flow control for whole interface" would just become the special
case where everyone is "queue-id 0" (the default value if the queue-id
TLV is missing).

This mechanics would also give us the option to have a special
"broadcast" queue but still group all unicast together.

What do you think?

> One thing that is starting to crystalize in my mind is that it's all about transmit queues, any receive queueing that occurs should be transparent to the DLEP session for simplicity's sake.

yes, flow control is mostly interesting at the bottleneck... unless we
assume a radio faster than the ethernet, we don't have a bottleneck
for receiving queues.

Henning