Subject: [sfc] Status of draft-ietf-sfc-nsh-ecn-support

From: Donald Eastlake <[email-masked]>
Date: Wed, 27 Oct 2021 14:30:56 -0400

Hi,

I believe this draft is ready for WG Last Call. The most recent
revisions have been minor editorial improvements and clarifications.

However, the IETF meeting is coming up and I understand there may be a
Transport Area review. So I suggest there should be a WG Last Call
starting in two or three weeks.

Thanks,
Donald
===============================
 Donald E. Eastlake 3rd   +1-508-333-2270 (cell)
 2386 Panoramic Circle, Apopka, FL 32703 USA
 [email-masked]

---------- Forwarded message ---------
From: <[email-masked]>
Date: Thu, Oct 21, 2021 at 9:36 PM
Subject: New Version Notification for draft-ietf-sfc-nsh-ecn-support-08.txt

A new version of I-D, draft-ietf-sfc-nsh-ecn-support-08.txt
has been successfully submitted by Donald E. Eastlake and posted to the
IETF repository.

Name:           draft-ietf-sfc-nsh-ecn-support
Revision:       08
Title:          Explicit Congestion Notification (ECN) and Congestion
Feedback Using the Network Service Header (NSH) and IPFIX
Document date:  2021-10-21
Group:          sfc
Pages:          33
URL:
https://www.ietf.org/archive/id/draft-ietf-sfc-nsh-ecn-support-08.txt
Status:         https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh-ecn-support/
Htmlized:
https://datatracker.ietf.org/doc/html/draft-ietf-sfc-nsh-ecn-support
Diff:
https://www.ietf.org/rfcdiff?url2=draft-ietf-sfc-nsh-ecn-support-08

Abstract:
   Explicit congestion notification (ECN) allows a forwarding element to
   notify downstream devices of the onset of congestion without having
   to drop packets. Coupled with a means to feed information about
   congestion back to upstream nodes, this can improve network
   efficiency through better congestion control, frequently without
   packet drops. This document specifies ECN and congestion feedback
   support within a Service Function Chaining (SFC) domain through use
   of the Network Service Header (NSH, RFC 8300) and IP Flow Information
   Export (IPFIX, RFC 7011).

The IETF Secretariat

---

From: <[email-masked]>
Date: Thu, 28 Oct 2021 11:52:20 +0000

Hi Donald, 

Thank you for your effort to mature the document. 

FWIW, please find my review of this version:

* pdf: https://raw.githubusercontent.com/boucadair/IETF-Drafts-Reviews/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.pdf 
* doc: https://github.com/boucadair/IETF-Drafts-Reviews/raw/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.docx 

Cheers,
Med

> -----Message d'origine-----
> De : sfc <[email-masked]> De la part de Donald Eastlake
> Envoyé : mercredi 27 octobre 2021 20:31
> À : [email-masked]
> Cc : [email-masked]
> Objet : [sfc] Status of draft-ietf-sfc-nsh-ecn-support
> 
> Hi,
> 
> I believe this draft is ready for WG Last Call. The most recent revisions
> have been minor editorial improvements and clarifications.
> 
>  2386 Panoramic Circle, Apopka, FL 32703 USA  [email-masked]
> 
> Feedback Using the Network Service Header (NSH) and IPFIX Document date:
> 2021-10-21
> Group:          sfc
> Pages:          33
> URL:
> _______________________________________________
> sfc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sfc

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 28 Oct 2021 18:10:58 +0100

Mohamed

Thank you for your extensive review.
Glossing over all the places where you would like the wording to be 
different, I detect the following main points of technical significance:

      The scheme is not how you imagine it - deliberately

Draft (§1.1): "The NSH is a natural place, in a domain where traffic is 
NSH encapsulated, to note congestion"
MB: "I would remove this mention as the outer-transport encapsulation 
would be “more” natural, IMO.  If maintained, you may consider 
s/natural/candidate "

I believe you have 'imagined' a different scheme (that wouldn't work). 
Then, from this point on, you edit the document to describe the scheme 
that you imagine.

But I think you've misunderstood. The outer transports /are/ usually 
ECN-capable. However, unless the congestion information propagates 
onward to something that can feed it back, it is just black-holed, and 
useless. This is why the NSH header is arranged to accumulate any ECN 
marking from each outer transport along an SFP. I'm not sure which of 
the following two options you are imagining, but neither work:
1) EITHER: the header encapsulated within the NSH could be used to 
accumulate congestion information across the SFC-enabled domain. Then 
NSH would have to be defined to require that, as SFs and SFFs strip each 
outer transport header, they propagate any ECN markings into the header 
encapsulated within the NSH. But this would only be legal for those 
packets where that inner header indicates that the e2e transport 
supports ECN. That makes this #2 option depend on e2e ECN deployment, 
which has now reached about 20%, but not close enough to 100% to be 
useful for controlling congestion routing within an SFC-enabled domain.
2) OR: the scheme you imagine consists of a sequence of outer transport 
headers, that are disjoint from the ECN point of view. Then whenever 
there was congestion marking of one of these outer transport headers, it 
would get dumped at the terminating SFF, which would then have to find 
somewhere to feed it back to. If each SFF fed it back to the ingress 
classifier, there would be a huge amount of feedback streams all 
converging on each ingress classifier, compared to the scheme as 
described, which accumulates the information forwards, then feeds it 
back in one message per SFP, from the egress.

I think we've confused you by describing the partially deployed ECN case 
first. Instead, perhaps we should describe the case with ECN deployed 
everywhere within the SFC-enabled domain first, then step back and 
describe partial deployment.

In case it helps, I'll describe the scheme with full ECN deployment:
The scheme uses the NSH (the one that remains associated with the 
packets/frames travelling along each SFP) to collect up any ECN marking 
added by each outer transport encapsulation. These all accumulate{Note 
1} in the NSH headers, for delivery to the egress of the SFC-enabled 
domain, which then regularly feeds back summaries of each path's 
congestion to the ingress. The ingress is then continually collecting a 
view of the congestion of all the SFPs it is using through the 
SFC-enabled domain, which it can use to inform routing decisions (or 
pass this info on to where these decisions are made).

In fact, the draft is primarily about partial ECN deployment - in two 
senses:
1) In theory, for packets to have ECN enabled, the two end-points (e2e) 
have to check that they both support ECN first. {Note 2} Nonetheless, if 
an SFC-enabled domain wants to use ECN to manage itself, it doesn't have 
to wait for all Internet endpoints to support ECN. It can fake ECN 
support edge-to-edge: by the ingress of the SFC-enabled domain turning 
on ECN support in the proposed NSH header it adds, then the egress 
forwards on any congestion markings in the header it decapsulates, 
unless ot sees that the decapsulated inner indicates no ECN support. In 
which case the egress converts an incoming congestion mark into a drop, 
which is the only congestion signal that this particular e2e transport 
understands.
2) Also, some of the SFFs or SFs might not support ECN. So the draft has 
to require all SFFs and SFs, as endpoints of an outer transport 
encapsulation, to check they both support ECN, and otherwise disable it 
in the outer between them. One assumes such cases would be rare in an 
SFC-enabled domain set up to use ECN. But, one has to cater for this 
possibility.

{Note 1}: When I say "accumulate", I mean in the same way ECN 
accumulates congestion marking along any path of network elements. For 
instance, if there are two congestion points on a path, the first one 
might mark 0.1% of packets, and the second might mark 1.2% of packets. 
Then roughly 1.3% of packets will be marked. Actually, the idea is that 
it's combinatorial probability, not additive, which happens naturally, 
because probabilistic marking at each congested point gives you
     [1 - (1-0.001)(1-0.012)] = 1.2988% marking.
For instance, if there was much higher congestion, e.g. 60% marking at 
three points, you would get 1-(1-0.6)(1-0.6)(1-0.6) = 93.6% marking, not 
180% which would be impossible.

{Note 2}: There's about 85% server support of ECN now, but only perhaps 
25% client support (don't quote me on that - it's a guess). So 25%*85% = 
21% of connections are ECN-capable.

      Connection end-points within an SFC-enabled domain

MB (re. §3): "Some of the legacy SFs may terminate TCP connections. 
These SFs will still follow the behavior in Section 6.1 of 3168. No? "
MB: (re. §3.2.2): "I’m afraid there is a deviation when the SF 
terminates the connection. The inner packet will thus be processed as 
per RFC5681 "
MB: (re. Figure 2 in §1.3) An SF may terminate the connection. It can be 
considered as a “final rcvr”.

Correct, but these TCP connections are control or management 
connectionsfor network operations, not forwarded connections.{Note 3} In 
these cases, all the layers are decapsulated on the SF all at once, and 
the path does not traverse an egress of the SFC-enabled domain, so the 
only appropriate congestion feedback loop is the outer one (e2e). This 
document is about creating and using an inner (edge-to-edge) loop, to 
gather congestion information to inform the /SFC/ routing decisions. 
Yes, there will be some control and management connections into that 
domain that are not under that control, but they are still congestion 
controlled end-to-end, they just can't contribute to the edge-to-edge 
congestion feedback.

{Note 3}: An SF by definition forwards packets. Both 'service' and 
'function' are ambiguous words, that would describe a web server, or a 
sandwich dispensing machine for that matter. But, put the words 
together, and you get an SF, which provides a /network/ function. It 
doesn't terminate connections, other than for management and control. 
Yes, it might terminate some packets, as in the DDoS scrubbing machine 
you mention. And it might also drop packets due to congestion. But none 
of this acts as a connection termination at the user plane.

      Responses to some specific comments

§2 "the IPv4 or IPv6 header as defined in Section 23.1 {?} of [RFC3168]"
Section 23.1, is the IANA registration of the codepoints, not thier 
definition. Section 5 is the definition.

* §3. Reason for citing RFC6040, not 
draft-ietf-tsvwg-rfc6040update-shim: the latter updates one specific 
part of RFC6040, so the convention is to cite the base RFC, unless only 
the update is relevant.

MB: "Why not simply say that these nodes must adhere to the reco in 
RFC6040? "
Because one cannot mandate that the past should not have happened (let 
alone MUST NOT happen!). This draft is all about incremental deployment

MB: "Why not just saying “all SFFs” as the egress node is the last SFF 
of an SFP?"
When ECN for the SFC-enabled domain is in the NSH (as we have 
deliberately defined it, rather than as you have 'imagined' it) decap 
/is/ only needed and supported at the egress of the domain. That's the 
whole point.

Draft: "This approach will inherently support all known variants of ECN, 
including the experimental L4S capability [RFC8311] [ecnL4S]."
MB: "I’m afraid some more elaboration is needed to back this statement "
Agreed. We need to describe this (probably in an appendix, rather like 
the similar draft about ECN support in TRILL).

MB: "If we want a pluggable detection logic, I would not require AQM as 
required, but just as an example. That approach is what is adopted by 
DC-TCP for example (RFC 8257)."
DCTCP uses an AQM that fits the recommendations of RFC7567, which are 
deliberately already generalized. You only have to read the list of 
recommendation headings from the Contents of RFC7567 to see that it is 
very general. The functions within an SFC-enabled domain still have to 
work with e2e congestion control. So I would strongly advise against 
going off into abstract hyperspace by saying some pluggable function is 
possible that does some other magic sort of active queue management.

Draft: "Congestion encountered in the SFF to SF and SF to SFF paths will 
be unmanaged. "
MB: "Why not handling this at the outer transport? "
This is defined as a case where the SF is ECN-unware. So it can't be 
handled in the outer transport either, because it terminates at a node 
that is ECN-unaware.

§3.2.3
Draft: "Other forwarding nodes, that are non-NSH forwarding nodes 
between NSH forwarding nodes, such as IP or label switched routers, 
might also contain potential bottlenecks. If so, they SHOULD implement 
an AQM"
MB: "This is not specific to SFC. Do we have a pointer where such reco 
is provided for routers? "
Answer: RFC7567

§4
"How the egress knows the ingress for a given chain, a given packet, 
flow, etc?"
Using a lookup against the Service Path Identifier?

Draft: "a flow that would preserve the number of packets in the absence 
of congestion"
MB: "How owns this information?"
This is an important open question that the draft raises, but does not 
answer, I believe. But when I was first involved with this draft, I 
imagined that SFs would have to be registered with a requirement to 
provide this information (boolean).

MB: "When a congestion is observed, to what extend these messages will 
further exacerbate the congestion conditions? "
The use IPFIX over partially reliable SCTP is designed to address this 
(the draft doesn't say the following but it should):
* This designs leads to compact congestion summaries sent regularly (so 
congestion control of user traffic will have already yielded the little 
amount of space this control traffic needs.
* IPFIX uses cumulative counters, so any losses can be retransmitted, 
but no need to repeatedly retransmit because the next one will catch up 
the counter.
* SCTP's partial reliability mode allows retransmission to be abandoned 
after a deadline passes.

§5
Draft: "IPFIX template records are exchanged between ingress and egress "
MB: "Which may be between every SFF and a classifier. "
No. This is the scheme you've imagined, not the one described.

Thank you very much for the time you've taken to review this.

Cheers

Bob

-- 
________________________________________________________________
Bob Briscoe                               http://bobbriscoe.net/

---

From: <[email-masked]>
Date: Fri, 29 Oct 2021 06:45:42 +0000

Hi Bob,

Thank you much for the follow up.

Please see inline.

Cheers,
Med

De : sfc <[email-masked]> De la part de Bob Briscoe
Envoyé : jeudi 28 octobre 2021 19:11
À : BOUCADAIR Mohamed INNOV/NET <[email-masked]>
Cc : Donald Eastlake <[email-masked]>; [email-masked]; [email-masked]
Objet : Re: [sfc] Status of draft-ietf-sfc-nsh-ecn-support

Mohamed

Thank you for your extensive review.
Glossing over all the places where you would like the wording to be different, I detect the following main points of technical significance:
The scheme is not how you imagine it - deliberately
Draft (§1.1): "The NSH is a natural place, in a domain where traffic is NSH encapsulated, to note congestion"
MB: "I would remove this mention as the outer-transport encapsulation would be “more” natural, IMO.  If maintained, you may consider s/natural/candidate "

I believe you have 'imagined' a different scheme (that wouldn't work). Then, from this point on, you edit the document to describe the scheme that you imagine.
But I think you've misunderstood. The outer transports /are/ usually ECN-capable.
[Med] They should if they follow the various guidelines there (6020, 6020update)
However, unless the congestion information propagates onward to something that can feed it back, it is just black-holed, and useless.
[Med] Sure, but this is not specific where the notification is carried.
 This is why the NSH header is arranged to accumulate any ECN marking from each outer transport along an SFP. I'm not sure which of the following two options you are imagining, but neither work:
1) EITHER: the header encapsulated within the NSH could be used to accumulate congestion information across the SFC-enabled domain. Then NSH would have to be defined to require that, as SFs and SFFs strip each outer transport header, they propagate any ECN markings into the header encapsulated within the NSH. But this would only be legal for those packets where that inner header indicates that the e2e transport supports ECN. That makes this #2 option depend on e2e ECN deployment, which has now reached about 20%, but not close enough to 100% to be useful for controlling congestion routing within an SFC-enabled domain.
2) OR: the scheme you imagine consists of a sequence of outer transport headers, that are disjoint from the ECN point of view. Then whenever there was congestion marking of one of these outer transport headers, it would get dumped at the terminating SFF, which would then have to find somewhere to feed it back to. If each SFF fed it back to the ingress classifier, there would be a huge amount of feedback streams all converging on each ingress classifier, compared to the scheme as described, which accumulates the information forwards, then feeds it back in one message per SFP, from the egress.
[Med] What I had in mind as an alternative is:
* uniform SFF capabilities with regards to ECN in an SFC-enabled domain
* a consistent setting of ECN at both outer and inner headers (this allows in particular a terminating SF when present)
* SFFs to glue the ECN during encap/decap along an SFP.
* A control plane that monitors the SFC-enabled domain.
As indicated in my comment, I suggest to use a more factual wording instead of indicating this is “natural”.

OLD:

The NSH is a

   natural place, in a domain where traffic is NSH encapsulated, to note

   congestion, avoiding possible confusion due, for example, to changes

   in the outer transport header in different parts of the domain.

NEW:
This document discusses how the NSH can be used to note congestion.

I think we've confused you by describing the partially deployed ECN case first. Instead, perhaps we should describe the case with ECN deployed everywhere within the SFC-enabled domain first, then step back and describe partial deployment.

In case it helps, I'll describe the scheme with full ECN deployment:
The scheme uses the NSH (the one that remains associated with the packets/frames travelling along each SFP) to collect up any ECN marking added by each outer transport encapsulation. These all accumulate{Note 1} in the NSH headers, for delivery to the egress of the SFC-enabled domain, which then regularly feeds back summaries of each path's congestion to the ingress. The ingress is then continually collecting a view of the congestion of all the SFPs it is using through the SFC-enabled domain, which it can use to inform routing decisions (or pass this info on to where these decisions are made).
[Med] This text is better, indeed. The main difference with what I described above is that NSH is the glue rather than relying on a local treatment at SFFs. Still, I don’t see how an egress can know the ingress of an SFP. There is no such thing in the NSH and in the lookup tables. One side question: when NSH-in-NSH (RFC 8459) is used, do you propagate the notification in all NSH levels?

In fact, the draft is primarily about partial ECN deployment - in two senses:
1) In theory, for packets to have ECN enabled, the two end-points (e2e) have to check that they both support ECN first. {Note 2} Nonetheless, if an SFC-enabled domain wants to use ECN to manage itself, it doesn't have to wait for all Internet endpoints to support ECN. It can fake ECN support edge-to-edge: by the ingress of the SFC-enabled domain turning on ECN support in the proposed NSH header it adds, then the egress forwards on any congestion markings in the header it decapsulates, unless ot sees that the decapsulated inner indicates no ECN support. In which case the egress converts an incoming congestion mark into a drop, which is the only congestion signal that this particular e2e transport understands.
[Med] I liked the faked ECN part even if it was “lost” in the text. I hope you will rework the draft to present this better. The “faked ECN” part is actually what motivates my comment about raising the comment about the intended status to consider Experimental track.

2) Also, some of the SFFs or SFs might not support ECN. So the draft has to require all SFFs and SFs, as endpoints of an outer transport encapsulation, to check they both support ECN, and otherwise disable it in the outer between them. One assumes such cases would be rare in an SFC-enabled domain set up to use ECN. But, one has to cater for this possibility.
[Med] I would assume that uniform capabilities at least for SFFs.

{Note 1}: When I say "accumulate", I mean in the same way ECN accumulates congestion marking along any path of network elements. For instance, if there are two congestion points on a path, the first one might mark 0.1% of packets, and the second might mark 1.2% of packets. Then roughly 1.3% of packets will be marked. Actually, the idea is that it's combinatorial probability, not additive, which happens naturally, because probabilistic marking at each congested point gives you
    [1 - (1-0.001)(1-0.012)] = 1.2988% marking.
For instance, if there was much higher congestion, e.g. 60% marking at three points, you would get 1-(1-0.6)(1-0.6)(1-0.6) = 93.6% marking, not 180% which would be impossible.

{Note 2}: There's about 85% server support of ECN now, but only perhaps 25% client support (don't quote me on that - it's a guess). So 25%*85% = 21% of connections are ECN-capable.
Connection end-points within an SFC-enabled domain
MB (re. §3): "Some of the legacy SFs may terminate TCP connections. These SFs will still follow the behavior in Section 6.1 of 3168. No? "
MB: (re. §3.2.2): "I’m afraid there is a deviation when the SF terminates the connection. The inner packet will thus be processed as per RFC5681 "
MB: (re. Figure 2 in §1.3) An SF may terminate the connection. It can be considered as a “final rcvr”.

Correct, but these TCP connections are control or management connectionsfor network operations, not forwarded connections.{Note 3} In these cases, all the layers are decapsulated on the SF all at once, and the path does not traverse an egress of the SFC-enabled domain
[Med] Not sure to get your point as the egress of an SFP is the last SFF of that chain, which is the SFF that services the terminating SF in this case.
, so the only appropriate congestion feedback loop is the outer one (e2e). This document is about creating and using an inner (edge-to-edge) loop, to gather congestion information to inform the /SFC/ routing decisions. Yes, there will be some control and management connections into that domain that are not under that control, but they are still congestion controlled end-to-end, they just can't contribute to the edge-to-edge congestion feedback.
[Med] The edge-to-edge is somehow “virtual” as any SFF in a domain be the terminating SFF of a chain. At least, the text should call the case of terminating SFs, what happens when ECN conflicts are observed at various levels (outer, inner, nsh), etc.

{Note 3}: An SF by definition forwards packets.
[Med] An SF is not a forwarder from the SFC architecture.
Both 'service' and 'function' are ambiguous words, that would describe a web server, or a sandwich dispensing machine for that matter. But, put the words together, and you get an SF, which provides a /network/ function. It doesn't terminate connections, other than for management and control. Yes, it might terminate some packets, as in the DDoS scrubbing machine you mention. And it might also drop packets due to congestion. But none of this acts as a connection termination at the user plane.
[Med] Hmm, we don’t make any assumption about the nature of SFs deployed in an SFC-enabled domain. But there are plenty SFs that terminates a connection: this can be content head-end, for example.
Responses to some specific comments

§2 "the IPv4 or IPv6 header as defined in Section 23.1 {?} of [RFC3168]"
Section 23.1, is the IANA registration of the codepoints, not thier definition. Section 5 is the definition.
[Med] OK. I hesitated when indicated that section; hence the “?” in the comment. I had a preference for 23.1 because it is really direct to point about the position of ECN bits.

* §3. Reason for citing RFC6040, not draft-ietf-tsvwg-rfc6040update-shim: the latter updates one specific part of RFC6040, so the convention is to cite the base RFC, unless only the update is relevant.
[Med] The reason for suggesting to cite the update I-D is that it covers many transports that are considered in SFC, especially the following ones:

   o  GRE (Generic Routing Encapsulation) [RFC2784] and NVGRE (Network
      Virtualization using GRE) [RFC7637];

   o  CAPWAP (Control And Provisioning of Wireless Access Points)
      [RFC5415];

   o  LISP (Locator/Identifier Separation Protocol) [RFC6830];

   o  VXLAN (Virtual eXtensible Local Area Network) [RFC7348] and VXLAN-
      GPE [I-D.ietf-nvo3-vxlan-gpe];

   o  Geneve [RFC8926];

MB: "Why not simply say that these nodes must adhere to the reco in RFC6040? "
Because one cannot mandate that the past should not have happened (let alone MUST NOT happen!). This draft is all about incremental deployment
[Med] see next point

MB: "Why not just saying “all SFFs” as the egress node is the last SFF of an SFP?"
When ECN for the SFC-enabled domain is in the NSH (as we have deliberately defined it, rather than as you have 'imagined' it) decap /is/ only needed and supported at the egress of the domain. That's the whole point.
[Med] I was actually echoing your text when saying “Section 3.3 requires that all the egress nodes …”. Knowing that every SFF can in theory terminate a chain, and thus behave as an, egress for that chain, one can conclude that all SFFs must support it!

Draft: "This approach will inherently support all known variants of ECN, including the experimental L4S capability [RFC8311] [ecnL4S]."
MB: "I’m afraid some more elaboration is needed to back this statement "
Agreed. We need to describe this (probably in an appendix, rather like the similar draft about ECN support in TRILL).
[Med] Thanks.

MB: "If we want a pluggable detection logic, I would not require AQM as required, but just as an example. That approach is what is adopted by DC-TCP for example (RFC 8257)."
DCTCP uses an AQM that fits the recommendations of RFC7567, which are deliberately already generalized. You only have to read the list of recommendation headings from the Contents of RFC7567 to see that it is very general. The functions within an SFC-enabled domain still have to work with e2e congestion control. So I would strongly advise against going off into abstract hyperspace by saying some pluggable function is possible that does some other magic sort of active queue management.
[Med] OK.

Draft: "Congestion encountered in the SFF to SF and SF to SFF paths will be unmanaged. "
MB: "Why not handling this at the outer transport? "
This is defined as a case where the SF is ECN-unware. So it can't be handled in the outer transport either, because it terminates at a node that is ECN-unaware.
[Med] You are right, even if I see it unlikely to have an SF that is NSH-aware but ECN-unaware.

§3.2.3
Draft: "Other forwarding nodes, that are non-NSH forwarding nodes between NSH forwarding nodes, such as IP or label switched routers, might also contain potential bottlenecks. If so, they SHOULD implement an AQM"
MB: "This is not specific to SFC. Do we have a pointer where such reco is provided for routers? "
Answer: RFC7567
[Med] I’m aware of BCP197, but I was looking for a profile document for routers such as draft-ietf-v6ops-ipv6rtr-reqs. But that ones expired and does not include AQM reco.

§4
"How the egress knows the ingress for a given chain, a given packet, flow, etc?"
Using a lookup against the Service Path Identifier?
[Med] We don’t have such feature in NSH. As currently defined, the feedback mechanism is broken. It is for the same reasons that, for example, the OAM work defines a TLV to indicate where to send a reply back. I’m afraid that you need such a TLV (draft-ietf-sfc-multi-layer-oam-16#section-6.3.1). Please note that there might be several classifiers, reclassification happening on path, etc.

Draft: "a flow that would preserve the number of packets in the absence of congestion"
MB: "How owns this information?"
This is an important open question that the draft raises, but does not answer, I believe. But when I was first involved with this draft, I imagined that SFs would have to be registered with a requirement to provide this information (boolean).
[Med] This is an important point to leave without elaboration. Although expired, you may refer to https://datatracker.ietf.org/doc/html/draft-ietf-sfc-control-plane#section-3.3.3 as a candidate to feed that information.

MB: "When a congestion is observed, to what extend these messages will further exacerbate the congestion conditions? "
The use IPFIX over partially reliable SCTP is designed to address this (the draft doesn't say the following but it should):
* This designs leads to compact congestion summaries sent regularly (so congestion control of user traffic will have already yielded the little amount of space this control traffic needs.
* IPFIX uses cumulative counters, so any losses can be retransmitted, but no need to repeatedly retransmit because the next one will catch up the counter.
* SCTP's partial reliability mode allows retransmission to be abandoned after a deadline passes.

[Med] These are good additions. Looking forward seeing them integrated.

§5

[Med] Hmm. I guess this is because you assume there are few nodes that behave as egress nodes, but in the SFC architecture the SFF that terminates a chain is the egress!

Cheers

Bob

On 28/10/2021 12:52, [email-masked]<mailto:[email-masked]> wrote:

Hi Donald,

Cheers,

Med

-----Message d'origine-----

De : sfc <[email-masked]><mailto:[email-masked]> De la part de Donald Eastlake

À : [email-masked]<mailto:[email-masked]>

Cc : [email-masked]<mailto:[email-masked]>

Hi,

Thanks,

Donald

 2386 Panoramic Circle, Apopka, FL 32703 USA  [email-masked]<mailto:[email-masked]>

---------- Forwarded message ---------

From: <[email-masked]><mailto:[email-masked]>

Subject: New Version Notification for draft-ietf-sfc-nsh-ecn-support-

08.txt

IETF repository.

Revision:       08

2021-10-21

Group:          sfc

Pages:          33

URL:

Status:         https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh-ecn-

support/

Htmlized:

Diff:

Abstract:

The IETF Secretariat

sfc mailing list

[email-masked]<mailto:[email-masked]>

Thank you.

--

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 29 Oct 2021 13:38:36 +0100

Med,

On 29/10/2021 07:45, [email-masked] wrote:
> *De :* sfc <[email-masked]> *De la part de* Bob Briscoe
> *Envoyé :* jeudi 28 octobre 2021 19:11
> *À :* BOUCADAIR Mohamed INNOV/NET <[email-masked]>
> *Cc :* Donald Eastlake <[email-masked]>; 
> [email-masked]; [email-masked]
> *Objet :* Re: [sfc] Status of draft-ietf-sfc-nsh-ecn-support
>
> Mohamed
>
> ECN-capable.
> */[Med] They should if they follow the various guidelines there (6020, 
> 6020update)/*
>
> However, unless the congestion information propagates onward to 
> something that can feed it back, it is just black-holed, and useless.
>
> */[Med] Sure, but this is not specific where the notification is 
> carried. /*
>
>  This is why the NSH header is arranged to accumulate any ECN marking 
> from each outer transport along an SFP. I'm not sure which of the 
> following two options you are imagining, but neither work:
> NSH would have to be defined to require that, as SFs and SFFs strip 
> each outer transport header, they propagate any ECN markings into the 
> header encapsulated within the NSH. But this would only be legal for 
> those packets where that inner header indicates that the e2e transport 
> 2) OR: the scheme you imagine consists of a sequence of outer 
> transport headers, that are disjoint from the ECN point of view. Then 
> whenever there was congestion marking of one of these outer transport 
> headers, it would get dumped at the terminating SFF, which would then 
> have to find somewhere to feed it back to. If each SFF fed it back to 
> the ingress classifier, there would be a huge amount of feedback 
> streams all converging on each ingress classifier, compared to the 
> scheme as described, which accumulates the information forwards, then 
> feeds it back in one message per SFP, from the egress.
>
> */[Med] What I had in mind as an alternative is:/*
>
> */* uniform SFF capabilities with regards to ECN in an SFC-enabled 
> domain/*
>
> */* a consistent setting of ECN at both outer and inner headers (this 
> allows in particular a terminating SF when present) /*
>
> */* SFFs to glue the ECN during encap/decap along an SFP./*
>
> */* A control plane that monitors the SFC-enabled domain. /*
>
> */As indicated in my comment, I suggest to use a more factual wording 
> instead of indicating this is “natural”. /*
>
> *OLD: *
>
> *The NSH is a*
>
> *natural place, in a domain where traffic is NSH encapsulated, to note*
>
> *congestion, avoiding possible confusion due, for example, to changes*
>
> *in the outer transport header in different parts of the domain.*
>
> **
>
> *NEW:*
>
> *This document discusses how the NSH can be used to note congestion.**//*
>
>
> I think we've confused you by describing the partially deployed ECN 
> case first. Instead, perhaps we should describe the case with ECN 
> deployed everywhere within the SFC-enabled domain first, then step 
> back and describe partial deployment.
>
> packets/frames travelling along each SFP) to collect up any ECN 
> marking added by each outer transport encapsulation. These all 
> accumulate{Note 1} in the NSH headers, for delivery to the egress of 
> the SFC-enabled domain, which then regularly feeds back summaries of 
> each path's congestion to the ingress. The ingress is then continually 
> collecting a view of the congestion of all the SFPs it is using 
> through the SFC-enabled domain, which it can use to inform routing 
> decisions (or pass this info on to where these decisions are made).
>
> */[Med] This text is better, indeed. The main difference with what I 
> described above is that NSH is the glue rather than relying on a local 
> treatment at SFFs. /*
>

[BB] I find it rather unusual that you talk relative to the scheme you 
describe. Surely you should be reviewing the scheme described in the 
draft, which doesn't need these magic "local treatments"?

> */Still, I don’t see how an egress can know the ingress of an SFP. 
> There is no such thing in the NSH and in the lookup tables. /*
>

[BB] I know my stuff on ECN, and on NFV. But I haven't been involved in 
SFC specifically, so I rely on my co-authors for a response on this 
question.

> */One side question: when NSH-in-NSH (RFC 8459) is used, do you 
> propagate the notification in all NSH levels?/*
>

[BB] Always only propagate into and out of whatever header is the outer.

> 1) In theory, for packets to have ECN enabled, the two end-points 
> (e2e) have to check that they both support ECN first. {Note 2} 
> Nonetheless, if an SFC-enabled domain wants to use ECN to manage 
> itself, it doesn't have to wait for all Internet endpoints to support 
> ECN. It can fake ECN support edge-to-edge: by the ingress of the 
> SFC-enabled domain turning on ECN support in the proposed NSH header 
> it adds, then the egress forwards on any congestion markings in the 
> header it decapsulates, unless ot sees that the decapsulated inner 
> indicates no ECN support. In which case the egress converts an 
> incoming congestion mark into a drop, which is the only congestion 
> signal that this particular e2e transport understands.
>
> */[Med] I liked the faked ECN part even if it was “lost” in the text. 
> I hope you will rework the draft to present this better. The “faked 
> ECN” part is actually what motivates my comment about raising the 
> comment about the intended status to consider Experimental track. /*
>

[BB] That's not new. It's how MPLS checks for the ECN capability 
[RFC5129]. There it's called "per-domain ECT checking", so we should 
probably use that term here:
https://datatracker.ietf.org/doc/html/rfc5129#section-3

> *//*
>
>
> 2) Also, some of the SFFs or SFs might not support ECN. So the draft 
> has to require all SFFs and SFs, as endpoints of an outer transport 
> encapsulation, to check they both support ECN, and otherwise disable 
> it in the outer between them. One assumes such cases would be rare in 
> an SFC-enabled domain set up to use ECN. But, one has to cater for 
> this possibility.
>
> */[Med] I would assume that uniform capabilities at least for SFFs. /*
>

[BB] Just SFFs is not enough. I was imagining that many SFs would 
already be written, and not necessarily already support ECN.

> Then roughly 1.3% of packets will be marked. Actually, the idea is 
> that it's combinatorial probability, not additive, which happens 
> naturally, because probabilistic marking at each congested point gives 
> you
> three points, you would get 1-(1-0.6)(1-0.6)(1-0.6) = 93.6% marking, 
> not 180% which would be impossible.
>
> {Note 2}: There's about 85% server support of ECN now, but only 
> perhaps 25% client support (don't quote me on that - it's a guess). So 
> 25%*85% = 21% of connections are ECN-capable.
>
>
> connectionsfor network operations, not forwarded connections.{Note 3} 
> In these cases, all the layers are decapsulated on the SF all at once, 
> and the path does not traverse an egress of the SFC-enabled domain
>
> */[Med] Not sure to get your point as the egress of an SFP is the last 
> SFF of that chain, which is the SFF that services the terminating SF 
> in this case. /*
>
> , so the only appropriate congestion feedback loop is the outer one 
> (e2e). This document is about creating and using an inner 
> (edge-to-edge) loop, to gather congestion information to inform the 
> /SFC/ routing decisions. Yes, there will be some control and 
> management connections into that domain that are not under that 
> control, but they are still congestion controlled end-to-end, they 
> just can't contribute to the edge-to-edge congestion feedback.
>
> */[Med] The edge-to-edge is somehow “virtual” as any SFF in a domain 
> be the terminating SFF of a chain. /*
>

[BB] OK. We would need to use the correct terminology - I'll leave that 
to Donald as editor. But I'm saying that we don't need to include 
'degenerate' points that act as the egress of the domain at the 
/management/ interfaces of the functions within the domain. But we do 
want {Note 4} to include points where traffic exits the SFC-enabled 
domain into an end-point (see discussion about context head-ends later).

It makes effectively no difference to the overall traffic control 
picture whether management interfaces are included or not, except it 
unnecessarily adds loads of extra feedback report loops about tiny 
amounts of traffic.

{Note 4}: 'We want' is not always 'we need'. In theory a congestion 
routing scheme (aka traffic engineering) can work and remain stable even 
if only a fraction of the traffic is under its control (there's a paper 
by Peter Key on that in the context of multipath routing - can't find it 
right now though).

> */At least, the text should call the case of terminating SFs, what 
> happens when ECN conflicts are observed at various levels (outer, 
> inner, nsh), etc. /*
>

[BB] I don't know what you mean here by 'ECN conflicts'.
Guessing what you mean, I can say that ECN has to propagate up and down 
as layers are added and stripped, and the rules in the draft ensure that 
is so. Then everything should work. And if anything doesn't follow the 
rules (bugs etc), drop will still act as the well-understood congestion 
signal of last resort, because a drop at any layer is a drop at all layers.

> */[Med] An SF is not a forwarder from the SFC architecture. /*
>
> Both 'service' and 'function' are ambiguous words, that would describe 
> a web server, or a sandwich dispensing machine for that matter. But, 
> put the words together, and you get an SF, which provides a /network/ 
> function. It doesn't terminate connections, other than for management 
> and control. Yes, it might terminate some packets, as in the DDoS 
> scrubbing machine you mention. And it might also drop packets due to 
> congestion. But none of this acts as a connection termination at the 
> user plane.
>
> */[Med] Hmm, we don’t make any assumption about the nature of SFs 
> deployed in an SFC-enabled domain. But there are plenty SFs that 
> terminates a connection: this can be content head-end, for example. /*
>

[BB] OK, I was assuming SFs were forwarding functions not endpoints. I 
actually checked the SFC architecture when you said this in your review, 
but I couldn't find anything about it, so I (apparently wrongly) assumed 
that the definition must be the same as for NFV, where forwarding is a 
defining feature of a 'network function'. Whatever, this means we will 
need to distinguish forwarding SFs from endpoint SFs. And we will 
want{Note 4} to define a domain egress at the interface to each endpoint 
function.

> */[Med] OK. I hesitated when indicated that section; hence the “?” in 
> the comment. I had a preference for 23.1 because it is really direct 
> to point about the position of ECN bits. /*
>

[BB] Fig 1 & Fig 2 in Section 5 of RFC3168 define the codepoints and the 
position of the field, resp.

> *//*
> part of RFC6040, so the convention is to cite the base RFC, unless 
> only the update is relevant.
>
> */[Med] The reason for suggesting to cite the update I-D is that it 
> covers many transports that are considered in SFC, especially the 
> following ones:/*
>
> *//*
>
> *//*
> */[Med] see next point/*
> */[Med] I was actually echoing your text when saying “Section 3.3 
> requires that all the egress nodes …”. Knowing that every SFF can in 
> theory terminate a chain, and thus behave as an, egress for that 
> chain, one can conclude that all SFFs must support it! /*
>

[BB] See earlier about forwarding vs endpoint SFs.

> *//*
>
> Draft: "This approach will inherently support all known variants of 
> ECN, including the experimental L4S capability [RFC8311] [ecnL4S]."
> */[Med] Thanks. /*
> MB: "If we want a pluggable detection logic, I would not require AQM 
> as required, but just as an example. That approach is what is adopted 
> by DC-TCP for example (RFC 8257)."
> going off into abstract hyperspace by saying some pluggable function 
> is possible that does some other magic sort of active queue management.
>
> */[Med] OK./*
>
> */[Med] You are right, even if I see it unlikely to have an SF that is 
> NSH-aware but ECN-unaware. /*
>
>
>
> §3.2.3
> */[Med] I’m aware of BCP197, but I was looking for a profile document 
> for routers such as draft-ietf-v6ops-ipv6rtr-reqs. But that ones 
> expired and does not include AQM reco. /*
>
>
>
> §4
> */[Med] We don’t have such feature in NSH. As currently defined, the 
> feedback mechanism is broken. It is for the same reasons that, for 
> example, the OAM work defines a TLV to indicate where to send a reply 
> back. I’m afraid that you need such a TLV 
> (draft-ietf-sfc-multi-layer-oam-16#section-6.3.1). Please note that 
> there might be several classifiers, reclassification happening on 
> path, etc./*
>

[BB] OK.

> */[Med] This is an important point to leave without elaboration. 
> Although expired, you may refer to 
> https://datatracker.ietf.org/doc/html/draft-ietf-sfc-control-plane#section-3.3.3 
> <https://datatracker.ietf.org/doc/html/draft-ietf-sfc-control-plane#section-3.3.3> 
> as a candidate to feed that information. /*
>
>
>
> * This designs leads to compact congestion summaries sent regularly 
> (so congestion control of user traffic will have already yielded the 
> little amount of space this control traffic needs.
> but no need to repeatedly retransmit because the next one will catch 
> up the counter.
> *//*
> */[Med] These are good additions. Looking forward seeing them 
> integrated. /*
>
>
>
> §5
> *//*
> */[Med] Hmm. I guess this is because you assume there are few nodes 
> that behave as egress nodes, but in the SFC architecture the SFF that 
> terminates a chain is the egress! /*
>
>

[BB] Need to distinguish forwarding and endpoint SFs, as already discussed.

Thank you for all this. V helpful.

Bob

> On 28/10/2021 12:52, [email-masked] 
> <mailto:[email-masked]> wrote:
>
>     Hi Donald,
>
>     * pdf:https://raw.githubusercontent.com/boucadair/IETF-Drafts-Reviews/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.pdf  <https://raw.githubusercontent.com/boucadair/IETF-Drafts-Reviews/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.pdf>  
>
>     * doc:https://github.com/boucadair/IETF-Drafts-Reviews/raw/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.docx  <https://github.com/boucadair/IETF-Drafts-Reviews/raw/master/draft-ietf-sfc-nsh-ecn-support-08-rev%20Med.docx>  
>
>     Cheers,
>
>     Med
>
>         De : sfc<[email-masked]>  <mailto:[email-masked]>  De la part de Donald Eastlake
>
>         À :[email-masked]  <mailto:[email-masked]>
>
>         Cc :[email-masked]  <mailto:[email-masked]>
>
>           2386 Panoramic Circle, Apopka, FL 32703 [email-masked]  <mailto:[email-masked]>
>
>         From:<[email-masked]>  <mailto:[email-masked]>
>
>         https://www.ietf.org/archive/id/draft-ietf-sfc-nsh-ecn-support-08.txt  <https://www.ietf.org/archive/id/draft-ietf-sfc-nsh-ecn-support-08.txt>
>
>         Status:https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh-ecn  <https://datatracker.ietf.org/doc/draft-ietf-sfc-nsh-ecn>-
>
>         support/
>
>         Htmlized:
>
>         https://datatracker.ietf.org/doc/html/draft-ietf-sfc-nsh-ecn-support  <https://datatracker.ietf.org/doc/html/draft-ietf-sfc-nsh-ecn-support>
>
>         Diff:
>
>         https://www.ietf.org/rfcdiff?url2=draft-ietf-sfc-nsh-ecn-support-08  <https://www.ietf.org/rfcdiff?url2=draft-ietf-sfc-nsh-ecn-support-08>
>
>         Abstract:
>
>         [email-masked]  <mailto:[email-masked]>
>
>         https://www.ietf.org/mailman/listinfo/sfc  <https://www.ietf.org/mailman/listinfo/sfc>
>
> Bob Briscoehttp://bobbriscoe.net/  <http://bobbriscoe.net/>