Subject: Adoption call on HBH drafts

From: [email-masked]
Date: Mon, 6 Dec 2021 11:13:25 +0100

Over the last 25 years, the community has spent a lot of time on IPv6 extension headers.
Lately there has been renewed interest in the Hop-By-Hop header.

There are successful uses of the HBH header within limited domains,
and considering all the work that already went in on HBH for RFC8200.
Is there anything more the IETF can or should do?

If we were to adopt the documents below, what is the desired outcome?

The HBH header is designed to signal to routers on the path to give a
packet special treatment. Can it be expected that an arbitrary router in
an arbitrary network will process a HBH header from a source outside the network?

What processing a node has to do is very specific to the particular option.
Does specifying general limits and behaviour on the generic HBH container make sense?
Or should that discussion be done on the specific option proposals themselves?

The two drafts under consideration for adoption are:

Title: IPv6 Hop-by-Hop Options Processing Procedures
Authors: R. Hinden, G. Fairhurst
File name: draft-hinden-6man-hbh-processing-01

Title: Limits on Sending and Processing IPv6 Extension Headers
Authors: T Herbert
File name: draft-herbert-6man-eh-limits-00

It would be useful to get the working group's view of the above questions.
And any other concerns of course.

I will let this discussion continue for the month of December.
As Bob is an author I will be handling the chair role for this adoption call.

Best regards,
Ole Troan, 6man chair

---

From: "Joel M. Halpern" <[email-masked]>
Date: Mon, 6 Dec 2021 12:42:45 -0500

I am having a little trouble parsing this email.
is this an adoption call?  Or a call for discussion on what we can / 
should / might do to improve the HBH situation?  Both?   Someething else?

Thank you,
Joel

On 12/6/2021 5:13 AM, [email-masked] wrote:
> --------------------------------------------------------------------
> IETF IPv6 working group mailing list
> [email-masked]
> Administrative Requests: https://www.ietf.org/mailman/listinfo/ipv6
> --------------------------------------------------------------------
>

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 7 Dec 2021 10:17:30 +1300

Hi,

I have concerns about these drafts as standards track documents. They both really mix up pragmatic operational concerns with standardization issues, and they both might prove restrictive for future development if they were committed to the standards track and ended up constraining implementations and procurements.

Just to pick on an example, both of them rely on the concept of "fast path". Maybe a few years from now, the dominant strain of routers will not *have* a fast path distinct from a "slow path". Also, maybe we will see a transformational change in the MTU problem space.

I don't think the IETF's crystal ball is accurate enough to know where the technology will be ten or twenty years from now. I think all we can do is make some practical recommendations (small "r") for the next few years, as a 6man/v6ops collaboration. There is a lot of good input to that discussion in both these drafts.

Regards
    Brian

---

From: "Pengshuping (Peng Shuping)" <[email-masked]>
Date: Tue, 7 Dec 2021 01:41:29 +0000

Hi,

Indeed, it does sound very long since RFC1883 (1995) which defined the IPv6 Extension Headers already. However, the truth is that over the 20+ years there were barely any IPv6 deployments with those extension headers being actually used till the recent years. We start to having PMTU/IOAM/Alternative Marking which uses HBH or DOH etc. Since these network services are being deployed in practice, people start tackling the issues that need to be resolved urgently therefore these current work in both 6man and v6ops. 

I agree with Brian that there is a lot of good descriptions in these drafts. If this is a problem of mixing up pragmatic operational concerns with standardization issues, we could move the operational concerns into this draft " Operational Issues with Processing of the Hop-by-Hop Options Header" in v6ops, which was also suggested by others before. 

Thank you!

Best regards, 
Shuping 

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Brian E Carpenter
Sent: Tuesday, December 7, 2021 5:18 AM
To: [email-masked]
Subject: Re: Adoption call on HBH drafts

Hi,

Regards
    Brian

On 06-Dec-21 23:13, [email-masked] wrote:
> packet special treatment. Can it be expected that an arbitrary router 
> in an arbitrary network will process a HBH header from a source outside the network?
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 6 Dec 2021 20:16:04 -0800

> I have concerns about these drafts as standards track documents. They both really mix up pragmatic operational concerns with standardization issues, and they both might prove restrictive for future…

Hi Brian,

I understand what you are saying, but we need to take reality and
history into account. IPv6 EH is currently defined to be arbitrarily
extensible and rich and basically unconstrained. That was a good
academic exercise in protocol development, however reality is that
after twenty years there's pretty much zero deployment of EH. The
problem is exactly that the protocol _is_ unconstrained making it no
feasible for hardware implementation. IMO, the irony is that if we
want to save IPv6 extension headers from extinction, we need to
constrain them.

I believe that a vague set of best practices would not have sufficient
teeth to fundamentally change anything-- we need guidance that is
specific and quantified. Consider,
draft-ietf-v6ops-ipv6-ehs-packet-drops-08. One of the reasons that
intermediate nodes drop packets with extension headers is that if they
make the IPv6 header chain "too long" then some devices will drop the
packet.  Intermediate nodes requiring access to the transport layer in
order to do simple routing seems to have become an implicit
requirement, the part about hosts can't send header chains that are
too long also seems to be some sort of implicit requirement, but no
one was willing to quantify exactly what the requirements are-- i.e.
precisely what length of header chains is a host allowed to send into
the Internet. Of course the standard doesn't specify this, and there's
currently no meaningful guidance to the host satck developer except
maybe "if you want your packets to cross the Internet then don't send
extension headers".

RFC8200 has the provision that allows intermediate nodes to ignore
Hop-by-Hop options. RFC8504 has limits for processing extension
headers on a host, and that is the basis for the limits in
draft-herbert-6man-eh-limits. As far constraining the future, the
limits in that draft are only baseline defaults. A receiver is free to
have bigger limits and a sender is free to exceed the baseline limits
if it knows that that is safe to do so (by have external knowlege of
the path, for example sending into a limited domain). Admittedly,
RFC8504 is a BCP, but my point is really how to create enforceable
limits to move things forward...

> Just to pick on an example, both of them rely on the concept of "fast path". Maybe a few years from now, the dominant strain of routers will not *have* a fast path distinct from a "slow path". Also…

I don't see why you think draft-herbert-6man-eh-limits relies on "fast
path/slow path". There is no mention of this in the draft, and as I
mentioned at the 6man meeting if a router can process things like HbH
options in the so called "fast path" then it should ignore them and in
particular not drop the packet or relegate processing to some slow
path that pretty much will kill the use of the HbH mechanisms.

> I don't think the IETF's crystal ball is accurate enough to know where the technology will be ten or twenty years from now. I think all we can do is make some practical recommendations (small "r")…

Again, we have at least twenty years of history with pretty much no
movement forward with the use of extension headers. Unless something
fundamentally changes, I don't see any change in the status quo over
the next twenty years.

Tom

---

From: Vasilenko Eduard <[email-masked]>
Date: Tue, 7 Dec 2021 10:43:34 +0000

Hi all,
Limiting different lengths and options would limit the future of the protocol
But would not help with the current problem that companies do not have a business case to just 1 option of minimal length.

It is useless to say to the person that does not see the reason even for 1 option to tell "1 would be the maximum".
It does not look encouraging.
Eduard
-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Tom Herbert
Sent: Tuesday, December 7, 2021 7:16 AM
To: Brian E Carpenter <[email-masked]>
Cc: [email-masked]
Subject: Re: Adoption call on HBH drafts

Hi Brian,

I understand what you are saying, but we need to take reality and history into account. IPv6 EH is currently defined to be arbitrarily extensible and rich and basically unconstrained. That was a good academic exercise in protocol development, however reality is that after twenty years there's pretty much zero deployment of EH. The problem is exactly that the protocol _is_ unconstrained making it no feasible for hardware implementation. IMO, the irony is that if we want to save IPv6 extension headers from extinction, we need to constrain them.

I believe that a vague set of best practices would not have sufficient teeth to fundamentally change anything-- we need guidance that is specific and quantified. Consider, draft-ietf-v6ops-ipv6-ehs-packet-drops-08. One of the reasons that intermediate nodes drop packets with extension headers is that if they make the IPv6 header chain "too long" then some devices will drop the packet.  Intermediate nodes requiring access to the transport layer in order to do simple routing seems to have become an implicit requirement, the part about hosts can't send header chains that are too long also seems to be some sort of implicit requirement, but no one was willing to quantify exactly what the requirements are-- i.e.
precisely what length of header chains is a host allowed to send into the Internet. Of course the standard doesn't specify this, and there's currently no meaningful guidance to the host satck developer except maybe "if you want your packets to cross the Internet then don't send extension headers".

RFC8200 has the provision that allows intermediate nodes to ignore Hop-by-Hop options. RFC8504 has limits for processing extension headers on a host, and that is the basis for the limits in draft-herbert-6man-eh-limits. As far constraining the future, the limits in that draft are only baseline defaults. A receiver is free to have bigger limits and a sender is free to exceed the baseline limits if it knows that that is safe to do so (by have external knowlege of the path, for example sending into a limited domain). Admittedly,
RFC8504 is a BCP, but my point is really how to create enforceable limits to move things forward...

> Just to pick on an example, both of them rely on the concept of "fast path". Maybe a few years from now, the dominant strain of routers will not *have* a fast path distinct from a "slow path". Also…

I don't see why you think draft-herbert-6man-eh-limits relies on "fast path/slow path". There is no mention of this in the draft, and as I mentioned at the 6man meeting if a router can process things like HbH options in the so called "fast path" then it should ignore them and in particular not drop the packet or relegate processing to some slow path that pretty much will kill the use of the HbH mechanisms.

> I don't think the IETF's crystal ball is accurate enough to know where the technology will be ten or twenty years from now. I think all we can do is make some practical recommendations (small "r")…

Again, we have at least twenty years of history with pretty much no movement forward with the use of extension headers. Unless something fundamentally changes, I don't see any change in the status quo over the next twenty years.

Tom

> > The HBH header is designed to signal to routers on the path to give 
> > a packet special treatment. Can it be expected that an arbitrary 
> > router in an arbitrary network will process a HBH header from a source outside the network?
> >
> > IETF IPv6 working group mailing list [email-masked] Administrative 
> > Requests: https://www.ietf.org/mailman/listinfo/ipv6

---

From: Michael Richardson <[email-masked]>
Date: Tue, 07 Dec 2021 07:28:53 -0500

Brian E Carpenter <[email-masked]> wrote:
    > I have concerns about these drafts as standards track documents. They
    > both really mix up pragmatic operational concerns with standardization
    > issues, and they both might prove restrictive for future development if
    > they were committed to the standards track and ended up constraining
    > implementations and procurements.

Frankly, I think that doing this kind of thing is among the IETF's greatest
strengths :-)

    > Just to pick on an example, both of them rely on the concept of "fast
    > path". Maybe a few years from now, the dominant strain of routers will
    > not *have* a fast path distinct from a "slow path". Also, maybe we will
    > see a transformational change in the MTU problem space.

I guess I'm lost as to why this is an issue. If hardware becomes more magic
and there are no slow paths, what's the problem?

We've had fastpaths and slowpaths since the first hand assembly coding of IP
forwarding engines of the 1980s demonstated that the BSD4.x kernel wasn't the
only way to forward packets.  I don't think it's ever going away.

    > I don't think the IETF's crystal ball is accurate enough to know where
    > the technology will be ten or twenty years from now. I think all we can
    > do is make some practical recommendations (small "r") for the next few
    > years, as a 6man/v6ops collaboration. There is a lot of good input to
    > that discussion in both these drafts.

So you'd like to adopt as a BCP then?
I recently watched the Spherical Router talk (I had missed it the first
time).   I really don't think that you want less than standards track if you
expect to ever use HBH.

I think that we don't have to be perfect at predicting the future to be able
to know what has and hasn't worked in the past.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Tue, 7 Dec 2021 15:48:10 +0000

Please note first that I am removing my AD hat but keeping my IPv6 one ;-)

Some comments on Ole's side question:
- no, we cannot expect unrelated routers/devices on the path to apply specific behavior (= waste CPU/memory/power/CO2) on random packets without some compensation, which requires some contract with source & destination(s)... Contractual networking is a research topic. BTW, I also do not expect/hope any router/device on the path to drop/modify extension headers for the very same reason
- as you wrote, many extension headers will only be processed by nodes (source & dest mainly) trusting each other's (i.e., my own definition of 'limited domain')
- within my own definition of limited domain, extension headers are the obvious way to extend IP features, which is what I really want to keep
- I shared Brian Carpenter's view that the difference between slow/fast path may become moot with VM being routers or with the generalization of Network processors

Now, some reviews on the 2 I-D and my own view (still no hat) on their adoption by 6MAN:

draft-hinden-6man-hbh-processing: 

While I appreciate that the authors have modified their I-D to allow more than 1 option :-), the MUST process the first option is too strong for me (except is the " if configured to do so" also applies to this MUST). 
I also appreciate that the Router Alert text has been revised from the -00. As it helps preventing ossification of the Internet (or even inside a limited domain).
I am not supporting "New options MUST NOT be defined that require Slow Path processing"... 
But this is "only" a call for adoption and rev -01 diff has demonstrated that Bob & Gory are listening to the WG feedback. Even with this flexibility and the text quality, I am not supporting putting constraints on the evolution of HbH, hence not supporting the adoption of this document.

draft-herbert-6man-eh-limits: 

This I-D actually lists a lot of current and valid operational concerns when ext headers must be processed (not for transit routers except if they want to apply L4 ACL !!). 
BTW, Section 1.2 should include a section on "be liberal on what you forward" ;-)  
I wonder what is the reason for the 104 number in "IPv6 header chain length up to 104 bytes" ;-) (OK explained later, so, something to fix in the flow)
I need to re-read 8200, but it does not allow multiple copies of the same ext header (except dest options), so, I am unsure about section 2.2.3 (happy to be corrected)
Many parts of section 2 do not put hard limits :-) or are quite sensible (about the padding options) but putting later a limit of 104 bytes (without using BCP14 normative language BTW) is weird... should 'old' router be removed from the Internet ? What about future routers ?
Section 3.1 should probably downgrade many MUST to SHOULD in order to fit my view. 
In section 3.1, the "unless it has explicit knowledge" is rather hand waving and, IMHO, cannot be in a standard track document, honestly (this sentence with my non-responsible AD hat).
Section 3.2, the MUST on intermediate nodes, isn't it wishful thinking ? Why would an intermediate node implement this ? 
Section 3.3, the intermediate destination nodes obviously have a 'contract' of some sort with the actual endpoints, so, they will anyway support what is required. So, probably no need for this section?
All in all, this I-D probably needs a lot of tuning (which is OK of course for a -00), but I am again ambivalent on the end goal of this I-D and its potential impact on the extensibility of IPv6, hence I am not supporting the adoption of this document.

Hope this helps (and sorry for Bob, Gorry, and Tom: good text on a real problem, but I cannot support work that blocks the IP extensions)

Regards,

-éric

On 06/12/2021, 11:14, "ipv6 on behalf of [email-masked]" <[email-masked] on behalf of [email-masked]> wrote:

    Best regards,
    Ole Troan, 6man chair

---

From: Michael Richardson <[email-masked]>
Date: Tue, 07 Dec 2021 11:14:47 -0500

> Please note first that I am removing my AD hat but keeping my IPv6 one

Is that a stable IPv6, Temporary/Privacy IPv6, or a classic OUI-derived IPv6-LL?

    > - no, we cannot expect unrelated routers/devices on the path to apply
    > specific behavior (= waste CPU/memory/power/CO2) on random packets
    > without some compensation, which requires some contract with source &
    > destination(s)... Contractual networking is a research topic. BTW, I
    > also do not expect/hope any router/device on the path to drop/modify
    > extension headers for the very same reason

If only we had built IPv6 after bitcoin :-)
People have been considering ways of compensating routers for forwarding packets.
It seems impossible to really do at the per-packet basis, and calls back to
circuit oriented architectures with call-setup, and therefore amortization of
compensation overhead.

    > demonstrated that Bob & Gory are listening to the WG feedback. Even
    > with this flexibility and the text quality, I am not supporting putting
    > constraints on the evolution of HbH, hence not supporting the adoption
    > of this document.

What do you suggest we do?
HBH is dead otherwise?

---

From: Vasilenko Eduard <[email-masked]>
Date: Tue, 7 Dec 2021 16:24:03 +0000

> What do you suggest we do?

What is the problem now to use HbH in the "limited domain"?
Carriers could test what they want and put it in production relatively easily.

The whole activity looks like an attempt to push HbH for the scale of the Internet.
IMHO: it would never happen. Who would pay for additional processing? No business case.

Eduard
-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Michael Richardson
Sent: Tuesday, December 7, 2021 7:15 PM
To: Eric Vyncke (evyncke) <evyncke=[email-masked]>; 6man WG <[email-masked]>
Subject: Re: Adoption call on HBH drafts

    > extension headers for the very same reason

If only we had built IPv6 after bitcoin :-) People have been considering ways of compensating routers for forwarding packets.
It seems impossible to really do at the per-packet basis, and calls back to circuit oriented architectures with call-setup, and therefore amortization of compensation overhead.

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Dec 2021 09:03:13 -0800

> IMHO: it would never happen. Who would pay for additional processing? No business case.

Eduard,

It's a circular argument: hosts can't send extension headers on the
Internet because they don't reliably traverse the intermediate nodes,
and router vendor routers don't want to support them because there's
no demonstrated use of them on the Internet.

To break this circular dependency, we're _not_ asking for routers to
do any additional processing. The ask is that if a router doesn't want
to support extensions headers then just _ignore_ them instead of
dropping packets with them instead of dropping them or relegating them
to some slow path which effectively renders them unusable. This is the
intent of the provisions in RFC8200, and the limits proposed in these
drafts expand upon the concept to allow different levels of support.
When routers drop packets that don't conform to their preferred
profile they are making an arbitrary decision for all nodes in the
path including end hosts that extension headers are of no value. Just
because you don't see a business case or potential for extension
headers in the Internet doesn't mean that everyone else believes that.

Tom

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Tue, 7 Dec 2021 17:05:17 +0000

On 07/12/2021, 17:17, "Michael Richardson" <[email-masked]> wrote:

Not the IPv4-mapped for sure ;-)

..%<... elided text as we agree on contractual networks

Suggest (still missing my AD hat) to keep HbH existing without any additional constraint... I.e., either vastly change the two I-D, or change the status to experimental, but not adopting them as they are. Even if I understand that the authors' intent to to 'save' HbH. In my view, there are no way to have HbH processed by "selfish" Internet router hence HbH is limited in some definition of "limited domains".

-éric

---

From: Ron Bonica <[email-masked]>
Date: Tue, 7 Dec 2021 17:13:42 +0000

+1

Juniper Business Use Only

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Joel M. Halpern
Sent: Monday, December 6, 2021 12:43 PM
To: [email-masked]; 6man WG <[email-masked]>
Subject: Re: Adoption call on HBH drafts

[External Email. Be cautious of content]

Thank you,
Joel

On 12/6/2021 5:13 AM, [email-masked] wrote:
> Administrative Requests: 
> https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ipv6
> __;!!NEt6yMaO-gk!TipNjXSdURz5hl001eUUiLOlNFqWkoGECc4QvL3BhQ66WfLS0p7W6
> O75gAyO02Y2$

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ipv6__;!!NEt6yMaO-gk!TipNjXSdURz5hl001eUUiLOlNFqWkoGECc4QvL3BhQ66WfLS0p7W6O75gAyO02Y2$
--------------------------------------------------------------------

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Dec 2021 10:25:44 -0700

On Tue, Dec 7, 2021, 9:06 AM Eric Vyncke (evyncke) <evyncke=
[email-masked]> wrote:

>         > Please note first that I am removing my AD hat but keeping my
> IPv6 one
>         > ;-)
>
> Suggest (still missing my AD hat) to keep HbH existing without any
> additional constraint... I.e., either vastly change the two I-D, or change
> the status to experimental, but not adopting them as they are. Even if I
> understand that the authors' intent to to 'save' HbH. In my view, there are
> no way to have HbH processed by "selfish" Internet router hence HbH is
> limited in some definition of "limited domains".
>

Eric, the selfish router is free to ignore HbH options; if they chose to
take apply some detrimental behavior like dropping packets that's where the
problem lies. RFC8200 allows intermediate devices to ignore HbH-- IMO that
is sufficient to facilitate transit of packets with HbH across the
Internet. The outstanding question is whether vendors are now doing that
since RFC8200 was published instead of dropping. I raised the question at
6man-v6ops meeting, but there seems to no answer yet (the lack of data for
deployment is really becoming a major problem in IETF IMO)

Tom

---

From: Vasilenko Eduard <[email-masked]>
Date: Tue, 7 Dec 2021 17:28:14 +0000

Hi Tom,
Hence, you accept, that everything is fine in the limited domain.
The one who would like to implement it - would have no problem.
I hope you would not try to restrict HbH for the people that have a business case for it (and would have even more in the future).

About the Internet scale and "ignore" option:
It is already permitted by RFC 8200 section 4: "may be examined or processed by any node along ..."
Because "may be" could be read as "may be not".
Done deal.
Did it help? No, because the problem is not on the technology -  it is about the business case.
It does not mean that it makes sense to restrict HbH for people that have a business case.

Eduard
-----Original Message-----
From: Tom Herbert [mailto:[email-masked]] 
Sent: Tuesday, December 7, 2021 8:03 PM
To: Vasilenko Eduard <[email-masked]>
Cc: Michael Richardson <[email-masked]>; Eric Vyncke (evyncke) <[email-masked]>; 6man WG <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Eduard,

It's a circular argument: hosts can't send extension headers on the Internet because they don't reliably traverse the intermediate nodes, and router vendor routers don't want to support them because there's no demonstrated use of them on the Internet.

To break this circular dependency, we're _not_ asking for routers to do any additional processing. The ask is that if a router doesn't want to support extensions headers then just _ignore_ them instead of dropping packets with them instead of dropping them or relegating them to some slow path which effectively renders them unusable. This is the intent of the provisions in RFC8200, and the limits proposed in these drafts expand upon the concept to allow different levels of support.
When routers drop packets that don't conform to their preferred profile they are making an arbitrary decision for all nodes in the path including end hosts that extension headers are of no value. Just because you don't see a business case or potential for extension headers in the Internet doesn't mean that everyone else believes that.

Tom

---

From: "Eric Vyncke (evyncke)" <[email-masked]>
Date: Tue, 7 Dec 2021 17:39:34 +0000

Tom,

> Eric, the selfish router is free to ignore HbH options; if they chose to take apply some detrimental behavior like dropping packets that's where the problem lies. RFC8200 allows intermediate devices to ignore HbH-- IMO that is sufficient >  to facilitate transit of packets with HbH across the Internet. The outstanding question is whether vendors are now doing that since RFC8200 was published instead of dropping. I raised the question at 6man-v6ops meeting, but there 
>  seems to no answer yet (the lack of data for deployment is really becoming a major problem in IETF IMO)

Tom, we are in full agreement on the above paragraph of yours ;-)

And for the deployment data, I am mentoring a Master Thesis at the University of Liège (Belgium) about this topic. Hoping to get a refresh in Spring (the season!) on the previous measurements by Jen & co.

Last vaguely educated point: those transit packet drops are most probably by configuration and not by design. But, this is a different topic ;-)

-éric

---

From: Ron Bonica <[email-masked]>
Date: Tue, 7 Dec 2021 19:13:15 +0000

Ole,

I support adoption of both drafts.

Network operators have demonstrated a need to extend IPv6 functionality. Use-cases include:

- Supporting more than 64 DiffServ classes
- MTU Discovery ( https://datatracker.ietf.org/doc/draft-ietf-6man-mtu-option/ )
- End-to-end loss detection ( https://datatracker.ietf.org/doc/draft-ietf-6man-ipv6-alt-mark/ )
- Programmed Sampling of packets ( https://datatracker.ietf.org/doc/draft-filsfils-6man-structured-flow-label/ )
- Postcard-based Telemetry using packet Marking (PBT-M) ( https://datatracker.ietf.org/doc/draft-filsfils-6man-structured-flow-label/ )

In order to support the above-mentioned use-cases, we need to extend the IPv6 metadata (i.e., header information) that is processed at each  node along a packet's delivery path.

Network operators have been able to satisfy some, but not all of their requirements for extensibility by:

- borrowing bits from existing fields in the base IPv6 header
- configuring forwarding policy to support their new, local definition of the existing field

This solution is sub-optimal because:

- it is operationally complex
- it precludes extending IPv6 functionality beyond a limited domain

Theoretically, IPv6 offers the HBH extension header as a method of extensibility. But sadly, it is not deployable as currently specified. Constraining it may be the only way to make it deployable.

Granted, the constraints that we apply today may not be necessary tomorrow. For that reason, it may be more appropriate to publish the HBH processing draft as a BCP instead of Standards Track document.

                                                                                                                      Ron

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 5:13 AM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

There are successful uses of the HBH header within limited domains, and considering all the work that already went in on HBH for RFC8200.
Is there anything more the IETF can or should do?

The HBH header is designed to signal to routers on the path to give a packet special treatment. Can it be expected that an arbitrary router in an arbitrary network will process a HBH header from a source outside the network?

Best regards,
Ole Troan, 6man chair

---

From: Tom Herbert <[email-masked]>
Date: Tue, 7 Dec 2021 13:21:51 -0800

Eduard,

Only to the extent that I understand what a limited domain is. There's no
normative definition of a limited domain, and even if there were we would
see push back in bifurcating protocol standards to have one version of a
protocol that only works over the Internet, and one that only works in a
limited domains. IETF standards, like RFC8200, don't make any such
distinction.

> I hope you would not try to restrict HbH for the people that have a
> business case for it (and would have even more in the future).

Again, you seem to have come the conclusion that there is no business case
on the Internet for extension headers, but there is for some definition of
limited domains. I do not believe there is consensus on that, nor am I even
sure there could ever be consensus on what the business case is. IETF is
about interoperable standards, and there seems to some major
interoperability problems in this area of the protocol. That's the problem
these drafts are trying to address.

> Did it help? No, because the problem is not on the technology -  it is

Without data, how can we tell if it helped or not?

It does not mean that it makes sense to restrict HbH for people that have a
> business case.

The draft doesn't restrict HbH in that manner. Only basic minimum limits of
support are required. People are free to exceed the limits if they have
knowledge of the environment (such as a limited domain).

Tom

> Cc: Michael Richardson <[email-masked]>; Eric Vyncke (evyncke) <
> [email-masked]>; 6man WG <[email-masked]>
> Internet because they don't reliably traverse the intermediate nodes, and
> router vendor routers don't want to support them because there's no
> demonstrated use of them on the Internet.
>
> To break this circular dependency, we're _not_ asking for routers to do
> any additional processing. The ask is that if a router doesn't want to
> support extensions headers then just _ignore_ them instead of dropping
> packets with them instead of dropping them or relegating them to some slow
> path which effectively renders them unusable. This is the intent of the
> provisions in RFC8200, and the limits proposed in these drafts expand upon
> the concept to allow different levels of support.
> When routers drop packets that don't conform to their preferred profile
> they are making an arbitrary decision for all nodes in the path including
> end hosts that extension headers are of no value. Just because you don't
> see a business case or potential for extension headers in the Internet
> doesn't mean that everyone else believes that.
>
> Tom
>
>
>
>
> > Eduard
> >     > Please note first that I am removing my AD hat but keeping my IPv6
> one
> >     > ;-)
> >
> > If only we had built IPv6 after bitcoin :-) People have been considering
> ways of compensating routers for forwarding packets.
> > It seems impossible to really do at the per-packet basis, and calls back
> to circuit oriented architectures with call-setup, and therefore
> amortization of compensation overhead.
> >

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 10 Dec 2021 09:00:28 +0000

Some comments on draft-herbert-6man-eh-limits-00.
1. This should be BCP as this is not intended to change 8200. The similar work as being referenced, RFC8504, is BCP. The device capabilities always change with time, with type, with vendor. We cannot limit the capability by a standard.
2. As BCP, I think the proposals should be recommendations not  "MUST" nor "MUST NOT".
3. The recommendations, like " Hosts MUST NOT send a packet with length of EH header chain greater than 104 bytes", should be based on a vast investigation on devices. Otherwise, it's only for some specific platform. 

Tianran
-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 6:13 PM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: "Xiejingrong (Jingrong)" <[email-masked]>
Date: Fri, 10 Dec 2021 12:32:32 +0000

Hi WG,

I have read draft-hinden-6man-hbh-processing and the related comments on this topic.
Fast path and slow path has proven to be "defacto standard" architecture not only in chip and hardware system but also in software forwarding system.
I don't think it will have any change in the near future, because it is even more widely used (in IP/IPv6/MPLS/...) and even more solid than IPv6 IMO.
This draft revises HBH processing based on the "defacto standard" architecture, and the main proposal -- "MUST process all (with one exception noted below) Hop-by-Hop options in the Fast Path." -- is a reasonable way to solve the problem.
I don't think a standard can be theoretic "correct" and "no need to update" but the various implementations based on the standard has been wrong for 20+ years.
Therefore I support the adoption of this document as Standard track, though I still have concerns for discussion [1].

[1] https://mailarchive.ietf.org/arch/msg/ipv6/weZ2nEwA6Zq_6qA4ujKDx08GjjM/

Thanks,
Jingrong

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 6:13 PM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: Gyan Mishra <[email-masked]>
Date: Mon, 13 Dec 2021 00:23:45 -0500

I support WG adoption of both drafts.

Responses in-line

Kind Regards

Gyan

> Lately there has been renewed interest in the Hop-By-Hop header.

    Gyan> Agreed.

> Is there anything more the IETF can or should do?

   Gyan> Can you point to documentation or drafts  of  limited domains HBH
usage?  I was not aware of any that exist.

There is still a lot of work that needs to be done on HBH.
My thoughts on this topic and is expressed in “
https://datatracker.ietf.org/doc/draft-ietf-v6ops-hbh/00/“ as far as
problem statement related to HBH to make HBH viable for development so that
operators can reap the benefits of new innovations.

> in an arbitrary network will process a HBH header from a source outside the

    Gyan> Yes the expectation is that HBH should work over any network
including the public Internet

>
> What processing a node has to do is very specific to the particular
> option.  Does specifying general limits and behaviour on the generic HBH
> container make sense?

    Gyan> As the goal is to make HBH processing in the forwarding plane
viable for development and break the endless cycle  so existing options and
new innovations using HBH can be developed to the benefit of operators

> Or should that discussion be done on the specific option proposals

Gyan> So you are asking what Hinden HBH section 6 talks about new options
processing .  Let’s say for Alt Mark, path MTU and in-situ OAM and all
other new innovations we limit the options. The disadvantage is existing
options that could benefit from line rate processing would not be benefit.
Also the overall issue unfortunately would not be resolved as the options
limit  is only helping new options and so operators would continue to drop
or rate limit HBH.  Problem not solved.  Of all the options that exist
today all could benefit from forwarding plane Processing and limiting the
options  except for router alert which remains for control plane processing

> File name: draft-hinden-6man-hbh-processing-01

    Gyan> Support

> File name: draft-herbert-6man-eh-limits-00

    Gyan> Support

    Gyan> I would like to ask the authors of Hinden HBH if as Shuping had
asked if maybe splitting the requirements into Shuping’s HBH problem
statement draft below:
https://datatracker.ietf.org/doc/html/draft-ietf-v6ops-hbh-00

That way the Hinden HBH can focus on solutions space and the Shuping draft
can be the main HBH problem statement draft focused on  by the WG.

-- 

<http://www.verizon.com/>

*Gyan Mishra*

*Network Solutions A**rchitect *

*Email [email-masked] <[email-masked]>*

*M 301 502-1347*

---

From: [email-masked]
Date: Mon, 13 Dec 2021 10:30:20 +0100

>    Gyan> Can you point to documentation or drafts  of  limited domains HBH usage?  I was not aware of any that exist.

I was thinking of:

IOAM in: https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-ipv6-options/06/
AltMark: https://datatracker.ietf.org/doc/draft-ietf-6man-ipv6-alt-mark/

Cheers,
Ole

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Mon, 13 Dec 2021 10:29:12 +0000

See also RFC 6553 (used by millions of deployed nodes in IoT networks) and https://datatracker.ietf.org/doc/html/draft-pthubert-detnet-ipv6-hbh, which allows ot signal DetNet at L3, the alternate being above UDP.

Keep safe;

Pascal

> Sent: lundi 13 décembre 2021 10:30
> To: Gyan Mishra <[email-masked]>
> Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
> >    Gyan> Can you point to documentation or drafts  of  limited domains
> HBH usage?  I was not aware of any that exist.
> 
> I was thinking of:
>

---

From: Haoyu Song <[email-masked]>
Date: Mon, 13 Dec 2021 17:32:36 +0000

I have followed all the discussions and still believe something should be done.
It's undeniable that recently much interest is expressed in support of HBH functions in IPv6 networks and it's clear that, due to the lack of BCP recommendations, we see big hurdle to actually apply them. Therefore I support the adoption of these drafts. They show good attempts to solve the issues. Although many details still need discussion, it's time for the entire WG to get involved.  

Best regards,
Haoyu

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 2:13 AM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: Tom Herbert <[email-masked]>
Date: Mon, 13 Dec 2021 10:04:55 -0800

> 1. This should be BCP as this is not intended to change 8200. The similar work as being referenced, RFC8504, is BCP. The device capabilities always change with time, with type, with vendor. We cann…

Yes, devices can change any time and the requirements they support can
differ widely between vendors. And that's basically the problem, there
is no consistency amongst them in certain areas such as how hop-by-hop
extension header is processed. Without a stanard, I don't believe we
can achieve some consistency for at least minimal support of extension
headers.

> 3. The recommendations, like " Hosts MUST NOT send a packet with length of EH header chain greater than 104 bytes", should be based on a vast investigation on devices. Otherwise, it's only for some…

IMO, this is a good example of why the limits need to be a standard.
As it stands, the requirement that intermediate nodes need to process
the transport layer is a de facto standard. RFC8200 does not mention
this. In fact, per RFC8200 the expectation is that intermediate nodes
can handle an arbitrary number of extension headers up to the MTU
length-- that is not practical for hardware implementations. In light
of that, it's very clear that vendors have taken it upon themselves to
fill in the gaps of the protocols where the protocols have become
detached from reality. Unfortunately, in this case, some vendors have
decided that they won't accept packets with any extension headers,
i.e. the EH chain is zero bytes. In doing that, they are completely
prohibiting the use of extension headers and since they are in the
datapath they are arbitrarily deciding for everyone in the path that
extension headers are unusable.

So what the limits in the draft do are provide the *minimum*
expectations of the Internet to make extension headers usable. If
devices exceed these limits that's great. But if a device can't even
meet the minimum requirements to at least make the standard protocol
feature viable then we want to call out those devices as being
non-conformant with the standard. If we don't have a standard, then we
can't do that and vendors will continue to take liberty and decide on
their own accord what the real standard for the Internet is.

As for investigation of vendor capabilities, that would be great. But,
frankly deployment data has been hard to come by. There are plenty of
vendors on this list, so they are free to comment.

Tom

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 14 Dec 2021 08:36:18 +1300

So, there's a case for saying that HBH is a great success as it is today, 
and the fact that it doesn't work across the open Internet is unimportant. (We under-estimated this in RFC8799, for example completely missing RFC6553.)

Regards
    Brian

On 13-Dec-21 23:29, Pascal Thubert (pthubert) wrote:
> See also RFC 6553 (used by millions of deployed nodes in IoT networks) and https://datatracker.ietf.org/doc/html/draft-pthubert-detnet-ipv6-hbh, 
which allows ot signal DetNet at L3, the alternate being above UDP.

---

From: Ron Bonica <[email-masked]>
Date: Mon, 13 Dec 2021 21:55:39 +0000

Brian,

Your case has two parts:

1) HBH is a great success as it is today
2) the fact that it doesn't work across the open Internet is unimportant

Although 1) may be true, it does not imply 2).

                                                             Ron

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Brian E Carpenter
Sent: Monday, December 13, 2021 2:36 PM
To: [email-masked]
Subject: Re: Adoption call on HBH drafts

So, there's a case for saying that HBH is a great success as it is today, and the fact that it doesn't work across the open Internet is unimportant. (We under-estimated this in RFC8799, for example completely missing RFC6553.)

Regards
    Brian

On 13-Dec-21 23:29, Pascal Thubert (pthubert) wrote:
> See also RFC 6553 (used by millions of deployed nodes in IoT networks) 
> and 
> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/html/draf
> t-pthubert-detnet-ipv6-hbh__;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sTz
> 3MrkdNBKegkJwJJdIYFGRq-UCUCd3fjJP_Fsw$ ,
>> IOAM in: 
>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-ie
>> tf-ippm-ioam-ipv6-__;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sTz3MrkdNB
>> KegkJwJJdIYFGRq-UCUCd3fsgkKY1M$
>> options/06/
>> AltMark: 
>> https://urldefense.com/v3/__https://datatracker.ietf.org/doc/draft-ie
>> tf-6man-ipv6-alt-mark/__;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sTz3Mr
>> kdNBKegkJwJJdIYFGRq-UCUCd3fhPmGauV$
>>
>> Cheers,
>> Ole
>
> __;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sTz3MrkdNBKegkJwJJdIYFGRq-UCU
> Cd3fkRDXrFC$

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://urldefense.com/v3/__https://www.ietf.org/mailman/listinfo/ipv6__;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sTz3MrkdNBKegkJwJJdIYFGRq-UCUCd3fkRDXrFC$
--------------------------------------------------------------------

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 14 Dec 2021 11:04:30 +1300

> Although 1) may be true, it does not imply 2).

True. But 1) refers to the RFC8200 version which has no size limits. We surely don't want to destroy 1) while trying to fix 2).

    Brian

---

From: Greg Mirsky <[email-masked]>
Date: Mon, 13 Dec 2021 16:06:21 -0800

Dear All,
I've read both drafts and I support the WG adopting both. Below, please
find my notes on the drafts:
draft-hinden-6man-hbh-processing:

   - I agree with the opening requirement in Section 5.2:

   Nodes that implement a differentiation between a Fast Path and a Slow

   Path MUST process all (with one exception noted below) Hop-by-Hop

   options in the Fast Path.

The distinction between the Slow and Fast paths is platform-specific and is
an ever-moving target. And then some specifications refer to
"bump-in-a-wire", which, as I interpret it, is something in between. It
seems that the "SHOULD" is a reasonable level of the requirement here. What
do you think?

On the same note, the requirement in Section 6 seems arbitrary:
New options MUST NOT be defined that require Slow Path processing.

   - nit Section 5.2 last para s/patch/path/

   - it seems that the construct "MUST NOT do A unless B" can be expressed
   as "SHOULD NOT do A. When B, MAY do C". What do you think?
   - also, it seems that the construct "SHOULD NOT do A unless B" can be
   re-worded as "SHOULD NOT do A. When B, MAY do C".
   - Also, specific numbers as limits seem as reflecting the current
   state-of-the-art. I wonder how much future-proof these limits could be?

Regards,
Greg

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Tue, 14 Dec 2021 01:20:48 +0000

And there is another application of HBH in https://datatracker.ietf.org/doc/html/draft-dong-6man-enhanced-vpn-vtn-id-06, which allows to identify the network resources allocated to a VTN for packet processing.

Best regards,
Jie

> From: ipv6 [mailto:[email-masked]] On Behalf Of Pascal Thubert
> (pthubert)
> Sent: Monday, December 13, 2021 6:29 PM
> To: [email-masked]; Gyan Mishra <[email-masked]>
> Subject: RE: Adoption call on HBH drafts
> 
> See also RFC 6553 (used by millions of deployed nodes in IoT networks) and
> https://datatracker.ietf.org/doc/html/draft-pthubert-detnet-ipv6-hbh, which
> allows ot signal DetNet at L3, the alternate being above UDP.
> 
> Keep safe;
> 
> Pascal
> 
> > https://datatracker.ietf.org/doc/draft-ietf-6man-ipv6-alt-mark/
> >
> > Cheers,
> > Ole
>

---

From: Mark Smith <[email-masked]>
Date: Tue, 14 Dec 2021 12:39:56 +1100

Hi,

So I'm wondering if looking for HBH options beyond the IPv6 fixed
header will ever be successfully adopted in the forwarding "fast
path".  Forwarding has been heavily optimised to use the destination
address because it is the common case and therefore has become the
cheapest case.

Would it be better to encode the need for HBH processing into a
packet's destination address itself? That would make it compatible
with the existing fast and cheap DA based forwarding fast paths.

Packets with these HBH encoding DAs can then be sent into a HBH
processing module via a route in the fast path FIB.

The HBH processing module could be hardware accelerated if necessary.
It could be added as a "sidecar" to existing routers via a link and a
route in the existing router FIB, as long as loops back out to the
sidecar are prevented. Existing DA tools such as ACLs can be used to
drop or rate limit these packets if required to protect the more
complex HBH processing resources.

We already have an example of encoding special case forwarding of a
packet in a DA - IPv6 multicast. The DA is also encoding the multicast
duplication processing in addition to the packet's final (group)
destination.

I think encoding the need for HBH processing in packets' DAs, that is
compatible with existing DA based forwarding, would have a much
greater chance of being adopted and deployed.

Regards,
Mark.

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 14 Dec 2021 07:49:12 +0000

Hello Brian

I read that HbH is useful and necessary even if Internet-wide success is not there (yet?) and I fully agree with that.

HbH is the only real extension to the IPv6 header for host to router signaling, and can be a great complement to SRH; though neither is limited to that, HbH can refer a state in the routers when SRH addresses stateless operation in select intermediate nodes.

The next and very hard step is to make it usable across the Internet as well. This involves building and deploying a new generation of ASICs and we must accept that it is a long term plan. One reason why current ASICs do not do much today is that there's no spec of what they are expected to do though there's a cost. We're stalled till we provide specs and value.

We need to agree on a road map of how much HbH space the ASICs should be able to process in 5, 10, 15, and even 20 years (IPv6 is 25 years old so what?). Tom's draft (draft-herbert-6man-eh-limits ) is making a step in that direction and I support it for that reason. If we specify such thing as this road map, it makes sense to make it STD track, because a lot od std track will depend on it.

Keep safe,

Pascal

> Sent: lundi 13 décembre 2021 23:05
> To: Ron Bonica <[email-masked]>; [email-masked]
> True. But 1) refers to the RFC8200 version which has no size limits. We
> surely don't want to destroy 1) while trying to fix 2).
> 
>     Brian
> 
> >
> >                                                               Ron
> >
> >
> >
> > So, there's a case for saying that HBH is a great success as it is
> > today, and the fact that it doesn't work across the open Internet is
> > unimportant. (We under-estimated this in RFC8799, for example
> > completely missing RFC6553.)
> >
> > Regards
> >      Brian
> >
> >> t-pthubert-detnet-ipv6-hbh__;!!NEt6yMaO-gk!Q3FfaSQWPEEHbnJhRfS1iq77sT
> >> z 3MrkdNBKegkJwJJdIYFGRq-UCUCd3fjJP_Fsw$ ,

---

From: Ole Troan <[email-masked]>
Date: Tue, 14 Dec 2021 09:33:22 +0100

Pascal,

> On 14 Dec 2021, at 08:49, Pascal Thubert (pthubert) <pthubert=[email-masked]> wrote:
> 

You allude to it in the first paragraph, but isn’t there two issues here? 1) how deep into the packet to parse 2) actual processing of the option in the header. 

I would assume 2 is orders of magnitude more costly than 1. And what would be the point of deploying ASICs parsing to <n> depth in packet unless they also implement real deployable options?

(I understand the limits proposed are useful in the ignore and skip over cases.)

O.

---

From: Giuseppe Fioccola <[email-masked]>
Date: Tue, 14 Dec 2021 14:53:42 +0000

Hi Ole, All,
I support the adoption of both drafts as the starting point to deal with the recent interest in the HBH header. 
There are several use cases that use HBH options to extend IPv6 functionality. For this reason, the recommendations on HBH Option processing should be reviewed in order to make it more practical.
I also think that it would be good to start with the controlled domain applications before considering larger internet-wide case.
In my opinion, a unified approach would be desired in order to specify general behavior and provide guidance for the implementation, with all the benefits of compatibility. But each HBH option proposal should be analyzed separately in its particular I-D, where the specific application domain and related security issues must be carefully defined.

Regards,

Giuseppe

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 11:13 AM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: Tom Herbert <[email-masked]>
Date: Tue, 14 Dec 2021 07:46:04 -0800

Ole,

The problem with #1 is those devices that have assumed the implicit
requirement that they MUST be able to read transport layer information
even in the case of normal forwarding operations. If the device can't
read the transport layer because it's too deep in the packet then they
may take detrimental actions such as outright dropping the packet.

This is why #1 is the more insidious problem. If routers ignore things
they don't understand or implement, like RFC8200 permits with HbH,
then others in the path can still productively use them; if routers
arbitrarily drop or impede packets with protocols they don't
understand or don't happen to like then no one in the path can enjoy
the benefits of the protocol.

Tom

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 14 Dec 2021 16:06:12 +0000

Hello Ole

Yes, there's typically both sides, how deep the HW can dig (or how deep we can cache heading bytes for software action), and resources that can be spent on each header. Certainly the processing cost has to be ~ linear with the size of the header, as it is for P4. It's something that the protocol designers will have to keep in mind when designing the new HbH EHs.

What I see is that the IPv6 header is full, and people are tempted to either reuse fields (such as IID or FL) or place things elsewhere (in the Destination Option Header, a RH or even above UDP) and expect DPI to do the right thing. 

If the IPv6 header needs extensions (we already know it does) then the HbH is the most sensible place for that, because it is set and read by the same parties. 

But for the last 25 years, it has been mostly unusable, except in niche operations. Considering how the HW has progressed, considering programmability in VPP and P4, I see that the lack of agreement (chicken and egg) between HW builders and protocol designers as the leading factor for the situation to perdure. We need to break the cyclic dependency now if we want anything in 5-10 years, because that's how long are cycles are.

For now, what I'm after is a common North star for HW builders and protocol designers that indicates where we will be in a number of years (5, 10, 15 ..) in terms of how minimum header bytes that are cached/ processed in any router. I expect tom's draft to provide that North Star.

Keep safe;

Pascal

> From: Ole Troan <[email-masked]>
> Sent: mardi 14 décembre 2021 9:33
> To: Pascal Thubert (pthubert) <[email-masked]>
> Cc: Brian E Carpenter <[email-masked]>; Ron Bonica
> <[email-masked]>; [email-masked]
> > On 14 Dec 2021, at 08:49, Pascal Thubert (pthubert)
> <pthubert=[email-masked]> wrote:
> >
> > The next and very hard step is to make it usable across the Internet as
> well. This involves building and deploying a new generation of ASICs and
> we must accept that it is a long term plan. One reason why current ASICs
> do not do much today is that there's no spec of what they are expected to
> do though there's a cost. We're stalled till we provide specs and value.
> >
> > We need to agree on a road map of how much HbH space the ASICs should be
> able to process in 5, 10, 15, and even 20 years (IPv6 is 25 years old so
> what?). Tom's draft (draft-herbert-6man-eh-limits ) is making a step in
> that direction and I support it for that reason. If we specify such thing
> as this road map, it makes sense to make it STD track, because a lot od
> std track will depend on it.
> 
> You allude to it in the first paragraph, but isn’t there two issues here?
> 1) how deep into the packet to parse 2) actual processing of the option in
> the header.
> 
> I would assume 2 is orders of magnitude more costly than 1. And what would
> be the point of deploying ASICs parsing to <n> depth in packet unless they
> also implement real deployable options?
>

---

From: [email-masked]
Date: Tue, 14 Dec 2021 17:15:57 +0100

Tom,

> the benefits of the protocol.

Yes, that was my sentence in parenthesis below.

>> (I understand the limits proposed are useful in the ignore and skip over cases.)

For this issue I do recall discussions of how long an extension header chain could possibly be in reality.
I can't recall if that ended up in an RFC or if it was just on the list. It might be something to include in the document.
E.g. how long are SRH or IOAM options?

O.

---

From: Haoyu Song <[email-masked]>
Date: Tue, 14 Dec 2021 18:30:20 +0000

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Pascal Thubert (pthubert)
Sent: Tuesday, December 14, 2021 8:06 AM
To: Ole Troan <[email-masked]>
Cc: [email-masked]
Subject: RE: Adoption call on HBH drafts

>>Yes, there's typically both sides, how deep the HW can dig (or how deep we can cache heading bytes for software action), and resources that can be spent on each header. 

>>Certainly the processing cost has to be ~ linear with the size of the header, as it is for P4. It's something that the protocol designers will have to keep in mind when designing the new HbH EHs.

The processing cost is not necessarily linear to the size of the header. We can say that the parsing cost is linear to the size but the processing cost depends on the nature of the header, otherwise we won't have the issue about the fast/slow path. 
On the other hand, parsing cost is very low which can hardly become a problem. So we have two real concerns:
1.  the header buffer size (or packet window size). If the size of all the headers that need to be parsed and processed exceeds this size, then the chip either fails or significantly reduces its throughput. 
2.  the header must be suitable for efficient processing (e.g., for fast path only)

Haoyu

---

From: Haoyu Song <[email-masked]>
Date: Tue, 14 Dec 2021 18:44:51 +0000

Not everything can be encoded in DA so we still need to solve the HBH problem, isn't it?
If we provide right limitation, mechanism, and behavior, then HBH can be very useful and indispensable. 

Haoyu

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Mark Smith
Sent: Monday, December 13, 2021 5:40 PM
To: Ole Troan <[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Hi,

So I'm wondering if looking for HBH options beyond the IPv6 fixed header will ever be successfully adopted in the forwarding "fast path".  Forwarding has been heavily optimised to use the destination address because it is the common case and therefore has become the cheapest case.

Would it be better to encode the need for HBH processing into a packet's destination address itself? That would make it compatible with the existing fast and cheap DA based forwarding fast paths.

Packets with these HBH encoding DAs can then be sent into a HBH processing module via a route in the fast path FIB.

The HBH processing module could be hardware accelerated if necessary.
It could be added as a "sidecar" to existing routers via a link and a route in the existing router FIB, as long as loops back out to the sidecar are prevented. Existing DA tools such as ACLs can be used to drop or rate limit these packets if required to protect the more complex HBH processing resources.

We already have an example of encoding special case forwarding of a packet in a DA - IPv6 multicast. The DA is also encoding the multicast duplication processing in addition to the packet's final (group) destination.

I think encoding the need for HBH processing in packets' DAs, that is compatible with existing DA based forwarding, would have a much greater chance of being adopted and deployed.

Regards,
Mark.

On Mon, 6 Dec 2021 at 21:13, <[email-masked]> wrote:
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.
> ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40f
> uturewei.com%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c75
> 3a1d5591fedc%7C1%7C1%7C637750428482355090%7CUnknown%7CTWFpbGZsb3d8eyJW
> IjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&
> amp;sdata=H3IxhAlsF2zZhZijWCKWCHTtHQjqMyUwyQlqaX8g5Mk%3D&amp;reserved=
> 0

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40futurewei.com%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637750428482355090%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=H3IxhAlsF2zZhZijWCKWCHTtHQjqMyUwyQlqaX8g5Mk%3D&amp;reserved=0
--------------------------------------------------------------------

---

From: Mark Smith <[email-masked]>
Date: Wed, 15 Dec 2021 07:22:07 +1100

> Not everything can be encoded in DA so we still need to solve the HBH

This isn't necessarily encoding the type of hop by hop processing
instruction in the DA, although it could be, similar to how specific
multicast groups are encoded in a DA.

Hop by hop processinging of a packet at a router, verses normal dumb and
fast forwarding involves:

1. identify if the packet is to be hop by hop processed
2. then performing the more complex processing on the packet, looking past
the IPv6 fixed header portion, into the subsequent EHs, transport and
packet payload to perform this more complex processing*

What more complex processing to perform on the packet could also be encoded
in the DA, or could be further encoded in a subsequent EH, that is looked
at once the DA has caused the packet to be identified as one for more
complex hop by hop processing.

As the packet is being "hop by hop" processed at the device that would
match on and therefore hold an instance of the packet's DA, this "hop by
hop" processing EH would actually be a Destination Option.

Regards,
Mark.

*I would describe this as "host processing" of the packets, because I think
forwarding is just about delivery of packets/packages to their destination,
not what is done to them when they arrive at their destination. Think about
how the physical postal/mail system works.

If we provide right limitation, mechanism, and behavior, then HBH can be
> very useful and indispensable.
>
> Haoyu
>
> So I'm wondering if looking for HBH options beyond the IPv6 fixed header
> will ever be successfully adopted in the forwarding "fast path".
> Forwarding has been heavily optimised to use the destination address
> because it is the common case and therefore has become the cheapest case.
>
> Would it be better to encode the need for HBH processing into a packet's
> destination address itself? That would make it compatible with the existing
> fast and cheap DA based forwarding fast paths.
>
> route in the existing router FIB, as long as loops back out to the sidecar
> are prevented. Existing DA tools such as ACLs can be used to drop or rate
> limit these packets if required to protect the more complex HBH processing
> resources.
>
> We already have an example of encoding special case forwarding of a packet
> in a DA - IPv6 multicast. The DA is also encoding the multicast duplication
> processing in addition to the packet's final (group) destination.
>
> compatible with existing DA based forwarding, would have a much greater
> chance of being adopted and deployed.
>
> Regards,
> Mark.
>
> option.
> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40futurewei.com%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637750428482355090%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=H3IxhAlsF2zZhZijWCKWCHTtHQjqMyUwyQlqaX8g5Mk%3D&amp;reserved=0

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 11 Dec 2021 22:49:42 -0500

On Mon, Dec 13, 2021 at 4:30 AM <[email-masked]> wrote:

> https://datatracker.ietf.org/doc/draft-ietf-ippm-ioam-ipv6-options/06/
> Gyan> My thoughts were that they are use cases for future work with HBH
but not existing today that are being utilized by any operator.
I think both of those have to be implemented by the vendor first to be
utilized by operators.

-- 

*Gyan Mishra*

*M 301 502-1347*

---

From: Gyan Mishra <[email-masked]>
Date: Sat, 11 Dec 2021 23:01:53 -0500

Hi Pascal

On Mon, Dec 13, 2021 at 5:29 AM Pascal Thubert (pthubert) <
[email-masked]> wrote:

> https://datatracker.ietf.org/doc/html/draft-pthubert-detnet-ipv6-hbh,
> Gyan> That's excellent that RFC 6553 RPL HBH is used with millions of IoT
networks.  Has the detnet hbh been implemented by vendors & and  deployed
to IoT networks as well?

-- 

*Gyan Mishra*

*M 301 502-1347*

---

From: Haoyu Song <[email-masked]>
Date: Tue, 14 Dec 2021 21:04:53 +0000

If the DA encoding alone is sufficient, then I think it's acceptable. SRv6 network programming also takes this approach. But encoding signals in DA to indicate the need for examining and processing EHs is not necessary and can be harmful instead. Some people may think this is an optimization to improve the performance. But the fact is, the EHs can be parsed without these signals with no extra cost. The real cost happens when the EHs are processed.

Haoyu

From: Mark Smith <[email-masked]>
Sent: Tuesday, December 14, 2021 12:22 PM
To: Haoyu Song <[email-masked]>
Cc: Ole Troan <[email-masked]>; 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

On Wed, 15 Dec 2021, 05:44 Haoyu Song, <[email-masked]<mailto:[email-masked]>> wrote:
Not everything can be encoded in DA so we still need to solve the HBH problem, isn't it?

This isn't necessarily encoding the type of hop by hop processing instruction in the DA, although it could be, similar to how specific multicast groups are encoded in a DA.

Hop by hop processinging of a packet at a router, verses normal dumb and fast forwarding involves:

1. identify if the packet is to be hop by hop processed
2. then performing the more complex processing on the packet, looking past the IPv6 fixed header portion, into the subsequent EHs, transport and packet payload to perform this more complex processing*

What more complex processing to perform on the packet could also be encoded in the DA, or could be further encoded in a subsequent EH, that is looked at once the DA has caused the packet to be identified as one for more complex hop by hop processing.

As the packet is being "hop by hop" processed at the device that would match on and therefore hold an instance of the packet's DA, this "hop by hop" processing EH would actually be a Destination Option.

Regards,
Mark.

*I would describe this as "host processing" of the packets, because I think forwarding is just about delivery of packets/packages to their destination, not what is done to them when they arrive at their destination. Think about how the physical postal/mail system works.

Haoyu

-----Original Message-----
From: ipv6 <[email-masked]<mailto:[email-masked]>> On Behalf Of Mark Smith
Sent: Monday, December 13, 2021 5:40 PM
To: Ole Troan <[email-masked]<mailto:[email-masked]>>
Cc: 6man WG <[email-masked]<mailto:[email-masked]>>; 6man Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Adoption call on HBH drafts

Hi,

Regards,
Mark.

On Mon, 6 Dec 2021 at 21:13, <[email-masked]<mailto:[email-masked]>> wrote:
> [email-masked]<mailto:[email-masked]>
> Administrative Requests:
> ietf.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fietf.org%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Yh4Az7R964na%2Fv4xoo5vAI9i8qEBepCAqe1zjikB6tE%3D&reserved=0>%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40f
> uturewei.com<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Futurewei.com%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=h3Il4Pea5y9PpMyC6KgzyBB4Z6rpz5XjMo8hG71irQg%3D&reserved=0>%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c75

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]<mailto:[email-masked]>
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40futurewei.com%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637750428482355090%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=H3IxhAlsF2zZhZijWCKWCHTtHQjqMyUwyQlqaX8g5Mk%3D&amp;reserved=0<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KgTGINRnRogknHtTPA4iAWTsnaRmMkN13M5Nk%2BR9188%3D&reserved=0>
--------------------------------------------------------------------

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Tue, 14 Dec 2021 21:10:01 +0000

Hello Gyan:

At the moment the DetNet HbH I-draft is just a personal submission.

 DetNet is well advanced on the MPLS front.

 The decision on IP has not been made and hints from this group could actually help DetNet do the right thing, whatever you think that is, for IPv6.

Regards,

Pascal

Le 14 déc. 2021 à 21:54, Gyan Mishra <[email-masked]> a écrit :

Hi Pascal

On Mon, Dec 13, 2021 at 5:29 AM Pascal Thubert (pthubert) <[email-masked]<mailto:[email-masked]>> wrote:
See also RFC 6553 (used by millions of deployed nodes in IoT networks) and https://datatracker.ietf.org/doc/html/draft-pthubert-detnet-ipv6-hbh, which allows ot signal DetNet at L3, the alternate being above UDP.

Gyan> That's excellent that RFC 6553 RPL HBH is used with millions of IoT networks.  Has the detnet hbh been implemented by vendors & and  deployed to IoT networks as well?

Keep safe;

Pascal

> From: ipv6 <[email-masked]<mailto:[email-masked]>> On Behalf Of [email-masked]<mailto:[email-masked]>
> To: Gyan Mishra <[email-masked]<mailto:[email-masked]>>

--

[http://ss7.vzw.com/is/image/VerizonWireless/vz-logo-email]<http://www.verizon.com/>

Gyan Mishra

Network Solutions Architect

Email [email-masked]<mailto:[email-masked]>

M 301 502-1347

---

From: Tom Herbert <[email-masked]>
Date: Tue, 14 Dec 2021 13:21:23 -0800

On Tue, Dec 14, 2021 at 1:05 PM Haoyu Song <[email-masked]> wrote:
>
> If the DA encoding alone is sufficient, then I think it’s acceptable. SRv6 network programming also takes this approach.

Haoyu,

I'm not sure exactly what you mean. SRv6 defines a rather complex
routing extension header replete with its own TLVs. If you are
referring to the proposals to place compressed Sids within the DA note
there was a lot of push back in 6man since that potentially breaks
Internet addressing architecture. IMO, overloading bits in the DA
seems like yet another proposal to find "available and extensible"
bits in the forty byte fixed IPv6 header that can be overloaded for
other purposes-- at best with such methods we might squeeze a few bits
and address a couple of short term problems, but as a long term
solution I believe it's a non-starter.

Tom

---

From: Haoyu Song <[email-masked]>
Date: Tue, 14 Dec 2021 21:31:32 +0000

-----Original Message-----
From: Tom Herbert <[email-masked]> 
Sent: Tuesday, December 14, 2021 1:21 PM
To: Haoyu Song <[email-masked]>
Cc: Mark Smith <[email-masked]>; 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

On Tue, Dec 14, 2021 at 1:05 PM Haoyu Song <[email-masked]> wrote:
>
> If the DA encoding alone is sufficient, then I think it's acceptable. SRv6 network programming also takes this approach.

Haoyu,

I'm not sure exactly what you mean. SRv6 defines a rather complex routing extension header replete with its own TLVs. 

[HS] I mean encoding function and parameter in IPv6 address suffix as in SRv6 network programming.

If you are referring to the proposals to place compressed Sids within the DA note there was a lot of push back in 6man since that potentially breaks Internet addressing architecture. IMO, overloading bits in the DA seems like yet another proposal to find "available and extensible"
bits in the forty byte fixed IPv6 header that can be overloaded for other purposes-- at best with such methods we might squeeze a few bits and address a couple of short term problems, but as a long term solution I believe it's a non-starter.

[HS] I agree. That approach is not extensible and sustainable for the long run. But on the other hand, if a single flag can do the job, it seems overkill to engage an extension header or TLV.

Tom

> 2. then performing the more complex processing on the packet, looking 
> past the IPv6 fixed header portion, into the subsequent EHs, transport 
> and packet payload to perform this more complex processing*
>
>
>
> > Requests:
> uturewei.com%7C20c25111941a4c089fcc08d9bf47ba83%7C0fee8ff2a3b240189c75
> 3a1d5591fedc%7C1%7C1%7C637751137079655882%7CUnknown%7CTWFpbGZsb3d8eyJW
> amp;sdata=h8j2O9cfC0Fgi32c8Y6%2F2wmChNI5NkLLOWR0h12OcP4%3D&amp;reserve
> d=0
> uturewei.com%7C20c25111941a4c089fcc08d9bf47ba83%7C0fee8ff2a3b240189c75
> 3a1d5591fedc%7C1%7C1%7C637751137079655882%7CUnknown%7CTWFpbGZsb3d8eyJW
> amp;sdata=h8j2O9cfC0Fgi32c8Y6%2F2wmChNI5NkLLOWR0h12OcP4%3D&amp;reserve
> d=0

---

From: Mark Smith <[email-masked]>
Date: Wed, 15 Dec 2021 08:31:46 +1100

On Wed, 15 Dec 2021, 08:04 Haoyu Song, <[email-masked]> wrote:

> If the DA encoding alone is sufficient, then I think it’s acceptable. SRv6
> network programming also takes this approach. But encoding signals in DA to
> indicate the need for examining and processing EHs is not necessary and can
> be harmful instead.
>

How is it harmful?

Some people may think this is an optimization to improve the performance.
> But the fact is, the EHs can be parsed without these signals with no extra
> cost. The real cost happens when the EHs are processed.
>

So this is an already solved problem then.

The "slow path" is already a fast path now because router control planes
have so much memory and CPU, being x86 based, they can host virtual
machines and containers.

Mandate HbH hop processing again by updating RFC8200.

If customers demand hardware processing of HbH and are willing to pay for
it, vendors will provide it. Those that don't will miss out on sales.

>
> Haoyu
>
>
>
> *From:* Mark Smith <[email-masked]>
> *Sent:* Tuesday, December 14, 2021 12:22 PM
> *To:* Haoyu Song <[email-masked]>
> *Cc:* Ole Troan <[email-masked]>; 6man WG <[email-masked]>; 6man
> Chairs <[email-masked]>
> *Subject:* Re: Adoption call on HBH drafts
>
>
>
>
>
> What more complex processing to perform on the packet could also be
> encoded in the DA, or could be further encoded in a subsequent EH, that is
> looked at once the DA has caused the packet to be identified as one for
> more complex hop by hop processing.
>
>
>
> *I would describe this as "host processing" of the packets, because I
> think forwarding is just about delivery of packets/packages to their
> destination, not what is done to them when they arrive at their
> destination. Think about how the physical postal/mail system works.
>
>
>
> > ietf.org
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fietf.org%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=Yh4Az7R964na%2Fv4xoo5vAI9i8qEBepCAqe1zjikB6tE%3D&reserved=0>
> %2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40f
> > uturewei.com
> <https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Futurewei.com%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=h3Il4Pea5y9PpMyC6KgzyBB4Z6rpz5XjMo8hG71irQg%3D&reserved=0>
> %7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c75
> <https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&data=04%7C01%7Chaoyu.song%40futurewei.com%7C299ddb431124426fba6a08d9bf3f7757%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751101583108855%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=KgTGINRnRogknHtTPA4iAWTsnaRmMkN13M5Nk%2BR9188%3D&reserved=0>

---

From: Haoyu Song <[email-masked]>
Date: Tue, 14 Dec 2021 21:46:06 +0000

Sent: Tuesday, December 14, 2021 1:32 PM

On Wed, 15 Dec 2021, 08:04 Haoyu Song, <[email-masked]<mailto:[email-masked]>> wrote:
If the DA encoding alone is sufficient, then I think it's acceptable. SRv6 network programming also takes this approach. But encoding signals in DA to indicate the need for examining and processing EHs is not necessary and can be harmful instead.

How is it harmful?

[HS] 1. Poor extensibility due to limited size; 2. Tamper with the address semantics; 3. Complicate the parsing FSM rather than simplify it; 4. Add one more place to care for packet processing (considering the case that the EH/TLVs are added or removed in a network).
Some people may think this is an optimization to improve the performance. But the fact is, the EHs can be parsed without these signals with no extra cost. The real cost happens when the EHs are processed.

The "slow path" is already a fast path now because router control planes have so much memory and CPU, being x86 based, they can host virtual machines and containers.

[HS] I don't agree. No matter how fast the control plane is, it's still considered "slow path" because it can't forward packets at line speed and is not supposed to be used for that purpose either.

If customers demand hardware processing of HbH and are willing to pay for it, vendors will provide it. Those that don't will miss out on sales.

[HS] The number, size, and processing complexity of each EH/TLV matter. We need clear guideline to make HBH truly usable.

Haoyu

From: Mark Smith <[email-masked]<mailto:[email-masked]>>
Sent: Tuesday, December 14, 2021 12:22 PM
To: Haoyu Song <[email-masked]<mailto:[email-masked]>>
Cc: Ole Troan <[email-masked]<mailto:[email-masked]>>; 6man WG <[email-masked]<mailto:[email-masked]>>; 6man Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Adoption call on HBH drafts

Regards,
Mark.

Haoyu

-----Original Message-----
From: ipv6 <[email-masked]<mailto:[email-masked]>> On Behalf Of Mark Smith
Sent: Monday, December 13, 2021 5:40 PM
To: Ole Troan <[email-masked]<mailto:[email-masked]>>
Cc: 6man WG <[email-masked]<mailto:[email-masked]>>; 6man Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Adoption call on HBH drafts

Hi,

Regards,
Mark.

On Mon, 6 Dec 2021 at 21:13, <[email-masked]<mailto:[email-masked]>> wrote:
> ietf.org<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fietf.org%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cc219e1a469924dfe473f08d9bf492930%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751143236206413%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=SebNqa7yOGeVAOtlAfWFAw3X10TfyZJpvfcdb0KWILw%3D&reserved=0>%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40f
> uturewei.com<https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Futurewei.com%2F&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cc219e1a469924dfe473f08d9bf492930%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751143236206413%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=itaJ81MYBMHYZYZqzY84Voyx1WHeY4FcI72ttjhKeDI%3D&reserved=0>%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c75

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]<mailto:[email-masked]>
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40futurewei.com%7Ca574bcaf6ebf42beb19308d9bea2bffa%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637750428482355090%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=H3IxhAlsF2zZhZijWCKWCHTtHQjqMyUwyQlqaX8g5Mk%3D&amp;reserved=0<https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&data=04%7C01%7Chaoyu.song%40futurewei.com%7Cc219e1a469924dfe473f08d9bf492930%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637751143236206413%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&sdata=DProkEtGt12S%2BWvawp8cLpBZVoqQGSB0e9%2FYguCkVi8%3D&reserved=0>
--------------------------------------------------------------------

---

From: Tianran Zhou <[email-masked]>
Date: Wed, 15 Dec 2021 06:28:13 +0000

Hi Tom,

Some other SDOs(IEEE, 3GPP) have road map and the top down design, for example, 100G eth, 400G eth.
And vendors will always comply and implement.
Your idea seems like this.
My concern is that IETF may not have this kind of mechanism. It's hard to make most vendors to agree on one *performance related* specification.
In RFC8200, there is already specification on HbH process
"it is now expected that nodes along a packet's delivery path only examine and process the Hop-by-Hop Options header if explicitly configured to do so " .
If the chip implementation can follow this, I think there is no need for the sender limitation.

Best,
Tianran

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Tom Herbert
Sent: Tuesday, December 14, 2021 2:05 AM
To: Tianran Zhou <zhoutianran=[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

> 1. This should be BCP as this is not intended to change 8200. The similar work as being referenced, RFC8504, is BCP. The device capabilities always change with time, with type, with vendor. We cann…

Yes, devices can change any time and the requirements they support can differ widely between vendors. And that's basically the problem, there is no consistency amongst them in certain areas such as how hop-by-hop extension header is processed. Without a stanard, I don't believe we can achieve some consistency for at least minimal support of extension headers.

IMO, this is a good example of why the limits need to be a standard.
As it stands, the requirement that intermediate nodes need to process the transport layer is a de facto standard. RFC8200 does not mention this. In fact, per RFC8200 the expectation is that intermediate nodes can handle an arbitrary number of extension headers up to the MTU
length-- that is not practical for hardware implementations. In light of that, it's very clear that vendors have taken it upon themselves to fill in the gaps of the protocols where the protocols have become detached from reality. Unfortunately, in this case, some vendors have decided that they won't accept packets with any extension headers, i.e. the EH chain is zero bytes. In doing that, they are completely prohibiting the use of extension headers and since they are in the datapath they are arbitrarily deciding for everyone in the path that extension headers are unusable.

So what the limits in the draft do are provide the *minimum* expectations of the Internet to make extension headers usable. If devices exceed these limits that's great. But if a device can't even meet the minimum requirements to at least make the standard protocol feature viable then we want to call out those devices as being non-conformant with the standard. If we don't have a standard, then we can't do that and vendors will continue to take liberty and decide on their own accord what the real standard for the Internet is.

As for investigation of vendor capabilities, that would be great. But, frankly deployment data has been hard to come by. There are plenty of vendors on this list, so they are free to comment.

Tom

---

From: "Chengli (Cheng Li)" <[email-masked]>
Date: Wed, 15 Dec 2021 09:23:59 +0000

I would like to support the adoption of draft-hinden-6man-hbh-processing-01 with some comments:

(1) I don't really understand what is the meaning of the following text:

Does it mean we MUST process ANY HBH option in the fast path except RA option, if the device support fast path forwarding?

(2) Fixed length will help the hardware processing, but how much it will help? A fixed size in 8-octet units means 8, 16, 26 octets? Or 8 octets only. I think it is not fixed 8-octet. Correct?

Then how many benefits we can get from 8-octet units?

"""
Fixed size in 8-octet units.  Specifically any new Hop-by-Hop
      options should not be variable size that could extend beyond what
      can be executed in the Fast Path.
Any new Hop-by-Hop option that is standardized that does not meet
   these criteria needs to explain in detail in its specification why
   this can not be accomplished and that there is a reasonable
   expectation that it can be proceed in most Fast Path implementations.
"""

But I do really think it is a good start to address the issue of HBH option. Many thanks for the authors!

Respect,
Cheng

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 6:13 PM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: Vasilenko Eduard <[email-masked]>
Date: Thu, 16 Dec 2021 08:22:42 +0000

++1

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Tom Herbert
Sent: Wednesday, December 15, 2021 12:21 AM
To: Haoyu Song <[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Haoyu,

I'm not sure exactly what you mean. SRv6 defines a rather complex routing extension header replete with its own TLVs. If you are referring to the proposals to place compressed Sids within the DA note there was a lot of push back in 6man since that potentially breaks Internet addressing architecture. IMO, overloading bits in the DA seems like yet another proposal to find "available and extensible"
bits in the forty byte fixed IPv6 header that can be overloaded for other purposes-- at best with such methods we might squeeze a few bits and address a couple of short term problems, but as a long term solution I believe it's a non-starter.

Tom

---

From: Vasilenko Eduard <[email-masked]>
Date: Thu, 16 Dec 2021 08:24:38 +0000

+1
5. Broken BGP protocol that assumes "next hop semantic" just looking to the filed size.

From: ipv6 [mailto:[email-masked]] On Behalf Of Haoyu Song
Sent: Wednesday, December 15, 2021 12:46 AM
To: Mark Smith <[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: RE: Adoption call on HBH drafts

How is it harmful?

Haoyu

Regards,
Mark.

Haoyu

-----Original Message-----
From: ipv6 <[email-masked]<mailto:[email-masked]>> On Behalf Of Mark Smith
Sent: Monday, December 13, 2021 5:40 PM
To: Ole Troan <[email-masked]<mailto:[email-masked]>>
Cc: 6man WG <[email-masked]<mailto:[email-masked]>>; 6man Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Adoption call on HBH drafts

Hi,

Regards,
Mark.

---

From: Mark Smith <[email-masked]>
Date: Fri, 17 Dec 2021 07:34:37 +1100

On Wed, 15 Dec 2021 at 08:21, Tom Herbert <[email-masked]> wrote:
> Internet addressing architecture.

Yes. The usid proposal is redefining IPv6 fields and field semantics
in IPv6 unicast addresses without formally defining them.

> IMO, overloading bits in the DA

I have suggested a formal IPv6 address space for this and other
purposes that copies much of the existing IPv6 multicast addressing
structure, and complies with an DA field holding a single address.

Using Anycast DAs to indicate hop by hop processing is mentioned on
slides 36 and 37, although I realise it is brief.

IPv6 Formal Anycast and Functional Anycast Addresses
https://datatracker.ietf.org/meeting/106/materials/slides-106-6man-sessa-ipv6-formal-anycast-addresses-and-functional-anycast-addresses-00

There's more text about it in section 4.8 of the draft. I need to
expand on it a bit more (and define a flag bit to indicate an "any
next hop" packet) , the ingress processing is not described because it
is standard longest match ingress DA packet matching. What processing
to perform and whether or not to send the packet on further after that
local processing is post-ingress.

https://datatracker.ietf.org/doc/html/draft-smith-6man-form-func-anycast-addresses-01#section-4.8

I'm very much motivated to try to solve problems by using and taking
advantage of existing hardware and mechanisms in networks, by making
compatible changes and improvements. Forklift upgrades of networks is
going to make adoption of any new IETF features much harder, is bad
for the environment, and is bad for my sleep, since I'll be one of the
people having to do that upgrade deployment work.

Regards,
Mark.

---

From: "Chengli (Cheng Li)" <[email-masked]>
Date: Tue, 21 Dec 2021 08:41:36 +0000

I support the adoption of draft-herbert-6man-eh-limits-00 with some comments.

1) The adoption does not mean that all the text has been agreed in the WG, I think it is a good start and it is the timing to let people involved to discuss.
2) Regarding the 64 Bytes, I think it is too small, 4 SRv6 SIDs encoded in SRH has exceeds 64 Bytes, though using G-SRv6 can support up to 13 32-bit compressed SIDs in reduced mode in 56 Bytes, but I still think 64 Bytes are too small. Think about telemetry, though we can use postcard mode iFIT. I suggest we can discuss this size. Right now the parser buff is 128, but we should raise it up to 160 B, so that more features can be supported. 
3) ALL the requirements with MUST word are required to be discussed thoroughly. 

"""
   *  A host MUST NOT send a packet with an extension header larger than
      64 bytes unless it has explicit knowledge that all nodes that
      might process the extension header are capable of processing a
      larger header.
"""

Thank you Tom and respect,
Cheng

-----Original Message-----
From: Chengli (Cheng Li) 
Sent: Wednesday, December 15, 2021 5:24 PM
To: [email-masked]' <[email-masked]>; 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: RE: Adoption call on HBH drafts

"""

Respect,
Cheng

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Monday, December 6, 2021 6:13 PM
To: 6man WG <[email-masked]>
Cc: 6man Chairs <[email-masked]>
Subject: Adoption call on HBH drafts

Best regards,
Ole Troan, 6man chair

---

From: "Chengli (Cheng Li)" <[email-masked]>
Date: Tue, 21 Dec 2021 09:37:39 +0000

Agree. No matter how many fields as long as the size does not exceed the parsing buffer size, the cost of parsing is nearly the same. The cost of EH is really about the processing.

Respect,
Cheng

Sent: Wednesday, December 15, 2021 5:05 AM

Haoyu

Regards,
Mark.

Haoyu

-----Original Message-----
From: ipv6 <[email-masked]<mailto:[email-masked]>> On Behalf Of Mark Smith
Sent: Monday, December 13, 2021 5:40 PM
To: Ole Troan <[email-masked]<mailto:[email-masked]>>
Cc: 6man WG <[email-masked]<mailto:[email-masked]>>; 6man Chairs <[email-masked]<mailto:[email-masked]>>
Subject: Re: Adoption call on HBH drafts

Hi,

Regards,
Mark.

---

From: "Dongjie (Jimmy)" <[email-masked]>
Date: Thu, 23 Dec 2021 04:17:27 +0000

Hi Ole, 

I support the adoption of draft-hinden-6man-hbh-processing-01. I have a few comments which may be considered either before or after the adoption. 

1. In the terminology section, the definition of fast path and slow path may be modified as: 

Fast path: A path through a router that is optimized for forwarding packets only based on the network layer headers.   The current text "without processing their payloads" may not be clear about which layer information would be used.

Slow path: A path through a router that can process both the network layer headers and the payloads.

2. Section 6 says the new hop-by-hop options should have fixed size in 8-octet units. And RFC 8200 says "Each extension header is an integer multiple of 8 octets long". 

Since the HBH header consists of:

- 1 octet Next Header field
- 1 octet Hdr Ext Len filed
- One or multiple options

If each option is in 8-octet units, it would always need to add 6-octet padN option to make the HBH extension header "an integer multiple of 8 octets long". Would this be a waste of this precious field?

Best regards,
Jie

> There are successful uses of the HBH header within limited domains, and
> considering all the work that already went in on HBH for RFC8200.
> The HBH header is designed to signal to routers on the path to give a packet
> special treatment. Can it be expected that an arbitrary router in an arbitrary
> network will process a HBH header from a source outside the network?
>

---

From: Tom Herbert <[email-masked]>
Date: Thu, 23 Dec 2021 08:36:44 -0800

> 2) Regarding the 64 Bytes, I think it is too small, 4 SRv6 SIDs encoded in SRH has exceeds 64 Bytes, though using G-SRv6 can support up to 13 32-bit compressed SIDs in reduced mode in 56 Bytes, but…

Hi Chengli,

For the parsing buffer requirement, bear in mind that we are trying to
set a minimum requirement that covers _all_ nodes in the Internet. As
the second clause in the requirements you quoted below state, the 64
byte can be exceeded when the sender knows what the capabilities of
the path. This will be the common case with  extension headers that
are really intended for use in a limited domain such as telemetry and
segment routing. To me the most critical aspect of this limit is that
it is a number greater than zero; we want to be able to definitively
say that a device that arbitrarily drops any and all packets with
extension headers is non-conformant with the standard.

Whether 64 bytes is the appropriate limit is good for discussion in
6man. However, I will point out that this is not the only WG setting
limits on extension headers; QUIC, RFC9000, has set a limit of 32
bytes of EH.

Tom

---

From: Joel Jaeggli <[email-masked]>
Date: Thu, 23 Dec 2021 14:22:09 -0800

On 12/6/21 02:13, [email-masked] wrote:
...
> File name: draft-hinden-6man-hbh-processing-01

Generally in favor of this one.

> File name: draft-herbert-6man-eh-limits-00

in this document the statement

    It should be noted that
    hosts stacks inherently have more processing capabilities than
    intermediate nodes, so it is exoected expect that they should be able
    to support higher limits.

I don't think can be held to be universally true for a couple of reasons. One is resource constraints, e.g. contiki's ipv6 stack can send and recieve packets with something on the order of 2KB of resident code. but also because of various forms of optimization that hosts may do to allow nics to perform datagram construction or disassembly in pursuit of performance. So while some hosts may be extremely flexible with respect to what they can send or recieve or the extent to which they are readily extensible to support new features others are less so, and may be variably so, e.g. supporting new icmp message types might be simple and costless, whereas supporting a new option to be parsed on delivery of a packet containing a tcp or udp stream might result in the need to disable an asic based optimization or bypass flow oriented policy in order to make use of a header.

While I think this limits proposed in this document seem realistic (104 bytes of header) I'm leary of hard limits that might unreasonably set expectations of future extensibility to high.

> Administrative Requests:https://www.ietf.org/mailman/listinfo/ipv6

---

From: <[email-masked]>
Date: Fri, 24 Dec 2021 09:48:30 +0800 (CST)

Hi Ole, et al,

I support wg adoption of the two drafts.
I heard a lot of interest in this aspect expressed at IETF 112, and I believe the two drafts are good starting point.
I also noticed there was debate on whether to deprecate Router Alert Option at the level of standards. I'm on the side of supporting deprecation, because IMHO the primary purpose of HBH drafts is to push forward the implementation and deployment of HbH Options, deprecating Router Alert Option does help on the primary purpose.

Best Regards,
Xiao Min
------------------原始邮件------------------
发件人：[email-masked]
收件人：6man WG;
抄送人：6man Chairs <[email-masked]>;
日 期 ：2021年12月06日 18:14
主 题 ：Adoption call on HBH drafts

---

From: "Chengli (Cheng Li)" <[email-masked]>
Date: Fri, 24 Dec 2021 02:36:22 +0000

Agree with 

"To me the most critical aspect of this limit is that it is a number greater than zero; we want to be able to definitively say that a device that arbitrarily drops any and all packets with extension headers is non-conformant with the standard."

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Tom Herbert
Sent: Friday, December 24, 2021 12:37 AM
To: Chengli (Cheng Li) <c.l=[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Hi Chengli,

For the parsing buffer requirement, bear in mind that we are trying to set a minimum requirement that covers _all_ nodes in the Internet. As the second clause in the requirements you quoted below state, the 64 byte can be exceeded when the sender knows what the capabilities of the path. This will be the common case with  extension headers that are really intended for use in a limited domain such as telemetry and segment routing. To me the most critical aspect of this limit is that it is a number greater than zero; we want to be able to definitively say that a device that arbitrarily drops any and all packets with extension headers is non-conformant with the standard.

Whether 64 bytes is the appropriate limit is good for discussion in 6man. However, I will point out that this is not the only WG setting limits on extension headers; QUIC, RFC9000, has set a limit of 32 bytes of EH.

Tom

---

From: Tianran Zhou <[email-masked]>
Date: Fri, 24 Dec 2021 06:09:02 +0000

I do not understand the logic here. If so, why not explicitly say, MUST support the EH or so?
If so, what's the difference with rfc2460 on HbH?

In addition, what's the situation when a network device is the ingress of a tunnel? So this network device can add the EH. Do the limitations also apply to this network device? 

Tianran

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Chengli (Cheng Li)
Sent: Friday, December 24, 2021 10:36 AM
To: Tom Herbert <[email-masked]>; Chengli (Cheng Li) <c.l=[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: RE: Adoption call on HBH drafts

Agree with 

-----Original Message-----
From: ipv6 [mailto:[email-masked]] On Behalf Of Tom Herbert
Sent: Friday, December 24, 2021 12:37 AM
To: Chengli (Cheng Li) <c.l=[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Hi Chengli,

Tom

---

From: Tom Herbert <[email-masked]>
Date: Thu, 23 Dec 2021 22:49:09 -0800

> If so, what's the difference with rfc2460 on HbH?

Hi TIanran,

The difference is that RFC2460 placed no limit on HBH, and in fact the
requirement was that all intermediate nodes were expected to process
all hop HBH. In practice this requirement is unrealistic and
infeasible to support in an implementation (really hardware and
software). The consequence of this requirement was that many vendors
arbitrarily decided to not support HBH options at all and simply drop
packets with them (the data in RFC7872 corroborate this). The
provision in RFC8200 solves part of the problem by relaxing the
requirement that all intermediate nodes must process extensions. This
addresses the processing problem of HBH, but there is still a problem
that the extension headers may push the transport headers too deep in
the packet for intermediate nodes to and they may drop the packet
(this is discussed in RFC9098). To solve the latter problem, I believe
we need more specific requirements than MUST support extensions
headers (which frankly is the current requirement and it's not
working).

> In addition, what's the situation when a network device is the ingress of a tunnel? So this network device can add the EH. Do the limitations also apply to this network device?

By adding the EH I assume you mean setting EHs in the encapsulating
IPv6 packet being sourced by the ingress device. The fact that this is
a tunneled packet is not pertinent, the ingress node is an IPv6 packet
so the host requirements are applicable. So the limits in this draft
would apply, however the common use case of tunnels is to transit a
limited domain so it's likely the ingress router would have knowledge
of the path so that it could use higher limits per the draft.

Tom

---

From: "Pascal Thubert (pthubert)" <[email-masked]>
Date: Fri, 24 Dec 2021 12:45:49 +0000

Agree with Joel

Tom’s draft cannot do much to existing hardware. The best hope is that it influences the software and hardware that are being developed. Meaning that it will take years to the draft to have a meaningful impact.

Looking at it that way it would be ideal to document not only the values but also the agenda.

For a given capacity of header buffering/processing we need baselines for

-When it is recommended that the hardware is available from vendors (per role of router)

-When it is recommended that the hardware be deployed

- which EH should be supported (per role of router)

That’s ambitious but can we really do with less?

Regards,

Pascal

Le 23 déc. 2021 à 23:22, Joel Jaeggli <[email-masked]> a écrit :

On 12/6/21 02:13, [email-masked]<mailto:[email-masked]> wrote:
...

   It should be noted that

Best regards,
Ole Troan, 6man chair

---

From: Tom Herbert <[email-masked]>
Date: Fri, 24 Dec 2021 08:07:06 -0800

> Tom’s draft cannot do much to existing hardware. The best hope is that it influences the software and hardware that are being developed. Meaning that it will take years to the draft to have a meani…

Hi Pascal,

I believe software programmable hardware changes the equation on this
to facilitate much faster deployment of protocols.

> -When it is recommended that the hardware is available from vendors (per role of router)
Baseline _minimal_ recommendation in the draft is that routers do not
drop packets that have up to sixty-four bytes of extension headers and
that HBH options are ignored.

> -When it is recommended that the hardware be deployed
Immediately :-) The anecdotal information from vendors is that parsing
buffers are typically at least 128 bytes which allows a sixty-four
bytes of extension headers, and ignoring HBH options is allowed by
RFC8200 which is over four years old. It may be the fact that the
majority of deployed devices already meet these minimum requirements
and we simply don't yet have the data showing that.

> - which EH should be supported (per role of router)
I don't think there's any expectations here other than minimal support
listed above (allow at least sixty-four bytes EH and ignore options if
they're not processed). Support for individual options, meaning
they're processed by a router as opposed to just being ignored, should
be discretionary and specific to the needs of a deployment. In
particular, hosts can never expect that any particular HBH option
would be universally processed by all routers in the Internet.

Tom

---

From: Brian E Carpenter <[email-masked]>
Date: Sat, 25 Dec 2021 08:51:14 +1300

Tom,

On 25-Dec-21 05:07, Tom Herbert wrote:
>> Tom’s draft cannot do much to existing hardware. The best hope 
is that it influences the software and hardware that are being developed. 
Meaning that it will take years to the draft to have a meaningful impact.
> to facilitate much faster deployment of protocols.

I've been waiting 20 years for that to become true. I hope you're right,
but even so there will be an enormous backlog of legacy devices which
are still stuck in the old equation. Even one such device on the path
is a problem.
  
>> Looking at it that way it would be ideal to document not only the values but also the agenda.

I have difficulty seeing the roadmap here, and it definitely needs to
be documented.

     Brian

>> I don't think can be held to be universally true for a couple of reasons. One is resource constraints, e.g. contiki's ipv6 stack can send and recieve packets with something on the order of 2KB of resident code. but also because of various forms of optimization that hosts may do to allow nics to perform datagram construction or disassembly in pursuit of performance. So while some hosts may be extremely flexible with respect to what they can send or recieve or the extent to which they are readily extensible to support new features others are less so, and may be variably so, e.g. supporting new icmp message types might be simple and costless, whereas supporting a new option to be parsed on delivery of a packet containing 
a tcp or udp stream might result in the need to disable an asic based optimization or bypass flow oriented policy in order to make use of a header.

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 24 Dec 2021 21:52:26 -0500

Brian

> I have difficulty seeing the roadmap here, and it definitely needs to

Gyan> See section 2  “Modern router architecture” of Processing of HBH
options draft.

NPUs are programmable thorough software and thus can make changes
immediately quick time to market for new features versus  fixed function
ASICs with limited flexibility.

https://datatracker.ietf.org/doc/html/draft-ietf-v6ops-hbh-00#section-2

> >> I don't think can be held to be universally true for a couple of
> reasons. One is resource constraints, e.g. contiki's ipv6 stack can send
> and recieve packets with something on the order of 2KB of resident code.
> but also because of various forms of optimization that hosts may do to
> allow nics to perform datagram construction or disassembly in pursuit of
> performance. So while some hosts may be extremely flexible with respect to
> what they can send or recieve or the extent to which they are readily
> extensible to support new features others are less so, and may be variably
> so, e.g. supporting new icmp message types might be simple and costless,
> whereas supporting a new option to be parsed on delivery of a packet
> containing
> a tcp or udp stream might result in the need to disable an asic based
> optimization or bypass flow oriented policy in order to make use of a
> header.
> >>
> >> While I think this limits proposed in this document seem realistic (104
> bytes of header) I'm leary of hard limits that might unreasonably set
> expectations of future extensibility to high.
> >>
-- 

*Gyan Mishra*

*M 301 502-1347*

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 24 Dec 2021 22:21:18 -0500

All,

On Fri, Dec 24, 2021 at 9:52 PM Gyan Mishra <[email-masked]> wrote:

> Gyan> A lot of the questions and issues discussed here are addressed in
> the above draft section 2 which provides a comprehensive analysis of the
> hardware HBH processing problem and possible solutions from a hardware POV
> and challenges.
>
>>
>>      Brian
>>
>> >>
>> >> While I think this limits proposed in this document seem realistic
>> (104 bytes of header) I'm leary of hard limits that might unreasonably set

*Gyan Mishra*

*M 301 502-1347*

---

From: Gyan Mishra <[email-masked]>
Date: Fri, 24 Dec 2021 23:17:31 -0500

Tom

On Tue, Dec 14, 2021 at 11:16 AM <[email-masked]> wrote:

> Tom,
>
> >>> The next and very hard step is to make it usable across the Internet
> as well. This involves building and deploying a new generation of ASICs and
> we must accept that it is a long term plan. One reason why current ASICs do
> not do much today is that there's no spec of what they are expected to do
> though there's a cost. We're stalled till we provide specs and value.
> >>>
> >>> We need to agree on a road map of how much HbH space the ASICs should
> be able to process in 5, 10, 15, and even 20 years (IPv6 is 25 years old so
> as this road map, it makes sense to make it STD track, because a lot od std
> track will depend on it.
> >>
> >> You allude to it in the first paragraph, but isn’t there two issues
> here? 1) how deep into the packet to parse 2) actual processing of the
> option in the header.
> >>
> >> I would assume 2 is orders of magnitude more costly than 1. And what
> would be the point of deploying ASICs parsing to <n> depth in packet unless
> they also implement real deployable options?
> >
> > Ole,
> >
> I can't recall if that ended up in an RFC or if it was just on the list.
> It might be something to include in the document.

Gyan> My thoughts are that as far as EH limits that definitely #1 packet
buffer and parsing and EH chaining Christmas tree affect of headers is an
overall IPv6 architectural issue that needs to be addressed and relates to
all extension headers and not just HBH.   #1 is an overarching issue but
does not diminish #2 as #2 hones in on a specific problem which HBH
processing.   #2 Option processing comes into play with HBH being deemed by
operators to be an attack vector and the punt to control plane slow path
issue.  So both are extremely important and so it’s really hard in my mind
to compare one against the other as both are critical issues to be
addressed.  Toms eh limits draft addresses #1 where #2 is the focus of
Hinden HBH by limiting the number of options as one method to make HBH
viable.  My worry with limiting the HBH options in Hinden draft which has
been mentioned is future scalability with hardware advancements,  how do
you set a HBH options limit that would be relevant 20+ years from now.

-- 

*Gyan Mishra*

*M 301 502-1347*

---

From: Tianran Zhou <[email-masked]>
Date: Sat, 25 Dec 2021 09:51:15 +0000

Hi Tom,
Happy holidays! Please see in line.
Tianran

-----Original Message-----
From: Tom Herbert [mailto:[email-masked]] 
Sent: Friday, December 24, 2021 2:49 PM
To: Tianran Zhou <[email-masked]>
Cc: Chengli (Cheng Li) <c.l=[email-masked]>; 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts

Hi TIanran,

The difference is that RFC2460 placed no limit on HBH, and in fact the requirement was that all intermediate nodes were expected to process all hop HBH. In practice this requirement is unrealistic and infeasible to support in an implementation (really hardware and software). The consequence of this requirement was that many vendors arbitrarily decided to not support HBH options at all and simply drop packets with them (the data in RFC7872 corroborate this). The provision in RFC8200 solves part of the problem by relaxing the requirement that all intermediate nodes must process extensions. This addresses the processing problem of HBH, but there is still a problem that the extension headers may push the transport headers too deep in the packet for intermediate nodes to and they may drop the packet (this is discussed in RFC9098). To solve the latter problem, I believe we need more specific requirements than MUST support extensions headers (which frankly is the current requirement and it's not working).

    ZTR> I agree with the first half statement. Let me try to understand the second half.
    I assume the limits specification is a further action beyond RFC8200. Right?
    You want to resolve the concern (from device vendor) how many EH supports is the base line, so that they can claim they conform with the standard.
    If so, I think you can still explicitly specify how many EHs(or length) the device must support.
    I do not understand why the limits on the host is better. 

By adding the EH I assume you mean setting EHs in the encapsulating
IPv6 packet being sourced by the ingress device. The fact that this is a tunneled packet is not pertinent, the ingress node is an IPv6 packet so the host requirements are applicable. So the limits in this draft would apply, however the common use case of tunnels is to transit a limited domain so it's likely the ingress router would have knowledge of the path so that it could use higher limits per the draft.

    ZTR> Your interpretation is exactly what I mean. 
    So my question is , whether this kind of network device need to conform with a lot of constraints(must, must not) in your draft.  
    As you said " it's likely the ingress router would have knowledge of the path so that it could use higher limits per the draft".
    Are these "higher limits" also considered in your draft? 
    Or would you like to explicitly exclude this scenario?

Tom

---

From: Haoyu Song <[email-masked]>
Date: Mon, 27 Dec 2021 17:53:05 +0000

Unfortunately, the software programmable hardware only provides flexibility (i.e., supporting customized headers). The limitation on header buffer size remains. If you need to look beyond that, you need to do packet recirculation which essentially halves the throughput.

Best,
Haoyu 

-----Original Message-----
From: ipv6 <[email-masked]> On Behalf Of Brian E Carpenter
Sent: Friday, December 24, 2021 11:51 AM
To: Tom Herbert <[email-masked]>; Pascal Thubert (pthubert) <pthubert=[email-masked]>
Cc: 6man WG <[email-masked]>; 6man Chairs <[email-masked]>
Subject: Re: Adoption call on HBH drafts - feedback on draft-herbert-6man-eh-limits-00

Tom,

> to facilitate much faster deployment of protocols.

I've been waiting 20 years for that to become true. I hope you're right, but even so there will be an enormous backlog of legacy devices which are still stuck in the old equation. Even one such device on the path is a problem.
  
>> Looking at it that way it would be ideal to document not only the values but also the agenda.

I have difficulty seeing the roadmap here, and it definitely needs to be documented.

     Brian

>> reasons. One is resource constraints, e.g. contiki's ipv6 stack can 
>> send and recieve packets with something on the order of 2KB of 
>> resident code. but also because of various forms of optimization that 
>> hosts may do to allow nics to perform datagram construction or 
>> disassembly in pursuit of performance. So while some hosts may be 
>> extremely flexible with respect to what they can send or recieve or 
>> the extent to which they are readily extensible to support new 
>> features others are less so, and may be variably so, e.g. supporting 
>> new icmp message types might be simple and costless, whereas 
>> supporting a new option to be parsed on delivery of a packet 
>> containing
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
>> .ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%4
>> 0futurewei.com%7C0460843a64654bf77e7308d9c716d274%7C0fee8ff2a3b240189
>> c753a1d5591fedc%7C1%7C1%7C637759723116573435%7CUnknown%7CTWFpbGZsb3d8
>> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
>> 3000&amp;sdata=Kzx%2Fnr5L6%2BzI5QFpFDIxBp01VvBYo0PGcCPXq6b7PH8%3D&amp
>> ;reserved=0
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
>> .ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%4
>> 0futurewei.com%7C0460843a64654bf77e7308d9c716d274%7C0fee8ff2a3b240189
>> c753a1d5591fedc%7C1%7C1%7C637759723116573435%7CUnknown%7CTWFpbGZsb3d8
>> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
>> 3000&amp;sdata=Kzx%2Fnr5L6%2BzI5QFpFDIxBp01VvBYo0PGcCPXq6b7PH8%3D&amp
>> ;reserved=0
>> https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww
>> .ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%4
>> 0futurewei.com%7C0460843a64654bf77e7308d9c716d274%7C0fee8ff2a3b240189
>> c753a1d5591fedc%7C1%7C1%7C637759723116573435%7CUnknown%7CTWFpbGZsb3d8
>> eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C
>> 3000&amp;sdata=Kzx%2Fnr5L6%2BzI5QFpFDIxBp01VvBYo0PGcCPXq6b7PH8%3D&amp
>> ;reserved=0
> uturewei.com%7C0460843a64654bf77e7308d9c716d274%7C0fee8ff2a3b240189c75
> 3a1d5591fedc%7C1%7C1%7C637759723116573435%7CUnknown%7CTWFpbGZsb3d8eyJW
> amp;sdata=Kzx%2Fnr5L6%2BzI5QFpFDIxBp01VvBYo0PGcCPXq6b7PH8%3D&amp;reser
> ved=0

--------------------------------------------------------------------
IETF IPv6 working group mailing list
[email-masked]
Administrative Requests: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ietf.org%2Fmailman%2Flistinfo%2Fipv6&amp;data=04%7C01%7Chaoyu.song%40futurewei.com%7C0460843a64654bf77e7308d9c716d274%7C0fee8ff2a3b240189c753a1d5591fedc%7C1%7C1%7C637759723116573435%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C3000&amp;sdata=Kzx%2Fnr5L6%2BzI5QFpFDIxBp01VvBYo0PGcCPXq6b7PH8%3D&amp;reserved=0
--------------------------------------------------------------------