Subject: [IPsec] Data Center changes to ESP, Re: leading versus trailing ICV

From: Michael Richardson <[email-masked]>
Date: Wed, 16 Sep 2020 18:02:04 -0400

Hi, I finally got to watching your presentation on the IETF youtube 
channel.  the illustration at https://youtu.be/IrNsFAPhx-Q?t=3410, which 
I guess is also at: 
https://www.ietf.org/proceedings/108/slides/slides-108-ipsecme-proposed-improvements-to-esp-01 
slide 3.

It is unfortunate that the youtube video does not capture the jabber 
discussion at all, and I have to go find that log.

As Bill Simpson pointed out, you don't need a new ESP protocol number 
because the layout of the ESP packet is already essentially driven by 
the SPI#.  The business of using this window,sender and sequence number 
as the IV might be AES-AEAD specific.  I'm sure that it can apply to a 
number of other ciphers, but perhaps not to classic 
3DES+MD5^WAES-128+SHA256.

I think that a way to negotiate this is as if it was unique cipher.
i.e: 
https://www.iana.org/assignments/ikev2-parameters/ikev2-parameters.xhtml#ikev2-parameters-5
which is Expert Review.  I agree with Tero that this is outside of the 
charter.
I suggest that you ask for an allocation, write some code, document it 
an individual ID and report on the WG on how well this works for you, 
and on whether the hardware people are happy with the results.

---

From: Toerless Eckert <[email-masked]>
Date: Thu, 17 Sep 2020 08:57:05 +0200

On Thu, Aug 06, 2020 at 08:22:28PM +0300, Tero Kivinen wrote:
> William Allen Simpson writes:
> > > So it might make sense to have the ICV at the end because it is
> > > likely cache hot when needed.
> > 
> > But after removing padding for these stream algorithms, then the ICV is
> > very likely not aligned.  For zero-copy RDMA, it is rather inconvenient.
> > And the IP header cache lines are likely still hot.
> > 
> > Anyway, that's why I'd like to consider at least a negotiated option --
> > as long as it's possible to implement efficiently in Linux and others.
> > We need to hear from more implementers.
> 
> I think it would be a bad idea to have such option. Yes, it might
> offer small gains on environments where the option is picked exactly
> when both ends like it, but when one ends pick it in a way which other
> end does not like, we usually end up big performance penalties.

If IKEv2/IPsec will not allow to negotiate the mutually best option,
then IMHO IKEv2/IPsec option negotiation needs to be improved.

But for a new option, i can not see short-term issues: Nodes for whom the new option
will not provide performance enhancements will simply not implement them.
Of course, when the mayority of implementations get a benefit of a new option and
the next profile for IPsec makes it mandatory, then those other implementation
will start having a challenge without better negotiations.

> And also it again multiples the testing effort as now you need to add
> to your test suites this combined with all posible ciphers etc.
> 
> This is was one of the main problems with IKEv1, there were so many
> different combinations of different options that to be able to test
> all of them required thousands or tens of thousands of test cases.

Automated regression testing. 

There is certainly the need to distinguish profiling of options between more
agile managed implementations and those who are not.

> Example of similar optimzation causing problems was found during
> interop events when some version (I think it was Linux) was sending
> fragmented packets in reverse order, so the first network packet
> sent/received was the last fragment. The idea was that when receiver
> saw that last fragment it can immediately know how big the final
> packet will be and it can allocate big enough buffer for the packet.
> 
> Then when you combined that with IPsec with per flow policy, meaning
> each TCP/UDP flow might be using different SA, that meant that SGW
> required to store all fragments in memory until it got the last packet
> from the network, which was the first fragment, and only after that it
> could check whether this packet is allowed to pass, and which SA it
> needs to use.
> 
> Then it sent the fragments out, but there was lots of added latency
> because of this. Actually I think there were also implementations
> which did not even store the later fragments, they simply checked the
> later fragments, and found out that they have not seen the first
> fragment that would allow them to be passed, so they dropped them.
>
> The SGW of course then sent the frames out in order, so that the
> receiving SGW can efficently do exit tunnel checks (i.e., check from
> the first fragment that this packet should be allowed, and match later
> fragments with same fragment id to that, and allow them to passed),
> and it then did not need to do same buffering. Of course the final
> destination host now did not benefit at all from this optimization as
> it was negatied by the SGWs in the middle.

If insights like there would be documented more prominentaly, like in a
living IETF document, then they would fsater trickle through implementers.

> So immediately when the options to use start to depend on the
> platform, implementation etc it gets harder and harder to find out
> which will be the optimal combination between two devices, and they
> might end up using suboptimal feature set. And all of those add more
> complexity and combinations that needs to be tested.

Before the next profile changes as to what options SOULD/MUST be supported,
there could be a collection round of evidence: If a new protocol option
has documented performance results in comparison to prior MUST options
that show e.g.: double performance, you will have a hard time to reject
it. If it doesn't have such data, you will have an easy way to reject it.

> Of course if this option is added, it should be something like IPCOMP
> or transport mode, meaning it is off by default, and everybody MUST
> implement case where it is not enabled, then you would only implement
> (and propose) it on cases where it actually benefits you, and
> everybody else would never ever implement it.

I think thats what i also said above.

Cheers
    Toerless

> -- 
> [email-masked]
> 
> _______________________________________________
> IPsec mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ipsec

-- 
---
[email-masked]

---

From: Tero Kivinen <[email-masked]>
Date: Thu, 17 Sep 2020 13:59:30 +0300

Toerless Eckert writes:
> If insights like there would be documented more prominentaly, like
> in a living IETF document, then they would fsater trickle through
> implementers.

That kind of things were usually found in the interop events where
there were few dozen vendors testing againt each other. We did
document some of those issues in documents like RFC4718 IKEv2
Clarifications and Implementation Guidelines. Most vendors are not
very keen on telling how they actually manage to get the performance
gains they do, because that is one of their benefits, if they are 3%
faster than competitor because they actually modify operating system
so that it always makes sure there is space at the end of packet for
ICV.

It is much easier to get vendors to disclose, issues that affect
interoperability than issues that affect performance. And even those
cases they do no want company names to be mentioned. 

They might tell their secret performance enhancing tricks during
friendly talk with other implementor, but might not want this to be
written down (as then they would need permission from the management,
which would most likely say no way).

> it. If it doesn't have such data, you will have an easy way to reject it.

If there would be someone who finds a way to double the performance
then yes, that would be huge difference, but when we are talking
implementations which should be able to use 95% of the capacity of the
network link anyways, the performance gains are usually much smaller.

For large packets the speed is usually restricted either by the
network speed, or the raw crypto speed. For small packets the
issues are quite often in general operating system overhead (i.e.,
even without crypto things tend to be slow). 
-- 
[email-masked]

---

From: Michael Richardson <[email-masked]>
Date: Fri, 18 Sep 2020 18:21:29 -0400

Michael Richardson <[email-masked]> wrote:
    > Hi, I finally got to watching your presentation on the IETF youtube channel.
    > the illustration at https://youtu.be/IrNsFAPhx-Q?t=3410, which I guess is
    > also at:

My attempt to CC Michael Rossbert failed. I was reading off the slides from a
youtube paused.  I hope he is on the ML.

    > I think that a way to negotiate this is as if it was unique cipher.

BTW: I recognize that this might require a new value for each cipher.

As such, it's not a great long term solution, but I would claim that it
probably applies to a few ciphers very specifically at first.  

Once the new layout is so popular, then we could persue some other way to do
this.  Probably that means the same Notify() mechanism we use for TRANSPORT_MODE/etc.
I'm not especially fond of this architecturally, but it certainly works.

-- 
]               Never tell me the odds!                 | ipv6 mesh networks [ 
]   Michael Richardson, Sandelman Software Works        | network architect  [ 
]     [email-masked]  http://www.sandelman.ca/        |   ruby on rails    [

---

From: Michael Rossberg <[email-masked]>
Date: Mon, 21 Sep 2020 10:38:49 +0200

Hi,

> I'm not especially fond of this architecturally, but it certainly works.

I guess this would be an option for our particular problem. Nevertheless, I agree the inflationary use
of IDs is a problem. Also, I see the possibility of confusion of the readers, ie. having three AES-GCM
modes.

This would also reduce the number of interop tests, as we would have one cipher that works a little
different. In contrast of having an option for full 64-bit sequence numbers, multiple replay windows,
etc. and a combination of each. Hence, in a data center environment one could simply use two code
paths depending on the IKE negotiation.

---

From: Michael Richardson <[email-masked]>
Date: Mon, 21 Sep 2020 11:03:47 -0400

> I guess this would be an option for our particular
    > problem. Nevertheless, I agree the inflationary use of IDs is a
    > problem. Also, I see the possibility of confusion of the readers,
    > ie. having three AES-GCM modes.

I don't think that the reader will be confused.

    > This would also reduce the number of interop tests, as we would have
    > one cipher that works a little different.

Yes, and likely implemented in a non-modular streamlined (or data pipelined)
way in hardware.  Hardware is not going to have more than two ciphers.
If as many as two.

Again, I suggest you write an Informational document, as for a code point,
and submit through ISE.

--
Michael Richardson <[email-masked]>   . o O ( IPv6 IøT consulting )
           Sandelman Software Works Inc, Ottawa and Worldwide