Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

From: Greg White <[email-masked]>
Date: Thu, 8 Aug 2019 22:58:46 +0000

Hi Giuseppe,

Thanks for sharing your work.   At the TSVWG meeting at IETF105 it was mentioned that there was a hardware implementation of SCE running in a 25Gbps switch (1:47:30 in https://www.ietf.org/audio/ietf105/ietf105-placeducanada-20190726-1220.mp3).  I presume this is the implementation that was being referred to.

If I’m reading this correctly, the marking implementation in the switch is actually DCTCP-style (marking CE based on instantaneous queuing delay) in a single queue, rather than SCE-style.   The CE marks are then transformed into SCE marking by the receiver itself before being handed to dctcp-sce or reno-sce.  So, in essence this test appears to be a study of various AQM ramp options and modified ECN congestion responses in a homogeneous DCTCP datacenter environment, rather than being an SCE experiment specifically.

Could you provide some information on the buffer size?  It looks like it may be ~200us.   How does this marking approach compare to existing DCTCP deployments (i.e. do they generally use a hard marking threshold or do they similarly use a ramp)?  Have you thought about adjusting the ramp so that 100% marking occurs before you reach the tail drop limit?  I wonder if that would eliminate the packet drops.

For comparison, you may as well use stock DCTCP which would avoid the receiver needing to translate the CE marks into SCE marks, since DCTCP can just interpret the CE marks directly.   But, I didn’t think that the goal of SCE was to replace DCTCP in datacenters anyway…

Are the tables missing some data?  I don’t see the bandwidth/retries/packet drop data for the continuous TCP connections.

Also, do you have an explanation for Pic 1, where the ping time when the link is idle is almost double the ping time when the link is saturated?

Best Regards,
Greg

From: tsvwg <[email-masked]> on behalf of "Scaglione, Giuseppe" <[email-masked]>
Date: Tuesday, August 6, 2019 at 3:36 PM
To: "[email-masked]" <[email-masked]>
Subject: [tsvwg] Switch testing at 25G with ECN --SCE Draft

Greetings,

The attached document contains data collected testing with an Ethernet switch and the new TCP-dctcp-sce and TCP-reno-sce algorithms, draft-morton-taht-tsvwg-sce-00.

Best regards,

Giuseppe Scaglione
Distinguished Technologist
R&D Lab Hardware
Aruba Networks
Hewlett Packard Enterprise
8000 Foothills Blvd.  (MS 5555)
Roseville, CA   95747
Tel : 1-916-471-9189

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Thu, 8 Aug 2019 16:33:28 -0700 (PDT)

> Thanks for sharing your work.   At the TSVWG meeting at IETF105 it was mentioned that there was a hardware implementation of SCE running in a 25Gbps switch (1:47:30 in https://www.ietf.org/audio/ie…

As the person that made that comment I'll confirm that Yes, this is the implementation I was refering to.

> If I?m reading this correctly, the marking implementation in the switch is actually DCTCP-style (marking CE based on instantaneous queuing delay) in a single queue, rather than SCE-style.   The CE marks are then transformed into SCE marking by the receiver itself before being handed to dctcp-sce or reno-sce.  So, in essence this test appears to be a study of various AQM ramp options and modified ECN congestion responses in a homogeneous DCTCP datacenter environment, rather than being an SCE experiment specifically.

I'll defer to Giuseppe for details on that, but I do not believe this is simple dctcp style ce threshold marking when they are running in sce test mode.  See figure 1 and associated text.

As a person somewhat familiar with SCE I can state that dctcp ce marking as used in a data center and the end node response to that ce mark is infact basically the same as SCE, ie there is no latching of CE state until CWR at the receiver, and the marks are in proportion to the queue depth.  BUT by using a different ECT() marking and a different TCP feedback bit you can do both CE and SCE at the same time, which the classical data center dctcp modifications do not allow for.

> For comparison, you may as well use stock DCTCP which would avoid the receiver needing to translate the CE marks into SCE marks, since DCTCP can just interpret the CE marks directly.   But, I didn?t think that the goal of SCE was to replace DCTCP in datacenters anyway?

The goal of SCE is to provide a robust congestion mechanism, without having to be restricted to use in a datacenter.

> 
> Are the tables missing some data?  I don?t see the bandwidth/retries/packet drop data for the continuous TCP connections.
> 

I do have one for this, when the link is idle your receiver is not going to see the packet until a device interrupt occurs, which may be a good deal of time as these are not simple single packet/single interrupt type nics.  Basically when idle the ping packets are having to wait until either the poll threshold of the driver occurs, or something else causes the driver to interrupt.  At high packet rates your delivering a near continuous stream of interrupts to the CPU and thus your pings get processed by the receiver almost instantly.

> 8000 Foothills Blvd.  (MS 5555)
Content-Description: image001.jpg

[ image/jpeg is not supported, skipping... ]

Content-Description: image002.png

[ image/png is not supported, skipping... ]

-- 
Rod Grimes                                                 [email-masked]

---

From: "Scaglione, Giuseppe" <[email-masked]>
Date: Fri, 9 Aug 2019 00:00:05 +0000

Greetings,

Replying to Greg:

>>If I'm reading this correctly, the marking implementation in the switch is actually DCTCP-style (marking CE
>>based on instantaneous queuing delay) in a single queue, rather than SCE-style.   

The switch only knows RFC 3168 rules. That is, if packet is IP AND header is marked ECN capable (10 | 01) the packet can be remarked. The remarking probability is given by the formula on page 5:

remark_probability = probability_rise *(current_mem_utilization -mem_start_point) ( all % values)

In the paper, anywhere it says SCE testing the settings were:
probability_rise = 1
mem_start_point = 0
Current_mem_utilization --> this is dynamic, goes from 0-100% provided by the HW queue.

Therefore, for SCE testing the switch was remarking purely function of the queue memory utilization:

remark_probability =  current_mem_utilization  

Which produces that 'early warning' of "some congestion" which is mentioned in the sce-draft, and would never occur in traditional switch setting for CE remarking. 
Yes -- all testing was using 1 of the 8 egress queues of the switch ports. It would be easy to do a multi queue one.

>>Could you provide some information on the buffer size?  It looks like it may be ~200us

Sorry cannot share any details on the HW and its memory at this point.  

>>How does this marking approach compare to existing DCTCP deployments (i.e. do they generally use a 
>>hard marking threshold or do they similarly use a ramp)?

In general, some hardware implementation will start remarking at given queue depth (hard start) and then they apply a probability curve more or less aggressive. Other hardware (like the one in the experiment) gets continuous feedback on the queue depth and then applies the probability curve function of such memory usage. 
Looking at the equation above would be interesting to try to play with the probability_rise as well, which was always left at 1.

>>I do have one for this, when the link is idle your receiver is not going to see the packet until a device 
>>interrupt occurs, which may be a good deal of time as these are not simple single packet/single interrupt 
>>type nics.  Basically when idle the ping packets are having to wait until either the poll threshold of the 
>>driver occurs, or something else causes the driver to interrupt.  At high packet rates your delivering a near 
>> continuous stream of interrupts to the CPU and thus your pings get processed by the receiver almost 
>>instantly.

I agree with Rodney. I am not sure if the Kernel is using NAPI and doing Interrupt driven Packet Processing and then switching to polled -- if just polled the explanation makes perfect sense.

Best Regards!
Giuseppe

-----Original Message-----
From: Rodney W. Grimes [mailto:[email-masked]] 
Sent: Thursday, August 8, 2019 4:33 PM
To: Greg White <[email-masked]>
Cc: Scaglione, Giuseppe <[email-masked]>; [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

> Hi Giuseppe,
> 
> Thanks for sharing your work.   At the TSVWG meeting at IETF105 it was mentioned that there was a hardware implementation of SCE running in a 25Gbps switch (1:47:30 in https://www.ietf.org/audio/ietf105/ietf105-placeducanada-20190726-1220.mp3 ).  I presume this is the implementation that was being referred to.

---

From: Greg White <[email-masked]>
Date: Fri, 9 Aug 2019 19:16:34 +0000

Rodney and Giuseppe, thanks for the clarification.

Just to be super clear, this isn't a hardware implementation of SCE running at 25Gbps.   It is essentially using a 25Gbps DCTCP switch implementation (with an adjustable marking probability slope) to study the behavior of the two experimental SCE congestion controllers in a data center environment.   If this is considered to be a scenario for deployment of SCE congestion controllers, then I would again suggest running the experiment also with the stock DCTCP congestion controller for comparison, since that is what you'd be trying to displace.

-Greg

On 8/8/19, 6:00 PM, "Scaglione, Giuseppe" <[email-masked]> wrote:

    Greetings,

    

    

    

    

    
    
    Best Regards!
    Giuseppe

---

From: "Scaglione, Giuseppe" <[email-masked]>
Date: Fri, 9 Aug 2019 20:44:48 +0000

Greg,

>> Just to be super clear, this isn't a hardware implementation of SCE running at 25Gbps.

I am not sure I follow. The Test setup section of the paper clearly describes the hardware used -- severs, switch, cables. 
What I cannot disclose at this point is the exact model and characteristics of the HPE Aruba Switch used. Yet, it is a "real" Ethernet Switch, providing 25Gbps connectivity, configured to do bridging across the four ports and implementing RFC3168 with the ECN remarking configuration described in my previous email and on the paper.

Cheers,
Giuseppe 

-----Original Message-----
From: Greg White [mailto:[email-masked]] 
Sent: Friday, August 9, 2019 12:17 PM
To: Scaglione, Giuseppe <[email-masked]>; [email-masked]
Cc: [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

-Greg

    Greetings,

    

    

    

    

    
    
    Best Regards!
    Giuseppe

---

From: Jonathan Morton <[email-masked]>
Date: Fri, 9 Aug 2019 23:58:32 +0300

> On 9 Aug, 2019, at 11:44 pm, Scaglione, Giuseppe <[email-masked]> wrote:
> 

I think the issue is with the way the switch itself marks with CE, not ECT(1).  It's a limitation I think is worth acknowledging and, hopefully, finding a way to remove.

 - Jonathan Morton

---

From: Greg White <[email-masked]>
Date: Fri, 9 Aug 2019 21:36:25 +0000

Right.  Per the SCE method, the switch would mark ECT(1) using a ramp, and then when the ramp would exceed 100% marking, it would change to using CE.  

The implementation discussed here marks CE using a ramp, and when the ramp would exceed 100% marking, it changes to packet drop.  This, as you said, is simply RFC3168 ECN marking, not SCE ECN marking.

I was just trying to set the record straight.  There was a claim made that a switch vendor had implemented SCE-style packet marking in hardware at 25Gbps, which wasn't accurate.

-Greg

    
    
     - Jonathan Morton

---

From: "Scaglione, Giuseppe" <[email-masked]>
Date: Fri, 9 Aug 2019 21:42:58 +0000

Greg,

We are working -- and in beta -- with having the switch natively set SCE bits instead of CE and removing the iptable configuration on the target server.  Yet, I do not expect any different result since the TCP-SCE stack would react the same.

Regards,
Giuseppe Scaglione

-----Original Message-----
From: Greg White [mailto:[email-masked]] 
Sent: Friday, August 9, 2019 2:36 PM
To: Jonathan Morton <[email-masked]>; Scaglione, Giuseppe <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

-Greg

    
    
     - Jonathan Morton

---

From: Greg White <[email-masked]>
Date: Fri, 9 Aug 2019 21:54:18 +0000

Agreed.  I would not expect any different result either, which begs the question why does SCE need two different signals (ECT(1) and CE) in a datacenter environment?

-Greg

On 8/9/19, 3:43 PM, "Scaglione, Giuseppe" <[email-masked]> wrote:

    Greg,

    Regards,
    Giuseppe Scaglione

    -----Original Message-----
    From: Greg White [mailto:[email-masked]] 
    Sent: Friday, August 9, 2019 2:36 PM
    To: Jonathan Morton <[email-masked]>; Scaglione, Giuseppe <[email-masked]>
    Cc: [email-masked]; [email-masked]
    Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

    
    
    -Greg

    

         - Jonathan Morton

---

From: Jonathan Morton <[email-masked]>
Date: Sat, 10 Aug 2019 02:29:17 +0300

> On 10 Aug, 2019, at 12:54 am, Greg White <[email-masked]> wrote:
> 

Simple: to allow using the same protocol *outside* the datacentre, because it coexists cleanly with existing RFC-3168 infrastructure and endpoints.  That was always the chief failing of DCTCP.

 - Jonathan Morton

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Fri, 9 Aug 2019 17:02:40 -0700 (PDT)

> The implementation discussed here marks CE using a ramp, and when the ramp would exceed 100% marking, it changes to packet drop.  This, as you said, is simply RFC3168 ECN marking, not SCE ECN marki…

Purely due to a choice to an expedient experiment.  This is not "simply RFC3168 ECN marking" which would occur at the point the switch was about to start dropping, it is "linear ramp response proportionaly to queue depth SCE marking" using the CE bit pattern.  I can not get into the details as to why this was done this way.

RFC3168 ECN marking is already in the switch, the switch was modified to behave in a different manner, using the
RFC3168 CE bits.  This different manner was turned OFF to do the dctcp tests and turned ON to do the dctcp-sce tests.

> I was just trying to set the record straight.  There was a claim made that a switch vendor had implemented SCE-style packet marking in hardware at 25Gbps, which wasn't accurate.

It was and still is accurate, the only inaccuracy is that the switch emmits CE rather than SCE, but the response form of the CE bits are driven by an SCE algorithm NOT the single threshold data center CE marking normally done by the switch.

> -Greg
> 
> 
> 
> ?On 8/9/19, 2:58 PM, "Jonathan Morton" <[email-masked]> wrote:
> 

Yes, that is the one non-SCE aspect to the implementation, use of CE instead of ECT(1), BUT the driving algorithm to create these marks is NOT classic RFC3168, but is linear ramp draft-morton-tsvwg-sce-00.

---

From: Dave Taht <[email-masked]>
Date: Sat, 10 Aug 2019 09:32:07 -0700

"Scaglione, Giuseppe" <[email-masked]> writes:

> Greg,
>
> We are working -- and in beta -- with having the switch natively set SCE bits
> instead of CE and removing the iptable configuration on the target server.  Yet,

Very cool.

> I do not expect any different result since the TCP-SCE stack would react the
> same.

In looking over the data I see that there were tcp "retries" which I
take to be "retransmits", even when there was no loss noted at the
switch.

This implies there was packet loss at the client or server (most likely
client, linux not being a R/T OS, getting stuff off the receive ring
rapidly enough is a PITA with napi and other processes interfering)

My guess is that the dctcp version under test did not do anything to
note the loss there and reduce cwnd and had better throughput
sometimes?, and the reno-sce version did?

Regardless tracking down the source of the loss and retransmit and
fixing it would help for a direct comparison. Increasing the rx ring
size, pinning stuff to cpus, and other methods to find where the
mising packets went will lead to more comparable results.

> Right.  Per the SCE method, the switch would mark ECT(1) using a ramp, and then
> when the ramp would exceed 100% marking, it would change to using CE.
>
> The implementation discussed here marks CE using a ramp, and when the ramp would
> exceed 100% marking, it changes to packet drop.  This, as you said, is simply
> RFC3168 ECN marking, not SCE ECN marking.
>
> I was just trying to set the record straight.  There was a claim made that a
> switch vendor had implemented SCE-style packet marking in hardware at 25Gbps,
> which wasn't accurate.
>
> -Greg
>
>
>

---

From: Wesley Eddy <[email-masked]>
Date: Sat, 10 Aug 2019 21:31:52 -0400

> Purely due to a choice to an expedient experiment.  This is not "simply RFC3168 ECN marking" which would occur at the point the switch was about to start dropping, it is "linear ramp response propo…

The advice in 3168 is to do marking based on average queue length, but 
*not* just before it needs to start dropping.  Is it more correct 
instead to understand what you're saying as the experiment being based 
on using the instantaneous queue length rather than average?

> RFC3168 CE bits.  This different manner was turned OFF to do the dctcp tests and turned ON to do the dctcp-sce tests.

Since the marking probability function and marking decision isn't part 
of 3168/ECN, the description here sounds like the switch is always 
conforming to 3168 behavior, but just using some custom decision logic 
for marking.

Just trying to be clear on what we're really talking about ...

---

From: Jonathan Morton <[email-masked]>
Date: Sun, 11 Aug 2019 10:27:36 +0300

> The advice in 3168 is to do marking based on average queue length, but *not* just before it needs to start dropping.  Is it more correct instead to understand what you're saying as the experiment being based on using the instantaneous queue length rather than average?

I think that is right.  In software we like to use the actual sojourn time within the queue of the candidate packet, but in hardware it is often convenient to use the queue length instead.

More generally, I notice it's a surprisingly common misconception on this list that RFC-3168 states that marking should happen when, or just before, packets would otherwise be dropped.  A little critical thought will show that this is absurd; AQM is supposed to leverage congestion-control behaviour to keep the queue depth well away from the hard tail-drop zone on average.  What the spec *actually* says is that an AQM should substitute packet drops for marking, when the packet is not ECT and therefore cannot be marked; this is practically the inverse of the misconception noted.

In particular, I believe marking should definitely occur by the time the queue is half-full, or more precisely so that there is a full BDP (including queuing delay already incurred at the point the mark is applied) of space remaining in the queue.  Why?  To accommodate the RTT of control feedback delay before the signal to exit slow-start takes effect at the queue, during which the cwnd of a typical TCP will double.  Most AQMs do in fact satisfy that principle.

>> RFC3168 ECN marking is already in the switch, the switch was modified to behave in a different manner, using the RFC3168 CE bits.  This different manner was turned OFF to do the dctcp tests and turned ON to do the dctcp-sce tests.
> 
> Since the marking probability function and marking decision isn't part of 3168/ECN, the description here sounds like the switch is always conforming to 3168 behavior, but just using some custom decision logic for marking.
> 

I think it's best to consider this an experimental prototype, in which the goal is to investigate whether the obtained behaviour is desirable and useful before continuing with more involved development.  From what I hear, and without going into needless detail, it was easier in the short term to set it up with CE marking and transform that into SCE codepoints at the receiver.  This was always going to be a temporary expedient.

Also from what I hear, experiments had previously been conducted using CE marking and the standard version of DCTCP in Linux, but without anywhere near the level of success seen in the results just posted.  It seems likely that faults in the DCTCP implementation were responsible, but this in turn raises the question of how well maintained that piece of code is, and how such faults were allowed to persist in the mainline codebase.

Linux is normally associated with better development practices than that.  The most reasonable explanation I can come up with is that DCTCP is not actually in widespread use, so when faults appeared they were not noticed by any party interested in having them fixed.  Or, perhaps more disturbingly, DCTCP *is* in widespread use in closed datacentre environments, but when the faults arose in production their effects were not recognised.

So in practice, this controlled experiment has validated certain principles associated with the DCTCP response function (well, a simplified version of it), and also shown that other response functions are similarly effective, when applied to an ultra-low-latency network.  It has also shown that the proposed feedback mechanism for SCE signals on TCP connections is viable, so the relatively complex AccECN mechanism isn't obviously needed. We have shown other results which show similarly good behaviour on Internet-scale latencies with real SCE marking, using a broadly similar marking function.  Overall these are encouraging results for us, and work will continue accordingly.

 - Jonathan Morton

---

From: "Black, David" <[email-masked]>
Date: Sun, 11 Aug 2019 11:29:18 +0000

Posting as an individual, not a WG chair.

> > Agreed.  I would not expect any different result either, which begs the
> question why does SCE need two different signals (ECT(1) and CE) in a
> datacenter environment?
> 
> Simple: to allow using the same protocol *outside* the datacentre, because
> it coexists cleanly with existing RFC-3168 infrastructure and endpoints.  That
> was always the chief failing of DCTCP.
> 
>  - Jonathan Morton

[David>] That coexistence is also interesting in a datacenter environment, as it stops things from breaking badly if RFC-3168 TCP traffic and DCTCP-SCE traffic are mixed (where DCTCP-SCE receivers are paying attention to both SCE and CE marks).   As for why or how this sort of traffic mixing could happen, the reasons start with "To err is human."

---

From: "Black, David" <[email-masked]>
Date: Mon, 12 Aug 2019 14:34:07 +0000

Writing as an author of RFC 3168, not as a WG chair ...

> More generally, I notice it's a surprisingly common misconception on this list
> that RFC-3168 states that marking should happen when, or just before, packets
> would otherwise be dropped.  A little critical thought will show that this is
> absurd; AQM is supposed to leverage congestion-control behaviour to keep
> the queue depth well away from the hard tail-drop zone on average.
> What the spec *actually* says is that an AQM should substitute packet drops
> for marking, when the packet is not ECT and therefore cannot be marked;
> this is practically the inverse of the misconception noted.

+1 - RFC 3168 mandates drop-equivalence of CE, *not* tail-drop-equivalence, as can be seen from the numerous mentions of AQM and the RED AQM in RFC 3168's text (yes, we've moved on from RED, but RFC 3168 was written nearly 2 decades ago).  Tail-drop equivalence is allowed, however, in practice, most CE marks are generated by AQMs, e.g., ...

For completeness, I should note that RFC 8511 (Experimental) "TCP Alternative Backoff with ECN (ABE)" allows the TCP sender reaction to reported CE marks to differ from the TCP sender reaction to drops - the TCP sender is allowed to back off by a smaller amount on the assumption that CE marks have been applied by AQMs, as opposed to last-ditch tail-drop-avoidance mechanisms.

Thanks,
--David

________________________________________
From: tsvwg [[email-masked]] on behalf of Jonathan Morton [[email-masked]]
Sent: Sunday, August 11, 2019 3:27 AM
To: Wesley Eddy
Cc: [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

[EXTERNAL EMAIL]

 - Jonathan Morton

---

From: "Scaglione, Giuseppe" <[email-masked]>
Date: Mon, 12 Aug 2019 15:28:32 +0000

Hi Greg,

My humble opinion based on the testing presented is that with SCE remarking purely function of the queue depth, the TCP stack has a notion of congestion present in the TCP connection that is independent of the buffering capability of the switch queue. The TCP stack is seeing a "5%" of packet remarked for example, which means somewhere in the network some queue is seeing "some" oversubscription period. 

While with CE RFC3168 you know that remarking typically only starts after a X% of queue depth (user configured). And that X is function of the queue depth and memory available, and if the X is not 'tuned' you either get not optimal link utilization or packet drops. 

Basically, it seems like having an 'early warning' marking makes the SW implementation of the TCP algo easier and more predictable, instead of relying on a switch configuration of X.

Yes -- more testing is required.

Best Regards,
Giuseppe  

-----Original Message-----
From: Greg White [mailto:[email-masked]] 
Sent: Friday, August 9, 2019 2:54 PM
To: Scaglione, Giuseppe <[email-masked]>; Jonathan Morton <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

-Greg

    Greg,

    Regards,
    Giuseppe Scaglione

    -----Original Message-----
    From: Greg White [mailto:[email-masked]] 
    Sent: Friday, August 9, 2019 2:36 PM
    To: Jonathan Morton <[email-masked]>; Scaglione, Giuseppe <[email-masked]>
    Cc: [email-masked]; [email-masked]
    Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

    
    
    -Greg

    

         - Jonathan Morton

---

From: <[email-masked]>
Date: Tue, 13 Aug 2019 06:33:03 +0000

Hi Giuseppe,

you are basically right. In your test, you were working with different set-ups. With one, marking started initially, with another, marking started after a settable queue threshold. Queues could be of different depths. One can imagine drop functions which aren't linear from 0 - 100%. That is to say, there are some variables whose configurations are likely unknown to an endpoint, which impact what is expressed by an x% packet mark of a bottleneck facing a queue build up.

You close with "makes the implementation of TCP more predictable", which I agree to. To me an interesting question is, to which extent the overall mechanism benefits from standardization. A simple standardized marking and queuing behavior resulting in a reliable and fairly good transport management for a range of RTTs and BDPs or queue depths - would be great. At least having an understanding which enhanced AQM functions and ranges of settings of the above variables make sense would be a fair standardization result too.

Regards,

Ruediger

-----Ursprüngliche Nachricht-----
Von: tsvwg <[email-masked]> Im Auftrag von Scaglione, Giuseppe
Gesendet: Montag, 12. August 2019 17:29
An: Greg White <[email-masked]>; Jonathan Morton <[email-masked]>
Cc: [email-masked]
Betreff: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

Hi Greg,

Best Regards,
Giuseppe  

-----Original Message-----
From: Greg White [mailto:[email-masked]] 
Sent: Friday, August 9, 2019 2:54 PM
To: Scaglione, Giuseppe <[email-masked]>; Jonathan Morton <[email-masked]>
Cc: [email-masked]; [email-masked]
Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

-Greg

    Greg,

    Regards,
    Giuseppe Scaglione

    -----Original Message-----
    From: Greg White [mailto:[email-masked]] 
    Sent: Friday, August 9, 2019 2:36 PM
    To: Jonathan Morton <[email-masked]>; Scaglione, Giuseppe <[email-masked]>
    Cc: [email-masked]; [email-masked]
    Subject: Re: [tsvwg] Switch testing at 25G with ECN --SCE Draft

    
    
    -Greg

    

         - Jonathan Morton

---

From: Gorry Fairhurst <[email-masked]>
Date: Tue, 13 Aug 2019 08:37:53 +0100

It seems you are comparing with RFC3168, but have you looked also at the 
updated response in RFC8511. If you are using shallow marking, this 
would seem to be giving you the differentiation between buffer over flow 
(loss) and queue building (CE) that you mention?

Gorry

On 12/08/2019, 16:28, Scaglione, Giuseppe wrote:
> To: Scaglione, Giuseppe<[email-masked]>; Jonathan Morton<[email-masked]>
> On 8/9/19, 3:43 PM, "Scaglione, Giuseppe"<[email-masked]>  wrote:
>
>      Greg,
>
>      To: Jonathan Morton<[email-masked]>; Scaglione, Giuseppe<[email-masked]>
>      On 8/9/19, 2:58 PM, "Jonathan Morton"<[email-masked]>  wrote:
>
>          >  On 9 Aug, 2019, at 11:44 pm, Scaglione, Giuseppe<[email-masked]>  wrote:
>          >

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Tue, 13 Aug 2019 10:11:35 -0700 (PDT)

Gorry,
I am not certain which "you" are asking about RFC8511, but if it is
the SCE developers that is much about what this test was, it was looking
at SCE type response (similiar to RFC8511, but not the same) at very
high bandwidth very short delay type paths.

Within SCE we have tried different responses to ESCE in the sender,
and at present I believe the published code is using a -sqrt(ackbytes)
as the backoff, which is what these tests results are from.

The work in these areas is ongoing, and we have had several different
funtions here (the very first was simply -(0.5 * ackbytes), but found
to be a bit over responsive).

As SCE gathers more data it is probable that adjustments to both
the marking function and the response function are going to need
to be made, infact iirc there are 3 existing marking functions
(codel, fg_codel, pie) and 2 new ones (LFQ, NCQ) being worked on.

But it is unlikely until wide scale experimentation that a standard
set of functions can be identified.  This test data shows probably
the simplest of possible implementations, and it was pleasing
to see positive test results without any iterative design process,
or tweaking for this test.

It is also possible that you are suggesting that the non-SCE
tests might be repeated using RFC8511 ABE type response in the
DCTCP used, I too believe that would be a meaningful data point.

> > ?On 8/9/19, 3:43 PM, "Scaglione, Giuseppe"<[email-masked]>  wrote:
> >
> >      Greg,
> >

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 21 Aug 2019 09:03:33 +0100

Sorry - That helps explain a little.

My email wanted to note that I had seen results comparing the new 
methods with the TCP behaviour specified in RFC3168, whereas RFC8511 was 
the current specification for the transport behaviour for CE marking of 
ECT(0).  RFC8511 was published as an experimental RFC - to collect 
experience with  deployment  and confirm acceptable safety.

Gorry