Subject: Spin bit discussion - where we're at

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Nov 2017 19:06:48 +1100

Hi everyone,

First, thanks to everyone for their input to date on this issue, whether as part of the Design Team (DT), in the meetings, or on the list. It'd good to see that we can stay civil and professional even when discussing contentious issues.

Since the DT didn't return a recommendation, several people have asked how we're going to move forward. After talking to our Area Director, we'd like to share our current thinking in this area.

The most important things here are to establish that there isn't any significant security or privacy exposure associated with proposed mechanism(s) to improve the manageability of QUIC, and to assure that the mechanism we ship actually serves its purpose. How we assess these two things are fundamentally different; security and privacy exposures are notoriously difficult to find (often only becoming apparent years after a protocol is adopted), while value to network operators is (comparatively) much easier to assert.

We can't prove that there is *no* security or privacy exposure (see: Russell's teapot), but we can take reasonable steps to assess proposals for risk. The Design Team performed this assessment for the Spin Bit proposal, and did not identify any immediate exposure. That does not mean it does not exist, but we've done a reasonable first review for that proposed mechanism. Further review of the Spin Bit can take place by circulating a full specification among security researchers (see below).

Importantly, any further proposals (e.g., for a "loss bit", or for alternative means of determining RTT) will need to undergo similar review and will therefore require a similar level of documentation.

There are a number of things that we think will help clarify the value provided by the Spin Bit:

1) A description of the use case(s) that motivate this proposal. We understand that the goal is to measure RTT, but some people are still unclear as to why that's necessary to operate a network. Detailed scenarios and ideally real-world examples (e.g., from TCP) would help tremendously. Saying "I need to debug the network" is not enough detail.

2) An actual protocol specification for the Spin Bit, including Privacy Considerations. There seems to be a fairly good common understanding of it (especially in the DT), but it would be helpful to have it written down, so we can make sure we're talking about the same thing. If it were proposed as an extension, current implementations (reminder: we have 12) could implement it to see how it works on the network. Doing so would also give the security research community something to look at (see above).

3) Data showing that the Spin Bit is fit for purpose under real-world conditions. Piet's experiment is a start here, but we need much more. Analysis of how well that data meets the use cases is also necessary. This might form part of the Manageability Considerations of the draft.

4) A comparison of other potential solutions to measuring delay, and their relative merits. Again, ideally with detailed, real-world data.

5) An exploration of what the effects of deploying the spin bit are likely to be. E.g.: Will even the perception of privacy issues change endpoint behaviour? Will networks treat traffic differently based upon the spin bit? If so, can that be gamed? Will firewalls and other gateways police spin bit behaviour (e.g., to prevent data loss), and what effects will that have?

We're looking for volunteers to start drafting one or more documents along these lines. If you're interested, please indicate this on-list, so you can coordinate with other interested parties. They won't (yet) be WG documents, but since the DT didn't resolve the issue, we will discuss them in the WG.

In terms of timeline - resolving this is NOT a blocking issue; i.e., we can discuss this concurrently with other work on the protocol, and because it's such a small change in the packet layout, we can add it right before shipping the protocol to the IESG.

In light of that, we will NOT discuss this (or related) issues at the Melbourne interim. That will allow the people attending to focus on continued protocol development, and means that people don't feel the need to travel to discuss just this issue. We would like to be able to move the discussion forward at IETF101 in London, provided that significant progress is made on the items above.

Cheers,

Your Chairs, Mark and Lars

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 10:14:54 +0100

hi Mark, all,

While I disagree with the chairs as to the state of consensus on this issue, I'm going to go ahead and (perhaps predictably) throw my name in the hat for this one, since we're actively working on it anyway.

I have a few questions and comments on the content of the work program requested, below:

> On 22 Nov 2017, at 09:06, Mark Nottingham <[email-masked]> wrote:

<snip>

> 1) A description of the use case(s) that motivate this proposal. We understand that the goal is to measure RTT, but some people are still unclear as to why that's necessary to operate a network. De…

This seems reasonable, though I would appreciate some clear acceptance criteria for the level of detail the WG would consider useful. "I need to debug the network" is clearly not detailed enough, but what is?

> 2) An actual protocol specification for the Spin Bit, including Privacy Considerations. There seems to be a fairly good common understanding of it (especially in the DT), but it would be helpful to…

We already have this in the form of PR 609, though it needs to be rewritten. I'd be happy to volunteer to carve this off into a separate I-D if that's what the WG wants.

The request for a privacy considerations section for a single protocol feature seems odd to me at best, and prejudicial at worst. Where is the privacy considerations analysis for other features, for instance, various header compression schemes? But, again, given that the work is already done (and mostly in markdown!), dumping it into an I-D is marginal work.

> 3) Data showing that the Spin Bit is fit for purpose under real-world conditions. Piet's experiment is a start here, but we need much more.

Piet's current work represents much more than was presented to the list during the Singapore meeting, but some indication of how much more we need would be useful here. Piet is producing a detailed analysis of a number of proposed measurability features (one-bit spin is done, and two-bit spin and a blocking bit as once proposed by Martin Duke are on the implementation schedule now), and we can certainly feed this into the draft.

> Analysis of how well that data meets the use cases is also necessary. This might form part of the Manageability Considerations of the draft.
> 

As in 1, there's a whole lot of literature here. I'll say that we'll probably produce some numbers as part of the same evaluation above comparing the latency spin bit to handshake RTT measurement (presuming that the handshake remains visible enough to measure) and to "find some TCP in your aggregate and compute latency based on that" (which reduces to "force 1/k QUIC flows to TCP in order to get enough TCP in your aggregate to compute latency" in a mostly-QUIC world, yay fallback!).

> 5) An exploration of what the effects of deploying the spin bit are likely to be. E.g.: Will even the perception of privacy issues change endpoint behaviour? Will networks treat traffic differently…

I don't see how this request is useful.

Perhaps if the proposal were to add some new form of exposure not already ubiquitously available in TCP, with which we did not have years of operational experience, this might arguably be necessary work. However, as it stands, the exercise would certainly lead to a fascinating exploration of the economics of modern Internet access, but given the lack of ability to run sensible experiments to form an evidentiary basis for decision, such an exercise would always reflect the prejudices of its authors. One could write an essay for 5 that will reasonably support any view one cares to have here, which yields it useless for helping us to make a decision.

> In terms of timeline - resolving this is NOT a blocking issue; i.e., we can discuss this concurrently with other work on the protocol, and because it's such a small change in the packet layout, we…

I will note that any bits we designate to measurement will rapidly become part of QUIC's invariants, whether we intend them to or not, so the details of any design we choose will have to be carefully considered. However, that probably does not imply that those bits (their number and position) need to be proactively added to the invariants currently under discussion. So I agree with your assessment that we can tip this into the short header relatively late in the game.

Cheers,

Brian

---

From: Stephen Farrell <[email-masked]>
Date: Wed, 22 Nov 2017 10:01:49 +0000

Just on this one aspect...

On 22/11/17 09:14, Brian Trammell (IETF) wrote:
> The request for a privacy considerations section for a single
> protocol feature seems odd to me at best, and prejudicial at worst.
> Where is the privacy considerations analysis for other features, for
> instance, various header compression schemes? 
I agree that asking that there be a privacy considerations section
in RFCs for every bit in every protocol would be OTT. But that was
not how I interpreted Mark's mail.

What I thought was being requested and what I do think is reasonable
is to document a privacy analysis for any quic protocol bits that are
visible to the path. Whether or not some or all of that text ends up
in some RFC is another day's work.

Cheers,
S.

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 11:10:45 +0100

hi Stephen,

> On 22 Nov 2017, at 11:01, Stephen Farrell <[email-masked]> wrote:
> 
> 
> visible to the path.

I think it makes more sense to document a privacy analysis of the QUIC protocol's wire image, holistically -- traffic analysis doesn't use one bit at a time, and the information available in one bit may be more readily available in others up and down the stack. So if the request is a contribution to an eventual privacy considerations section for the whole wire image, then great, I'm happy to contribute it for this one bit, as I've basically already written it. :)

In light of the rest of the message, though (especially point 5, which I really can't see how to turn into something useful either for the decision process or for future users of such a facility), I read this request more as a proof of work, which is not what we do here, at least not intentionally.

Cheers,

Brian

> Whether or not some or all of that text ends up

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 10:35:54 +0000

Hi,

> in some RFC is another day's work.

for the Spin Bit specifically, the intent was to permanently capture the analysis the DT has done, so that when others review the proposed Spin Bit specification, they can take that as a given and direct any further analysis to other aspects. It made sense to the chairs that that specific analysis should become part of the Spin Bit specification. I think we'd be open to a discussion on whether a broader document analyzing the QUIC wire image would be a better home for this. The main point is for the work that the DT has done to be documented.

For proposals other than the Spin Bit (I think I have seen individual contributors at least mention "loss" and "congestion" bits, but without much detail), we wanted to clarify that we'd like to see an analysis and discussion of their privacy aspects to roughly the same degree as the DT has performed for the Spin Bit proposal.

Lars

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 11:58:53 +0100

hi Lars,

> On 22 Nov 2017, at 11:35, Eggert, Lars <[email-masked]> wrote:
> 
> Hi,
> 

Okay. That's somewhat more reasonable than what I read the ask to be ("we're going to gate this on the people who care about this doing some non-trivial amount of work"). Those of us who volunteer (help, please, anyone? :) ) can certainly pull together what we have in a single I-D and ask the WG what more it thinks it needs. To me all this seems pretty clear, but I've been working on this topic for a while.

> For proposals other than the Spin Bit (I think I have seen individual contributors at least mention "loss" and "congestion" bits, but without much detail), we wanted to clarify that we'd like to se…

Ok. A couple of process questions remain open here, then:

(1) can the chairs articulate why, before the finalization of v1, they are proposing to go to a method of working that is somewhat more heavyweight than the issues-and-PRs-plus-ML that enhancements or change requests to the protocol to date have required?

(2) to what proposals, specifically, does this heavyweight process apply? the spin bit and measurement proposals articulated to date? proposals impacting the measurability of the protocol? proposals impacting specific unencrypted bits in headers? all proposals impacting the wire image of the protocol?

Thanks, cheers,

Brian

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 11:27:01 +0000

Hi,

Mark is asleep (I hope), so below is my personal view.

> (1) can the chairs articulate why, before the finalization of v1, they are proposing to go to a method of working that is somewhat more heavyweight than the issues-and-PRs-plus-ML that enhancements…

The discussion leading up to and around the Spin Bit has demonstrated that proposals that affect the cleartext part of the wire image are incredibly controversial, and take the WG a lot of time to chew through. The Spin Bit alone took six months and a DT, and it's arguably amongst the most simple proposals one could make in this space.

We're trying to find a way to have future discussions on such proposals in an efficient manner, in a way that minimizes further delays to our main deliverables (the base drafts).

Part of this is that such proposals should be contributed to the WG in a way that lets other contributors understand the rational, design, and implications of the proposal, i.e., a somewhat longer and more structured format than we'd usually see for GitHub issues and PRs.

So yes, we're asking the proponents of new schemes that affect the wire image to do a bit more work and produce a more comprehensive proposal, compared to what we get in GitHub. But the hope is that this will reduce the effort the WG needs to spend overall on understanding and discussing such proposals, i.e., we hope for a net win.

For the Spin Bit specifically, most of the things that Mark listed exist in one way or another, it'll be mostly a short exercise in assembling that material?

> (2) to what proposals, specifically, does this heavyweight process apply? the spin bit and measurement proposals articulated to date? proposals impacting the measurability of the protocol? proposal…

My personal view is that I'd like to see this for proposals that affect the cleartext wire format in ways that may affect user privacy. For example, renumbering the packet types is probably not a change we need to be overly concerned with. (I realize this is not a clear-cut definition.)

Lars

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 22 Nov 2017 22:44:08 +1100

> On 22 Nov 2017, at 10:27 pm, Eggert, Lars <[email-masked]> wrote:
> 
> Hi,
> 

Not asleep, but close. Personal views likewise.

> The discussion leading up to and around the Spin Bit has demonstrated that proposals that affect the cleartext part of the wire image are incredibly controversial, and take the WG a lot of time to…

+1. Brian, I would remind you that the DT did *not* gain consensus on doing the spin bit. This is (still) a contentious issue.

> We're trying to find a way to have future discussions on such proposals in an efficient manner, in a way that minimizes further delays to our main deliverables (the base drafts).

Exactly. Brian, please don't view this as a series of hoops to jump through; we are trying to find you (and other proponents) a path that is productive.

Since you asked earlier: I see item 5 as an extension of item 3. Since the spin bit is disconnected from the application payload, it might be used -- both by endpoints and networks -- for other purposes, thereby creating incentives that reduce its value and/or have unfortunate side effects. If this happens, it might not remain fit for purpose (and I've heard a few people expressing concern about this). An exploration of this area -- by whoever chooses to do so -- would help address those concerns. Not "required", just something we think would be helpful to have.

> My personal view is that I'd like to see this for proposals that affect the cleartext wire format in ways that may affect user privacy. For example, renumbering the packet types is probably not a c…

Agreed. If data is being added to the cleartext, we'll likely need to examine it in a similar way.

OK, *now* I'm going to sleep.

--
Mark Nottingham   https://www.mnot.net/

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 12:49:45 +0100

Hi Lars,

Comments inline.

> On 22 Nov 2017, at 12:27, Eggert, Lars <[email-masked]> wrote:
> 
> Hi,
> 
> The discussion leading up to and around the Spin Bit has demonstrated that proposals that affect the cleartext part of the wire image are incredibly controversial, and take the WG a lot of time to chew through.

(...again, it seems to me that the discussion converged in Singapore, so I'm a little perplexed as to why we're still having this conversation, especially given a explicit statement from the chairs that consensus may necessarily be rougher as we work to meet milestones in a timely manner, but if I'm the only one who believed that we were done, I'm willing to be declared in the rough here.)

> The Spin Bit alone took six months and a DT, and it's arguably amongst the most simple proposals one could make in this space.

Indeed, it was specifically designed to be the simplest possible proposal in this space. And if my read of this thread is correct, that the chairs believe there is no consensus to add the spin bit, and that a proposal to add the spin bit will be considered at IETF 101, it'll end up taking basically a year. PR 609 was submitted in Paris, but the discussion about explicit measurability predates it a bit.

> Part of this is that such proposals should be contributed to the WG in a way that lets other contributors understand the rational, design, and implications of the proposal, i.e., a somewhat longer…

Ok, this makes sense to me.

> So yes, we're asking the proponents of new schemes that affect the wire image to do a bit more work and produce a more comprehensive proposal, compared to what we get in GitHub. But the hope is tha…

This net win comes from shifting the effort to the proposers, which still has uncomfortable proof-of-work optics, but I see your point.

> For the Spin Bit specifically, most of the things that Mark listed exist in one way or another, it'll be mostly a short exercise in assembling that material?

Everything except point 5 we basically already have in some form or another, and point 5 does not really seem worth spending effort on if the point is to come to a technical decision about this proposal.

> My personal view is that I'd like to see this for proposals that affect the cleartext wire format in ways that may affect user privacy. For example, renumbering the packet types is probably not a c…

I think I agree that this is the right aim. But if "affect the cleartext wire format in ways that may affect user privacy" is the criterion, then I would submit that the one thing the DT came to conclusion on is that the spin bit has negligible impact on user privacy, and as such would not be subject thereto. So the criterion is probably something different... let me propose "changes the information content of the wire image".

Thanks, cheers,

Brian

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Nov 2017 12:03:47 +0000

<snip>
> 
> Since you asked earlier: I see item 5 as an extension of item 3. Since the spin
> bit is disconnected from the application payload, it might be used -- both by
> endpoints and networks -- for other purposes, thereby creating incentives
> that reduce its value and/or have unfortunate side effects. If this happens, it
> might not remain fit for purpose (and I've heard a few people expressing
> concern about this). An exploration of this area -- by whoever chooses to do
> so -- would help address those concerns. Not "required", just something we
> think would be helpful to have.
[Roni Even] This concern can be applied to other fields in the clear part of the header. For example the connection id in a peer to peer case. Even in client server an application with a co-operating server can use this long field to whatever they want and it will not break the protocol. We expect that people will  comply with protocol specification but we cannot prevent them from creating non-compliant usages.
> 
> 
> > Part of this is that such proposals should be contributed to the WG in a way
> that lets other contributors understand the rational, design, and implications
> of the proposal, i.e., a somewhat longer and more structured format than
> we'd usually see for GitHub issues and PRs.
> >
> > So yes, we're asking the proponents of new schemes that affect the wire
> image to do a bit more work and produce a more comprehensive proposal,
> compared to what we get in GitHub. But the hope is that this will reduce the
> effort the WG needs to spend overall on understanding and discussing such
> proposals, i.e., we hope for a net win.
> >
> > For the Spin Bit specifically, most of the things that Mark listed exist in one
> way or another, it'll be mostly a short exercise in assembling that material?
> >
> >> (2) to what proposals, specifically, does this heavyweight process apply?
> the spin bit and measurement proposals articulated to date? proposals
> impacting the measurability of the protocol? proposals impacting specific
> unencrypted bits in headers? all proposals impacting the wire image of the
> protocol?
> >
> > My personal view is that I'd like to see this for proposals that affect the
> cleartext wire format in ways that may affect user privacy. For example,
> renumbering the packet types is probably not a change we need to be overly
> concerned with. (I realize this is not a clear-cut definition.)
>

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 13:07:13 +0100

hi Mark,

> On 22 Nov 2017, at 12:44, Mark Nottingham <[email-masked]> wrote:
> 

The DT did not, but the WG discussed it after the DT report. I'll take your statement here as an indication that the rough consensus I thought I saw for the spin bit as harmless and useful in Singapore and on the list afterward is not an impression shared by the chairs and the rest of the WG.

> Exactly. Brian, please don't view this as a series of hoops to jump through; we are trying to find you (and other proponents) a path that is productive.

Okay. I'm trying not to, but it seems to me to be difficult to avoid the perception of a series of hoops to jump through when an enumeration of work items is laid out under the signature of a WG chair speaking as such, especially when much of the information is available, albeit in a less organized form.

(in re productive: at this point I've spent my time budget on this for the day on this thread, which is basically my fault, but I think I have a much clearer picture of the state of things, so thank you for that.)

> Since you asked earlier: I see item 5 as an extension of item 3. Since the spin bit is disconnected from the application payload, it might be used -- both by endpoints and networks -- for other pur…

Okay. I'm not saying it wouldn't be interesting, I'm saying it won't be a useful input to the WG's decision. I won't spend any time on it, except what naturally comes out of point 3, and I'll hope that's enough for the WG to decide.

> Agreed. If data is being added to the cleartext, we'll likely need to examine it in a similar way.

s/data/information content/ but yeah, the boundary here is looking pretty clear, and quite useful to ensure this process is not applied in an arbitrary way.

'Night. I'm off to enjoy the last nice day in Zürich.

Thanks, cheers,

Brian

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Nov 2017 12:15:12 +0000

Hi ,
 I was surprised to read the chairs email and agree with Brian's comments.
My impression is that the email reflects a view that manageability is not important (and not blocking) and this is the reason for not doing the spin bit based on issue 609 as a PR.
My product development experience is that lack of tools to manage and maintain a product will prevent its distribution. I understand that the developer  of the protocol care only about the protocol and not about making it easy to support in the field. Yet the network product manufacturers and the service providers gave a clear message that RTT  is crucial for deployment.  I provided use cases from TCP in service providers networks in https://tools.ietf.org/id/draft-even-quic-troubleshooting-video-delivery-00.txt 
I expect that the WG chair and AD will treat protocol maintenance and management the same as the protocol itself. What emphasis my point is the effort the WG does on https://tools.ietf.org/html/draft-ietf-quic-manageability-01 

Roni Even 

> -----Original Message-----
> From: QUIC [mailto:[email-masked]] On Behalf Of Brian Trammell
> (IETF)
> Sent: יום ד 22 נובמבר 2017 11:15
> To: Mark Nottingham
> Cc: QUIC WG; Lars Eggert
> Subject: Re: Spin bit discussion - where we're at
> 
> hi Mark, all,
> 
> While I disagree with the chairs as to the state of consensus on this issue, I'm
> going to go ahead and (perhaps predictably) throw my name in the hat for
> this one, since we're actively working on it anyway.
> 
> > 1) A description of the use case(s) that motivate this proposal. We
> understand that the goal is to measure RTT, but some people are still unclear
> as to why that's necessary to operate a network. Detailed scenarios and
> ideally real-world examples (e.g., from TCP) would help tremendously.
> Saying "I need to debug the network" is not enough detail.
> 
> This seems reasonable, though I would appreciate some clear acceptance
> criteria for the level of detail the WG would consider useful. "I need to debug
> the network" is clearly not detailed enough, but what is?
> 
> > 2) An actual protocol specification for the Spin Bit, including Privacy
> Considerations. There seems to be a fairly good common understanding of it
> (especially in the DT), but it would be helpful to have it written down, so we
> can make sure we're talking about the same thing. If it were proposed as an
> extension, current implementations (reminder: we have 12) could
> implement it to see how it works on the network. Doing so would also give
> the security research community something to look at (see above).
> 
> We already have this in the form of PR 609, though it needs to be rewritten.
> I'd be happy to volunteer to carve this off into a separate I-D if that's what
> the WG wants.
> 
> The request for a privacy considerations section for a single protocol feature
> seems odd to me at best, and prejudicial at worst. Where is the privacy
> considerations analysis for other features, for instance, various header
> compression schemes? But, again, given that the work is already done (and
> mostly in markdown!), dumping it into an I-D is marginal work.
> 
> > 3) Data showing that the Spin Bit is fit for purpose under real-world
> conditions. Piet's experiment is a start here, but we need much more.
> 
> Piet's current work represents much more than was presented to the list
> during the Singapore meeting, but some indication of how much more we
> need would be useful here. Piet is producing a detailed analysis of a number
> of proposed measurability features (one-bit spin is done, and two-bit spin
> and a blocking bit as once proposed by Martin Duke are on the
> implementation schedule now), and we can certainly feed this into the draft.
> 
> > Analysis of how well that data meets the use cases is also necessary. This
> might form part of the Manageability Considerations of the draft.
> >
> > 4) A comparison of other potential solutions to measuring delay, and their
> relative merits. Again, ideally with detailed, real-world data.
> 
> As in 1, there's a whole lot of literature here. I'll say that we'll probably
> produce some numbers as part of the same evaluation above comparing the
> latency spin bit to handshake RTT measurement (presuming that the
> handshake remains visible enough to measure) and to "find some TCP in
> your aggregate and compute latency based on that" (which reduces to "force
> 1/k QUIC flows to TCP in order to get enough TCP in your aggregate to
> compute latency" in a mostly-QUIC world, yay fallback!).
> 
> > 5) An exploration of what the effects of deploying the spin bit are likely to
> be. E.g.: Will even the perception of privacy issues change endpoint
> behaviour? Will networks treat traffic differently based upon the spin bit? If
> so, can that be gamed? Will firewalls and other gateways police spin bit
> behaviour (e.g., to prevent data loss), and what effects will that have?
> 
> Perhaps if the proposal were to add some new form of exposure not already
> ubiquitously available in TCP, with which we did not have years of operational
> experience, this might arguably be necessary work. However, as it stands,
> the exercise would certainly lead to a fascinating exploration of the
> economics of modern Internet access, but given the lack of ability to run
> sensible experiments to form an evidentiary basis for decision, such an
> exercise would always reflect the prejudices of its authors. One could write
> an essay for 5 that will reasonably support any view one cares to have here,
> which yields it useless for helping us to make a decision.
> 
> > We're looking for volunteers to start drafting one or more documents along
> these lines. If you're interested, please indicate this on-list, so you can
> coordinate with other interested parties. They won't (yet) be WG
> documents, but since the DT didn't resolve the issue, we will discuss them in
> the WG.
> >
> > In terms of timeline - resolving this is NOT a blocking issue; i.e., we can
> discuss this concurrently with other work on the protocol, and because it's
> such a small change in the packet layout, we can add it right before shipping
> the protocol to the IESG.
> 
> I will note that any bits we designate to measurement will rapidly become
> part of QUIC's invariants, whether we intend them to or not, so the details of
> any design we choose will have to be carefully considered. However, that
> probably does not imply that those bits (their number and position) need to
> be proactively added to the invariants currently under discussion. So I agree
> with your assessment that we can tip this into the short header relatively late
> in the game.
> 
> Cheers,
> 
> Brian
> 
> > In light of that, we will NOT discuss this (or related) issues at the Melbourne
> interim. That will allow the people attending to focus on continued protocol
> development, and means that people don't feel the need to travel to discuss
> just this issue. We would like to be able to move the discussion forward at
> IETF101 in London, provided that significant progress is made on the items
> above.
> >
> > Cheers,
> >
> >

---

From: Eliot Lear <[email-masked]>
Date: Wed, 22 Nov 2017 13:16:08 +0100

Hard to follow the bouncing ball here, but on these two points and
elsewhere:

> The DT did not, but the WG discussed it after the DT report. I'll take your statement here as an indication that the rough consensus I thought I saw for the spin bit as harmless and useful in Singa…

Mark did say that he was speaking as a contributor.

On the general point, general security considerations principles should
apply.  If we know how to abuse a field, we should state the risk.  So
long as the benefit is articulated, at that point the E in IETF kicks
in: make your design tradeoffs.  That's what engineers do.  Network
management is often the flipside of privacy, so...

> 'Night. I'm off to enjoy the last nice day in Zürich.

Is Armageddon coming?  I missed the memo.

Eliot

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 13:21:09 +0100

> On 22 Nov 2017, at 13:16, Eliot Lear <[email-masked]> wrote:
> 

Ah, good point. Apologies, strike that, I won't take that as a declaration of (non)consensus then.

> Is Armageddon coming?  I missed the memo.

It's twelve degrees and the sun is shining. in November! GO OUTSIDE! he yells to himself...

cheers, B

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 12:35:23 +0000

Hi Roni,

> My impression is that the email reflects a view that manageability is not important (and not blocking) and this is the reason for not doing the spin bit based on issue 609 as a PR.

there hasn't been a consensus call on the Spin Bit proposal. Contributors have expressed their views on the results presented by the DT on the list, as requested by the chairs in the session in Singapore. But that is not the same as a consensus call, and I apologize if it was misunderstood as one.

In order to hold an informed consensus call on the Spin Bit in the future, we're asking for a more comprehensive writeup (as detailed in the original email.) As I've said before, I think the material for that already exists in some form or other, but it's currently in several places, not all of which may be public (e.g., the DT mailing list.)

For *other* proposals that intend to address manageability by adding information to the cleartext wire image, we're asking for a similarly comprehensive description, so that the WG can begin discussing a concrete contribution.

Lars

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Nov 2017 12:36:11 +0000

<snip>
> 
> > The Spin Bit alone took six months and a DT, and it's arguably amongst the
> most simple proposals one could make in this space.
> 
> Indeed, it was specifically designed to be the simplest possible proposal in
> this space. And if my read of this thread is correct, that the chairs believe
> there is no consensus to add the spin bit, and that a proposal to add the spin
> bit will be considered at IETF 101, it'll end up taking basically a year. PR 609
> was submitted in Paris, but the discussion about explicit measurability
> predates it a bit.
> 
[Roni Even] The process of having a DT and blocking discussion during the time for the 4 month between the meeting slowed the progress. This WG trues to work in a fast pace and I think that in Singapore and on the mailing list we saw strong support from the parties who will use the spin bit to move it fast. Waiting for London will further delay.
I was planning to provide replacement text for 609 this week or no later than next week, defining the spin bit based on the discussions we had in Singapore and on the list for a new PR for the QUIC transport document.
We can have in parallel a non WG document on RTT measurements using the spin bit even though it place may be in the QUIC manageability document.
Roni Even

---

From: Salvatore Loreto <[email-masked]>
Date: Wed, 22 Nov 2017 13:31:43 +0000

Hi Lars, Mark

One clarification question about the process specifically for the Spin Bit proposal and the time frame for the consensus call

While it is clear to me that a more comprehensive writeup (as detailed in the original email.) is a necessary
condition, it is not clear if it is also a sufficient one for a consensus call.

Indeed in the original mail Mark is on one side proposing to discuss it in London (instead that into the next interim)
but at same time is leaving open the possibility to postpone the final decision until right before shipping the protocol to the IESG
While I agree with Mark assessment that we can tip this into the short header relatively late in the game, 
I am concerned for this huge potential delay for a proposal that I would argue is already clear to all people actively involved in the QUIC design.

Can you please which are the next steps you are going to follow here?

Sal 

-----Original Message-----
From: QUIC [mailto:[email-masked]] On Behalf Of Eggert, Lars
Sent: den 22 november 2017 13:35
To: Roni Even <[email-masked]>
Cc: Brian Trammell <[email-masked]>; Mark Nottingham <[email-masked]>; QUIC WG <[email-masked]>
Subject: Re: Spin bit discussion - where we're at

Hi Roni,

> My product development experience is that lack of tools to manage and 
> maintain a product will prevent its distribution. I understand that 
> the developer  of the protocol care only about the protocol and not 
> about making it easy to support in the field. Yet the network product 
> manufacturers and the service providers gave a clear message that RTT  
> is crucial for deployment.  I provided use cases from TCP in service 
> providers networks in 
> https://tools.ietf.org/id/draft-even-quic-troubleshooting-video-delive
> ry-00.txt I expect that the WG chair and AD will treat protocol 
> maintenance and management the same as the protocol itself. What 
> emphasis my point is the effort the WG does on 
> https://tools.ietf.org/html/draft-ietf-quic-manageability-01

Lars

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 13:55:46 +0000

Hi,

> but at same time is leaving open the possibility to postpone the final decision until right before shipping the protocol to the IESG

I think the decision to not discuss it at the interim is based on the observation that the registered attendees at the interim are probably not representative of the overall WG constituency - we want to give all sides an ability to participate equally in the discussion.

> Can you please which are the next steps you are going to follow here?

I think what Mark tried to express is that the discussion on the Spin Bit can be mostly decoupled from the (large amount of) work remaining on the base spec. That is, we don't have to conclude the Spin Bit discussion significantly prior to the conclusion of the overall work (although we can), since it should be easy to merge at the later stages before handing things to the IESG.

Lars

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Nov 2017 14:18:25 +0000

Hi Lars,

> From: Eggert, Lars [mailto:[email-masked]]
> Sent: יום ד 22 נובמבר 2017 14:35
> To: Roni Even
> Cc: Brian Trammell; Mark Nottingham; QUIC WG
> > My impression is that the email reflects a view that manageability is not
> important (and not blocking) and this is the reason for not doing the spin bit
> based on issue 609 as a PR.
> 
> there hasn't been a consensus call on the Spin Bit proposal. Contributors
> have expressed their views on the results presented by the DT on the list, as
> requested by the chairs in the session in Singapore. But that is not the same
> as a consensus call, and I apologize if it was misunderstood as one.
> 
> In order to hold an informed consensus call on the Spin Bit in the future,
> we're asking for a more comprehensive writeup (as detailed in the original
> email.) As I've said before, I think the material for that already exists in some
> form or other, but it's currently in several places, not all of which may be
> public (e.g., the DT mailing list.)
[Roni Even]  I agree that there is was no consensus call in Singapore, there was no text to agree on. The issue is if we want a spin bit solution and the discussion on the list supports it.  From the discussion on the Microphone I noticed that some of the group members did not agree with the content of the presentation. So to me it look like that the only good thing that came out from the DT is that the spin bit does not add any security risks with regards to the security issues that were presented to them. 
I do not understand why spin bit issues in the github are "parked" 
>From my point of view I intend to try to expedite the needed text for the spin bit and hope that the expectation will be reasonable comparing to the requirements from other features in the protocol.   This is why I ask the chairs not to wait for IETF 101 if the interested parties will provide text. Even though you can add this text at any stage towards V1 I would not like to get it blocked because it did not get enough discussion since it came late and people need more time. Based on the interest I expect to discuss it also in the Interim if we have enough text.  I was not aware that the decision on what to discuss is based on the onsite participants
> 
> For *other* proposals that intend to address manageability by adding
> information to the cleartext wire image, we're asking for a similarly
> comprehensive description, so that the WG can begin discussing a concrete
> contribution.
>

---

From: Roni Even <[email-masked]>
Date: Wed, 22 Nov 2017 14:22:27 +0000

> Sent: יום ד 22 נובמבר 2017 15:56
> To: Salvatore Loreto
> Cc: Roni Even; Brian Trammell; Mark Nottingham; QUIC WG
> > Indeed in the original mail Mark is on one side proposing to discuss
> > it in London (instead that into the next interim) but at same time is
> > leaving open the possibility to postpone the final decision until
> > right before shipping the protocol to the IESG
> 
> I think the decision to not discuss it at the interim is based on the observation
> that the registered attendees at the interim are probably not representative
> of the overall WG constituency - we want to give all sides an ability to
> participate equally in the discussion.
[Roni Even] This is why we have remote participants at IETF meetings !!!!!!!!
You do not have to be in person to discuss!!!!
> 
> > While I agree with Mark assessment that we can tip this into the short
> > header relatively late in the game, I am concerned for this huge potential
> delay for a proposal that I would argue is already clear to all people actively
> involved in the QUIC design.
> >
> I think what Mark tried to express is that the discussion on the Spin Bit can be
> mostly decoupled from the (large amount of) work remaining on the base
> spec. That is, we don't have to conclude the Spin Bit discussion significantly
> prior to the conclusion of the overall work (although we can), since it should
> be easy to merge at the later stages before handing things to the IESG.
[Roni Even] delaying the discussion may prevent the acceptance.  If people will put effort at submitting text then the chairs should allow for discussion and not delay the work.

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 14:42:54 +0000

Hi,

On 2017-11-22, at 15:18, Roni Even <[email-masked]> wrote:
> [Roni Even]  I agree that there is was no consensus call in Singapore, there was no text to agree on.

exactly. The email Mark sent earlier outlines what the chairs think would be useful for the WG to see, in terms of text.

> So to me it look like that the only good thing that came out from the DT is that the spin bit does not add any security risks with regards to the security issues that were presented to them.

That seems to have been the main result the DT came to consensus on, and it is a useful bit of information.

> I do not understand why spin bit issues in the github are "parked"

I think the editors parked it, because we had the DT chartered.

Lars

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 22 Nov 2017 14:51:09 +0000

Hi,

> You do not have to be in person to discuss!!!!

the IETF (vie MeetEcho) provides remote participation at the main meetings. At the interims, it's Mark. It seems to have mostly worked OK for passive participation during two out of the last three interims, but we haven't tried to hold a discussion with many active remote participants.

> [Roni Even] delaying the discussion may prevent the acceptance.  If people will put effort at submitting text then the chairs should allow for discussion and not delay the work.

I disagree with this characterization.

What I said above in the paragraph you quote is that we do not *have to* conclude the Spin Bit discussion urgently *although we can*. Specifically, we can begin (and conclude) the discussion once there is an actual proposal that addresses what Mark's original email outlined.

Lars

---

From: Marcus Ihlar <[email-masked]>
Date: Wed, 22 Nov 2017 15:57:44 +0000

Hi,
Just chipping in to say that I’m happy to work with Brian on this draft.

/Marcus

> On 22 Nov 2017, at 22:51, Eggert, Lars <[email-masked]> wrote:
> 
> Hi,
>

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 22 Nov 2017 17:25:06 +0100

> On 22 Nov 2017, at 14:55, Eggert, Lars <[email-masked]> wrote:
> 
> Hi,
> 

Quoting this for truth, and agreement. One of the nice properties of this particular proposal is that it can be implemented (almost) completely independently of the rest of the transport protocol, and as such can be developed independently as well.

However, this is probably not the case for the other proposals I'm aware of: a simple loss/ECN re-feedback mechanism as in conex impacts ack handling, for instance, and some of the debug bits signaling whether a flow is flow control or congestion control limited (see 279 / 418) also require slightly deeper integration with the transport mechanics. We already pretty much knew that discussion on these was a topic for v2, though, so the additional process doesn't actually change anything there.

Cheers,

Brian

---

From: Roni Even <[email-masked]>
Date: Thu, 23 Nov 2017 06:08:41 +0000

Hi,
I will work with Brian and Marcus on this draft
Roni

From: QUIC [mailto:[email-masked]] On Behalf Of Marcus Ihlar
Sent: יום ד 22 נובמבר 2017 17:58
To: Eggert, Lars
Cc: Salvatore Loreto; Brian Trammell; Roni Even; QUIC WG; Mark Nottingham
Subject: Re: Spin bit discussion - where we're at

Hi,

/Marcus

On 22 Nov 2017, at 22:51, Eggert, Lars <[email-masked]<mailto:[email-masked]>> wrote:
Hi,

On 2017-11-22, at 15:22, Roni Even <[email-masked]<mailto:[email-masked]>> wrote:

Lars

---

From: <[email-masked]>
Date: Thu, 23 Nov 2017 09:45:36 +0000

Hi

I propose to contribute as a telco representative. We already described troubleshooting use cases in https://tools.ietf.org/html/draft-stephan-quic-interdomain-troubleshooting-00.

Regards
Emile

De : QUIC [mailto:[email-masked]] De la part de Roni Even
Envoyé : jeudi 23 novembre 2017 07:09
À : Marcus Ihlar; Eggert, Lars
Cc : Salvatore Loreto; Brian Trammell; Mark Nottingham; QUIC WG
Objet : RE: Spin bit discussion - where we're at

Hi,

Hi,

/Marcus

Lars

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Fioccola Giuseppe <[email-masked]>
Date: Thu, 23 Nov 2017 11:10:56 +0000

Hi,
I can also help on this topic, seen my experience with IPPM marking methodologies and as a telco delegate.
Thanks,

Giuseppe

Da: QUIC [mailto:[email-masked]] Per conto di Roni Even
Inviato: giovedì 23 novembre 2017 07:09
A: Marcus Ihlar; Eggert, Lars
Cc: Salvatore Loreto; Brian Trammell; Mark Nottingham; QUIC WG
Oggetto: RE: Spin bit discussion - where we're at

Hi,

Hi,

/Marcus

Lars

Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate. La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate. Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione, Grazie. 

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only. Dissemination, copying, printing or use by anybody else is unauthorised. If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail, Thanks. 

Rispetta l'ambiente. Non stampare questa mail se non è necessario.

---

From: Martin J. Dürst <[email-masked]>
Date: Fri, 24 Nov 2017 16:55:40 +0900

> It's twelve degrees and the sun is shining. in November! GO OUTSIDE! he yells to himself...

As a native and long-time resident of Zürich, and for the benefits of 
those not familiar with its weather, I'd like to add that for Zürich, 
November means fog, lots of it. After that, for the following few 
months, usually until somewhere in March, the fog moves higher so that 
it's observable as a permanent cloud cover. So if there's sun in 
November, I agree with Brian that going outside is the only reasonable 
choice.

Regards,   Martin.

P.S.: There are two main ways to escape the above predicament: a) Go 
above the clouds, e.g. skiing on weekends (that's one of the reasons 
Switzerland has mountains :-). b) Move somewhere else, (such as Tokyo, 
where I'm now) where winters are mostly sunny.

---

From: "MORTON, ALFRED C (AL)" <[email-masked]>
Date: Sun, 26 Nov 2017 19:26:54 +0000

Hi Brian, Stephen, Lars, Mark and all,

one join, one suggestion, and one question below.
see [ACM]
> Sent: Wednesday, November 22, 2017 5:59 AM
> To: Eggert, Lars
> Cc: Mark Nottingham; QUIC WG; Stephen Farrell
> > Lars wrote:
> > for the Spin Bit specifically, the intent was to permanently capture
> the analysis the DT has done, so that when others review the proposed
> Spin Bit specification, they can take that as a given and direct any
> further analysis to other aspects. It made sense to the chairs that that
> specific analysis should become part of the Spin Bit specification. I
> think we'd be open to a discussion on whether a broader document
> analyzing the QUIC wire image would be a better home for this. The main
> point is for the work that the DT has done to be documented.

> Brian wrote:
> Okay. That's somewhat more reasonable than what I read the ask to be
> ("we're going to gate this on the people who care about this doing some
> non-trivial amount of work"). Those of us who volunteer (help, please,
> anyone? :) ) can certainly pull together what we have in a single I-D
> and ask the WG what more it thinks it needs. To me all this seems pretty
> clear, but I've been working on this topic for a while.
[ACM] 
Having reached the end of the "Thanksgiving thread",
I'm scrolling back a few pages to join the task of 
permanently capturing the work of the DT in an I-D (at least):
There should be lasting value in some of our findings
and IMO they are worthy of a persistent reference.

> Lars wrote:
> > For proposals other than the Spin Bit (I think I have seen individual
> contributors at least mention "loss" and "congestion" bits, but without
> much detail), we wanted to clarify that we'd like to see an analysis and
> discussion of their privacy aspects to roughly the same degree as the DT
> has performed for the Spin Bit proposal.
> 
[ACM] 
I suggest that this might be a different I-D, at least to start.
Question: Is there a privacy analysis of present ECN available?
(a search yielded many results with Missing: privacy)

regards,
Al

---

From: Jana Iyengar <[email-masked]>
Date: Sun, 26 Nov 2017 23:03:42 -0800

In addition to how the spin bit can be designed and used, I'd like those
taking on this effort to consider this question: Whatever network
management problem you're considering solving with a bit, what does it take
to solve this problem without the bit exposed? I'm asking you to consider
"zero-bit" solutions, or at least for a cost/benefit analysis of developing
solutions with and without the spin bit for specific network management
functions. Specifically,

(i) What's impossible or very difficult to do, from a network management
perspective, of not having the spin bit? Note that I'm not asking for what
is and is not measurable -- not everything measurable is useful. I'm
specifically asking in terms of network management functions, in practice,
as used by operators. Some of this has been addressed in the
discussions/drafts I've seen, but I think it's most useful when framed in
terms of network management functions.

(ii) What are the alternatives for building the same functions without the
spin bit? Let's consider for a moment that the spin bit isn't actually
available in practice. What will operators do to continue managing their
networks? This might be expensive, but that's precisely what I'm looking
for -- the cost to operators of not having the spin bit. For instance,
active probes are a fine way to measure network RTT within operator
networks, and that is an alternative. There are surely others too. Their
costs and limitations are important to know about in order to reason about
their viability, so I'd like to see alternatives considered.

I don't mean to start the discussion on this thread, but I'd like to urge
those going off to do the writing to consider these questions.

- jana

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Mon, 27 Nov 2017 16:27:37 +0900

hi Jana,

Thanks for these questions. A couple of points inline.

> On 27 Nov 2017, at 16:03, Jana Iyengar <[email-masked]> wrote:
> 
> In addition to how the spin bit can be designed and used, I'd like those taking on this effort to consider this question: Whatever network management problem you're considering solving with a bit, what does it take to solve this problem without the bit exposed? I'm asking you to consider "zero-bit" solutions, or at least for a cost/benefit analysis of developing solutions with and without the spin bit for specific network management functions. Specifically,

> (i) What's impossible or very difficult to do, from a network management perspective, of not having the spin bit? Note that I'm not asking for what is and is not measurable -- not everything measurable is useful.

...while I agree with you here, in principle, I'll point out that we're not talking about some obscure metric you need two pages of latex mathmode to describe... latency measurement is pretty basic stuff, from which you can derive most everything else you need, which is why this is the proposal we'd go with if we only had one bit...

> I'm specifically asking in terms of network management functions, in practice, as used by operators. Some of this has been addressed in the discussions/drafts I've seen, but I think it's most useful when framed in terms of network management functions.

For the most part this is what you think it is: "detecting, localizing, and troubleshooting latency issues". For access providers, specifically, this is a matter of having an answer as to what the provider is doing wrong and/or why it's not the provider's fault when customers call to complain.

> (ii) What are the alternatives for building the same functions without the spin bit? Let's consider for a moment that the spin bit isn't actually available in practice. What will operators do to continue managing their networks?
> This might be expensive, but that's precisely what I'm looking for -- the cost to operators of not having the spin bit. For instance, active probes are a fine way to measure network RTT within operator networks, and that is an alternative.

...for some values of "fine", yes.

> There are surely others too.

Handshake RTT is the other one that'll almost certainly work, at the cost of drastically reduced sample rate and bias toward differential treatment of first packets / additional compute delay for crypto bootstrapping on endpoints small enough that that matters.

Next up is a two-stage approach that leverages peculiarities in implementations / CC algorithms to classify senders in order to probabilistically match packets to get component RTTs as with TCP. But that is, as you say, kind of expensive.

Third, if a provider needs passive RTT measurement at higher than handshake RTT resolution, and QUIC won't expose it, TCP still will. So one could always selectively/probabilistically block udp/443 to force fallback and get enough samples. Since the TCP and QUIC stacks are (for now) completely separate codepaths, there will be a bias on the endpoint delay side. It's also kind of expensive.

The latter two are speculative enough that I'd prefer to leave them out of the draft, though.

> Their costs and limitations are important to know about in order to reason about their viability, so I'd like to see alternatives considered.
> 
> I don't mean to start the discussion on this thread, but I'd like to urge those going off to do the writing to consider these questions.

Fortunately, I just wrote a first draft of that section :). Al signed up to add some text to it shortly. Hope to get a complete draft out in the next couple of weeks, travel and other deadlines permitting.

Cheers,

Brian

> - jana
> 
> 
> On Sun, Nov 26, 2017 at 11:26 AM, MORTON, ALFRED C (AL) <[email-masked]> wrote:

---

From: "Eggert, Lars" <[email-masked]>
Date: Mon, 27 Nov 2017 08:11:59 +0000

Hi,

> (a search yielded many results with Missing: privacy)

not that I'm aware of; CC'ing tsv-area@ for some broader input.

(On first glance, the - user - privacy aspects here seem to be much more contained, since ECN is mostly about the network exposing information to the end systems, and not vice versa.)

Lars

---

From: "Gorry (erg)" <[email-masked]>
Date: Mon, 27 Nov 2017 09:08:58 +0000

Hi Jana 

So... +1 to Brian’s reply.

We need to decide as a WG whether manageability is needed: Do we really need more operations and measurement people to speak up? 

“Handshake RTT” - turns out to be a rather poor tool in many radio-based networks (eg mobile)... quite simply the need us to measure latency (reordering,loss) as the session progresses ... I think this is already clear. 

Gorry 

> On 27 Nov 2017, at 07:27, Brian Trammell (IETF) <[email-masked]> wrote:
> 
> hi Jana,
>

---

From: Roland Zink <[email-masked]>
Date: Mon, 27 Nov 2017 12:32:40 +0100

Am 27.11.2017 um 08:03 schrieb Jana Iyengar:
> In addition to how the spin bit can be designed and used, I'd like 
> those taking on this effort to consider this question: Whatever 
> network management problem you're considering solving with a bit, what 
> does it take to solve this problem without the bit exposed? I'm asking 
> you to consider "zero-bit" solutions, or at least for a cost/benefit 
> analysis of developing solutions with and without the spin bit for 
> specific network management functions. Specifically,
>
> (i) What's impossible or very difficult to do, from a network 
> management perspective, of not having the spin bit? Note that I'm not 
> asking for what is and is not measurable -- not everything measurable 
> is useful. I'm specifically asking in terms of network management 
> functions, in practice, as used by operators. Some of this has been 
> addressed in the discussions/drafts I've seen, but I think it's most 
> useful when framed in terms of network management functions.
I want to express the question about manageability from a different 
angle. QUIC moves the congestion control from being a common 
functionality in the kernel to application functionality. It also hides 
the protocol handling of congestion from the network. This gives 
incentives for application developers to change congestion control to 
give their application an advantage over others. No special rights are 
necessary. Now the question is does QUIC provide enough manageability to 
avoid an Internet (or some part of it) breakdown when it is misused? Can 
it be shown that this can't happen?

> (ii) What are the alternatives for building the same functions without 
> the spin bit? Let's consider for a moment that the spin bit isn't 
> actually available in practice. What will operators do to continue 
> managing their networks? This might be expensive, but that's precisely 
> what I'm looking for -- the cost to operators of not having the spin 
> bit. For instance, active probes are a fine way to measure network RTT 
> within operator networks, and that is an alternative. There are surely 
> others too. Their costs and limitations are important to know about in 
> order to reason about their viability, so I'd like to see alternatives 
> considered.
Solutions that only work within an operators network are not enough when 
more than one operator is involved.

>
> I don't mean to start the discussion on this thread, but I'd like to 
> urge those going off to do the writing to consider these questions.
>
> - jana
>
>
> On Sun, Nov 26, 2017 at 11:26 AM, MORTON, ALFRED C (AL) 
> <[email-masked] <mailto:[email-masked]>> wrote:
>
>     > From: QUIC [mailto:[email-masked]
>     <mailto:[email-masked]>] On Behalf Of Brian Trammell
>     > (IETF)
>     > > On 22 Nov 2017, at 11:35, Eggert, Lars <[email-masked]
>     <mailto:[email-masked]>> wrote:
>     > >
>     > > Hi,
>     > >
>     > > On 2017-11-22, at 11:01, Stephen Farrell
>     <[email-masked] <mailto:[email-masked]>>
>     > wrote:

---

From: "Gorry (erg)" <[email-masked]>
Date: Mon, 27 Nov 2017 14:14:33 +0100

See below:

> On 27 Nov 2017, at 12:32, Roland Zink <[email-masked]> wrote:
> 
>> (i) What's impossible or very difficult to do, from a network management perspective, of not having the spin bit? Note that I'm not asking for what is and is not measurable -- not everything measurable is useful. I'm specifically asking in terms of network management functions, in practice, as used by operators. Some of this has been addressed in the discussions/drafts I've seen, but I think it's most useful when framed in terms of network management functions.
> I want to express the question about manageability from a different angle. QUIC moves the congestion control from being a common functionality in the kernel to application functionality. It also hides the protocol handling of congestion from the network. This gives incentives for application developers to change congestion control to give their application an advantage over others. No special rights are necessary. Now the question is does QUIC provide enough manageability to avoid an Internet (or some part of it) breakdown when it is misused? Can it be shown that this can't happen?
> 
I think this may touch on some of the topics in:
https://tools.ietf.org/html/draft-fairhurst-tsvwg-transport-encrypt-04

Although this is focussed on issues wider than Quic, I think the points you raise are relevant. This was presented last IETF-100 in TSVWG and INTAREA. We would appreciate insight, and comments on this.  (I am just about to push new version to correct typos!)

>> (ii) What are the alternatives for building the same functions without the spin bit? Let's consider for a moment that the spin bit isn't actually available in practice. What will operators do to continue managing their networks? This might be expensive, but that's precisely what I'm looking for -- the cost to operators of not having the spin bit. For instance, active probes are a fine way to measure network RTT within operator networks, and that is an alternative. There are surely others too. Their costs and limitations are important to know about in order to reason about their viability, so I'd like to see alternatives considered.
> Solutions that only work within an operators network are not enough when more than one operator is involved.
> 
Yes, also true.

Gorry

---

From: Ingemar Johansson S <[email-masked]>
Date: Mon, 27 Nov 2017 14:14:53 +0000

Hi

I would say that it is perfectly possible to trick e.g. the Linux TCP stack to something like what is described below. Just clone tcp_cong.c ,
Change the following lines
u32 tcp_reno_ssthresh<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_reno_ssthresh>(struct sock<https://elixir.free-electrons.com/linux/v3.12/ident/sock> *sk)
{
        const struct tcp_sock<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sock> *tp = tcp_sk<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sk>(sk);
        return max<https://elixir.free-electrons.com/linux/v3.12/ident/max>(tp->snd_cwnd >> 1U, 2U);
}
To something like

u32 tcp_bloat_ssthresh<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_reno_ssthresh>(struct sock<https://elixir.free-electrons.com/linux/v3.12/ident/sock> *sk)

{

        const struct tcp_sock<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sock> *tp = tcp_sk<https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sk>(sk);

        return tp->snd_cwnd;

}
Build it and make it a loadable kernel module, you need to rename a few other functions as well, and you can call it tcp_bloat.c 😊

So.. my opinion is that you can perfectly well collapse the internet also with TCP. ConEx was introduced as a medicine for this but it never gained traction.

/Ingemar

From: Gorry (erg) [mailto:[email-masked]]
Sent: den 27 november 2017 14:15
To: Roland Zink <[email-masked]>
Cc: [email-masked]
Subject: Re: Spin bit discussion - where we're at

See below:

On 27 Nov 2017, at 12:32, Roland Zink <[email-masked]<mailto:[email-masked]>> wrote:
Am 27.11.2017 um 08:03 schrieb Jana Iyengar:
In addition to how the spin bit can be designed and used, I'd like those taking on this effort to consider this question: Whatever network management problem you're considering solving with a bit, what does it take to solve this problem without the bit exposed? I'm asking you to consider "zero-bit" solutions, or at least for a cost/benefit analysis of developing solutions with and without the spin bit for specific network management functions. Specifically,

Gorry

- jana

On Sun, Nov 26, 2017 at 11:26 AM, MORTON, ALFRED C (AL) <[email-masked]<mailto:[email-masked]>> wrote:
Hi Brian, Stephen, Lars, Mark and all,

> From: QUIC [mailto:[email-masked]<mailto:[email-masked]>] On Behalf Of Brian Trammell
> (IETF)
> > On 22 Nov 2017, at 11:35, Eggert, Lars <[email-masked]<mailto:[email-masked]>> wrote:
> >
> > Hi,
> >
> > On 2017-11-22, at 11:01, Stephen Farrell <[email-masked]<mailto:[email-masked]>>
> wrote:

[ACM]

[ACM]

regards,
Al

---

From: Roland Zink <[email-masked]>
Date: Mon, 27 Nov 2017 15:47:57 +0100

I guess the guys which really want to do it will somehow get the 
necessary rights (like to load a kernel module) and do it although this 
may be difficult if you want to do it on million of devices. I'm more 
thinking of a slow process, say app 1 increase the initial window then 
app 2 thinks it needs to do the same and back out less than the default. 
Then app 1 is doing something and at some point nobody follows what is 
(will be) in the RFC. They not necessarily want to break something but 
at the end it can result into the same. It is no longer a single policy 
which applies to all apps instead each app comes with its own and the 
apps often can control both sides.

Roland

Am 27.11.2017 um 15:14 schrieb Ingemar Johansson S:
>
> Hi
>
> I would say that it is perfectly possible to trick e.g. the Linux TCP 
> stack to something like what is described below. Just clone tcp_cong.c ,
>
> u32 *tcp_reno_ssthresh* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_reno_ssthresh>(struct*sock* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/sock> *sk)
>
> {
>
> conststruct*tcp_sock* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sock> *tp 
> =*tcp_sk* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sk>(sk);
>
> return*max* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/max>(tp->snd_cwnd 
> >>1U,2U);
>
> }
>
> To something like
>
> u32*tcp_**bloat**_ssthresh* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_reno_ssthresh>(struct*sock* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/sock> *sk)
> {
> conststruct*tcp_sock* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sock> 
> *tp=*tcp_sk* 
> <https://elixir.free-electrons.com/linux/v3.12/ident/tcp_sk>(sk);
> returntp->snd_cwnd;
> }
>
> Build it and make it a loadable kernel module, you need to rename a 
> few other functions as well, and you can call it tcp_bloat.c 😊
>
> So.. my opinion is that you can perfectly well collapse the internet 
> also with TCP. ConEx was introduced as a medicine for this but it 
> never gained traction.
>
> /Ingemar
>
> *From:* Gorry (erg) [mailto:[email-masked]]
> *Sent:* den 27 november 2017 14:15
> *To:* Roland Zink <[email-masked]>
> *Cc:* [email-masked]
> *Subject:* Re: Spin bit discussion - where we're at
>
> See below:
>
>
>         In addition to how the spin bit can be designed and used, I'd
>         like those taking on this effort to consider this question:
>         Whatever network management problem you're considering solving
>         with a bit, what does it take to solve this problem without
>         the bit exposed? I'm asking you to consider "zero-bit"
>         solutions, or at least for a cost/benefit analysis of
>         developing solutions with and without the spin bit for
>         management perspective, of not having the spin bit? Note that
>         I'm not asking for what is and is not measurable -- not
>         everything measurable is useful. I'm specifically asking in
>         terms of network management functions, in practice, as used by
>         operators. Some of this has been addressed in the
>         discussions/drafts I've seen, but I think it's most useful
>         when framed in terms of network management functions.
>
>     I want to express the question about manageability from a
>     different angle. QUIC moves the congestion control from being a
>     common functionality in the kernel to application functionality.
>     It also hides the protocol handling of congestion from the
>     network. This gives incentives for application developers to
>     change congestion control to give their application an advantage
>     over others. No special rights are necessary. Now the question is
>     does QUIC provide enough manageability to avoid an Internet (or
>     some part of it) breakdown when it is misused? Can it be shown
>     that this can't happen?
>
> Although this is focussed on issues wider than Quic, I think the 
> points you raise are relevant. This was presented last IETF-100 in 
> TSVWG and INTAREA. We would appreciate insight, and comments on this. 
>  (I am just about to push new version to correct typos!)
>
>
>
>         (ii) What are the alternatives for building the same functions
>         without the spin bit? Let's consider for a moment that the
>         spin bit isn't actually available in practice. What will
>         operators do to continue managing their networks? This might
>         be expensive, but that's precisely what I'm looking for -- the
>         cost to operators of not having the spin bit. For instance,
>         active probes are a fine way to measure network RTT within
>         operator networks, and that is an alternative. There are
>         surely others too. Their costs and limitations are important
>         to know about in order to reason about their viability, so I'd
>         like to see alternatives considered.
>
>     Solutions that only work within an operators network are not
>     enough when more than one operator is involved.
>
> Yes, also true.
>
> Gorry
>
>         I don't mean to start the discussion on this thread, but I'd
>         like to urge those going off to do the writing to consider
>         these questions.
>
>         - jana
>
>             > > On 22 Nov 2017, at 11:35, Eggert, Lars
>             <[email-masked] <mailto:[email-masked]>> wrote:
>             > >
>             > > Hi,
>             > >

---

From: Christian Huitema <[email-masked]>
Date: Mon, 27 Nov 2017 08:38:00 -0800

The network cannot control the congestion algorithm in the end points. QUIC does not create the problem, just makes it crystal clear. But the network can manage queues. So does the kernel.

-- Christian Huitema 

> On Nov 27, 2017, at 6:47 AM, Roland Zink <[email-masked]> wrote:
> 
> I guess the guys which really want to do it will somehow get the necessary rights (like to load a kernel module) and do it although this may be difficult if you want to do it on million of devices. I'm more thinking of a slow process, say app 1 increase the       initial window then app 2 thinks it needs to do the same and back out less than the default. Then app 1 is doing something and at some point nobody follows what is (will be) in the RFC. They not necessarily want to break something but at the end it can result into the same. It is no longer a single policy which applies to all apps instead each app comes with its own and the apps often can control both sides.
> 
> Roland
> 
> 
>> u32 tcp_reno_ssthresh(struct sock *sk)
>> {
>>         const struct tcp_sock *tp = tcp_sk(sk);
>>         return max(tp->snd_cwnd >> 1U, 2U);
>> }
>> To something like
>> u32 tcp_bloat_ssthresh(struct sock *sk)
>> {
>>         const struct tcp_sock *tp = tcp_sk(sk);
>>         return tp->snd_cwnd;
>> }

---

From: Roland Zink <[email-masked]>
Date: Mon, 27 Nov 2017 18:36:08 +0100

If QUIC makes the problem crystal clear then now seems to be a good time 
to solve or mitigate it before risking it becomes a real world problem.

Roland

Am 27.11.2017 um 17:38 schrieb Christian Huitema:
> The network cannot control the congestion algorithm in the end points. 
> QUIC does not create the problem, just makes it crystal clear. But the 
> network can manage queues. So does the kernel.
>
> -- Christian Huitema
>
> On Nov 27, 2017, at 6:47 AM, Roland Zink <[email-masked] 
> <mailto:[email-masked]>> wrote:
>
>> necessary rights (like to load a kernel module) and do it although 
>> this may be difficult if you want to do it on million of devices. I'm 
>> more thinking of a slow process, say app 1 increase the initial 
>> window then app 2 thinks it needs to do the same and back out less 
>> than the default. Then app 1 is doing something and at some point 
>> nobody follows what is (will be) in the RFC. They not necessarily 
>> want to break something but at the end it can result into the same. 
>> It is no longer a single policy which applies to all apps instead 
>> each app comes with its own and the apps often can control both sides.
>>
>>
>> Roland
>>
>>
>>
>>> I would say that it is perfectly possible to trick e.g. the Linux 
>>> TCP stack to something like what is described below. Just clone 
>>> tcp_cong.c ,
>>>
>>>         In addition to how the spin bit can be designed and used,
>>>         I'd like those taking on this effort to consider this
>>>         question: Whatever network management problem you're
>>>         considering solving with a bit, what does it take to solve
>>>         this problem without the bit exposed? I'm asking you to
>>>         consider "zero-bit" solutions, or at least for a
>>>         cost/benefit analysis of developing solutions with and
>>>         without the spin bit for specific network management
>>>         functions. Specifically,
>>>
>>>         (i) What's impossible or very difficult to do, from a
>>>         network management perspective, of not having the spin bit?
>>>         Note that I'm not asking for what is and is not measurable
>>>         -- not everything measurable is useful. I'm specifically
>>>         asking in terms of network management functions, in
>>>         practice, as used by operators. Some of this has been
>>>         addressed in the discussions/drafts I've seen, but I think
>>>         it's most useful when framed in terms of network management
>>>         functions.
>>>
>>>     over others. No special rights are necessary. Now the question
>>>     is does QUIC provide enough manageability to avoid an Internet
>>>     (or some part of it) breakdown when it is misused? Can it be
>>>     shown that this can't happen?
>>>
>>> TSVWG and INTAREA. We would appreciate insight, and comments on 
>>> this.  (I am just about to push new version to correct typos!)
>>>
>>>
>>>
>>>         (ii) What are the alternatives for building the same
>>>         functions without the spin bit? Let's consider for a moment
>>>         that the spin bit isn't actually available in practice. What
>>>         will operators do to continue managing their networks? This
>>>         might be expensive, but that's precisely what I'm looking
>>>         for -- the cost to operators of not having the spin bit. For
>>>         instance, active probes are a fine way to measure network
>>>         RTT within operator networks, and that is an alternative.
>>>         There are surely others too. Their costs and limitations are
>>>         important to know about in order to reason about their
>>>         viability, so I'd like to see alternatives considered.
>>>

---

From: Christian Huitema <[email-masked]>
Date: Mon, 27 Nov 2017 10:02:27 -0800

On 11/27/2017 9:36 AM, Roland Zink wrote:

> If QUIC makes the problem crystal clear then now seems to be a good
> time to solve or mitigate it before risking it becomes a real world
> problem.

First, let's start by agreeing that this is not a problem specific QUIC.
You could say the same about Bit Torrent, any protocol that sends video
over UDP, or any application that opens multiple TCP connection. So,
yes, there may well be a problem, but since it is not specific to QUIC
the mitigation does not have to be specific to QUIC either.

But let's not panic. Practical mitigation has started long ago, with
ISPs effectively controlling how much data can be sent through a
customer's up-link. Maybe these current mitigations are not sufficient,
but if that is the case I would expect more work on better queue
management algorithm.

-- Christian Huitema

---

From: Ted Hardie <[email-masked]>
Date: Mon, 27 Nov 2017 10:30:07 -0800

Howdy,

> their viability, so I'd like to see alternatives considered.

I think the trade-offs here are harder to assess than this request
implies.  Let's imagine, for a moment, that the operator chooses to use the
coloring methods described in
https://datatracker.ietf.org/doc/draft-ietf-ippm-alt-mark .  There are
clearly expenses required for this and those might be higher than the
expenses for analyzing a QUIC-marked spin bit (with some variation,
depending on whether you do or don't require synchronization).  But the
other side of the assessment is the privacy assessment of this method.  If
an operator begins using this method, the very nature of the packet
coloring exposes the same data as the QUIC-marked spin bit does (and a bit
more because of the countable packet sets in flights).

Unless the replacement measurement uses purely synthetic flows, in other
words, it will also have privacy implications and, potentially, performance
implications.    Asking for a full-field analysis of all the options here
is not a reasonable task, at least in my opinion.  I think the best we can
realistically do is compare this against nothing (no bit and not
replacement effort), against synthetic flows (a.k.a. active measurements),
and against one or two systems either already deployed or fully described.

> those going off to do the writing to consider these questions.
While I am happy to consider the questions, I hope you are equally happy to
bound the universe of "alternatives for the same function" to be
considered.  Without doing so now, we run the risks of the goalposts
receding ever out of sight.

regards,

Ted

---

From: Roland Zink <[email-masked]>
Date: Mon, 27 Nov 2017 20:45:46 +0100

I agree it is not only QUIC. I also don't think it is necessary to panic 
now. We have issue #740 
(https://github.com/quicwg/base-drafts/issues/740) about adding a 
security issues section were existing means could be added. When QUIC 
can help to address such issues it can become more reliable so I 
wouldn't say this is out of scope just because other protocols have 
similar issues. If it really turns out that the current mitigations are 
not sufficient then work on better queue management algorithm will 
probably come too late.

Roland

---

From: Roberto Peon <[email-masked]>
Date: Mon, 27 Nov 2017 21:52:46 +0000

Application developers have been abusing TCP for years (open up many concurrent connections, etc.), so this isn’t really new.
In any case, QUIC hides its congestion avoidance from the network only partially: the network will always be able to see the number of packets, inter-arrival timings, etc. for a flow on a 5-tuple.
“The network” also has the ability to tunnel packets however it likes within an ASN. Spin bits or other data could be included in such an encapsulation.
-=R

From: QUIC <[email-masked]> on behalf of Roland Zink <[email-masked]>
Date: Monday, November 27, 2017 at 3:33 AM
To: "[email-masked]" <[email-masked]>
Subject: Re: Spin bit discussion - where we're at

- jana

[ACM]

[ACM]

regards,
Al

---

From: Ingemar Johansson S <[email-masked]>
Date: Tue, 28 Nov 2017 07:01:21 +0000

Hi

I really don't see that QUIC is able to solve this (congestion) issue. 
I pointed out ConEx (see eg. RFC6789) as one solution. And as Christian pointed out, operators already use rate shaping or policing or limit the amount of data.

Finally, even though there can be reasons to hack a congestion control algorithm that will help your QUIC connection to outcompete other flows, it also comes with the risk that said connection bloats its own last mile access link. So in the end you create most of the problems for yourself. For instance your LTE connection will only reward you with either high RTT or high loss or both if your congestion control is not enough responsive to congestion signals. 

/Ingemar

> From: Roland Zink [mailto:[email-masked]]
> Sent: den 27 november 2017 20:46
> To: Christian Huitema <[email-masked]>
> Cc: Ingemar Johansson S <[email-masked]>; Gorry (erg)
> <[email-masked]>; [email-masked]
> I agree it is not only QUIC. I also don't think it is necessary to panic now. We
> have issue #740
> (https://github.com/quicwg/base-drafts/issues/740) about adding a security
> issues section were existing means could be added. When QUIC can help to
> address such issues it can become more reliable so I wouldn't say this is out
> of scope just because other protocols have similar issues. If it really turns out
> that the current mitigations are not sufficient then work on better queue
> management algorithm will probably come too late.
> 
> 
> Roland
> 
> 
> 
> > You could say the same about Bit Torrent, any protocol that sends
> > video over UDP, or any application that opens multiple TCP connection.
> > So, yes, there may well be a problem, but since it is not specific to
> > QUIC the mitigation does not have to be specific to QUIC either.
> >
> > customer's up-link. Maybe these current mitigations are not
> > sufficient, but if that is the case I would expect more work on better
> > queue management algorithm.
> >
> > -- Christian Huitema
> >

---

From: Roland Zink <[email-masked]>
Date: Tue, 28 Nov 2017 11:52:27 +0100

Taking the spin bit and LTE as an example does it help? Assuming you 
placed an app on my phone and you activate it to do a DoS attack to 
bloat my own last mile access link. Then the RTT should go up for all 
the connections of my phone and the spin bit would allow to measure the 
RTT giving a hint what is going on. It's only a small piece of 
information but it may help.

Even worse would be if the LTE scheduler use a single queue for all 
phones of the cell and you fill the queue with useless packets doing a 
DoS attack to all users of a LTE cell. Here most implementations already 
seem to act fair against users resource block allocation and give them 
equal shares but your mileage may vary.

Of cause you can do the same attack today when you root / jailbreak my 
phone. Similar you can just use the amount of packets send/received but 
in LTE you don't know from this value if the link is saturated or not.

Regards,
Roland

---

From: "Black, David" <[email-masked]>
Date: Tue, 28 Nov 2017 23:52:40 +0000

> (On first glance, the - user - privacy aspects here seem to be much more contained, since ECN is mostly about the network exposing information to the end systems, and not vice versa.)

That sounds like the right high-level summary.  The fact that a transport protocol implementation supports ECN does not intrinsically expose any information about the user to the network. The primary information flow is congestion information flowing from the network to the endpoints.

Thanks, --David

From: tsv-area [mailto:[email-masked]] On Behalf Of Eggert, Lars
Sent: Monday, November 27, 2017 3:12 AM
To: MORTON, ALFRED C (AL) <[email-masked]>
Cc: QUIC WG <[email-masked]>; [email-masked]
Subject: Re: Spin bit discussion - where we're at

Hi,

On 2017-11-26, at 20:26, MORTON, ALFRED C (AL) <[email-masked]<mailto:[email-masked]>> wrote:

Lars

---

From: "Brian Trammell (IETF)" <[email-masked]>
Date: Wed, 29 Nov 2017 09:26:29 +0900

hi, David,

> On 29 Nov 2017, at 08:52, Black, David <[email-masked]> wrote:
> 

There are three possible states for an ECN negotiation: not attempted, failed, and succeeded. Each of these can add a fractional bit of information about the client and server TCP implementations. If a server negotiates ECN, you can be reasonably certain that it supports ECN, and can leverage the observations that sysadmins love defaults and servers are mostly Linux these days to figure out whether it's running a kernel before or after server side defaults were turned on.

Long-term observation of the ECN negotiation attempts of a client can be used to determine if the client is using probabilistic negotiation by default; i.e. is an Apple device of a certain vintage.

These are fractional bits of fingerprinting information, though, if that. I'm almost certain (but won't take the time to do the research now) that any of these observations would be useless. Anyone in a position to make them could also get much more information about the behavior of the TCP stacks at each end, such that the ECN bits add negligible information. cf. p0f, if that's still a thing.

Cheers, B

---

From: "Eggert, Lars" <[email-masked]>
Date: Wed, 29 Nov 2017 07:09:06 +0000

Hi,

> These are fractional bits of fingerprinting information, though, if that. I'm almost certain (but won't take the time to do the research now) that any of these observations would be useless. Anyone…

this is certainly all true for TCP and ECN, where whether ECN is available and/or tried by default is tied to a certain kernel versions.

But for QUIC (= ECN with UDP), whether ECN is used or not is completely up to the app (= the QUIC stack), so you can't draw conclusions about whether a certain kernel or OS flavor is underneath (other than they less than about ten years old, i.e., have support for the IP_TOS/IP_RECVTOS socket options.)

So the fractional information leakage seems to be less than with TCP.

Lars