Subject: RE: [PCN] PCN architecture, revised sub-section on probing

From: "Geib, Ruediger" <[email-masked]>
Date: Mon, 1 Oct 2007 10:46:22 +0200

Lars,

thanks, I fully agree. I'm not looking for perfect solutions. 

To me, the first generation of PCN specs should result in simple, 
stable and secure in implementation and operation. The specs 
should state their shortcomings and future standards may propose 
more perfect solutions.

If PCN domains can provide the desired service quality for admitted 
flows without probing and ECMP awareness, I prefer not to work on 
both for now. 

Regards,

Ruediger

|-----Original Message-----
|From: Lars Eggert [mailto:[email-masked]]
|Sent: Saturday, September 29, 2007 8:31 AM
|To: ext Steven Blake
|Cc: pcn
|Subject: Re: [PCN] PCN architecture, revised sub-section on probing
|
|
|On 2007-9-28, at 21:15, ext Steven Blake wrote:
|> If a probe packet for a PCN-managed flow is intended to follow the  
|> same
|> ECMP path as the flow, then the probe packet has to look exactly like
|> flow packets in the header fields usually hashed by routers to load
|> split for ECMP: IP source/destination address, TCP/UDP
|> source/destination port; maybe the protocol/next header 
|field as well.
|>
|> If you do this, you are essentially spoofing traffic for the host
|> originating the flow.  I'm sure the security review comments for this
|> will make for entertaining reading.
|>
|> At the very least, you need to ensure that there is an *airtight*
|> mechanism for ensuring that the probe traffic can never leak out of  
|> the
|> PCN domain, and that no ICMP traffic can be sent back towards the
|> host.
|
|Let me add that probe traffic (other than for supporting ECMP) seems  
|only useful when a domain can go from "no congestion" to "must  
|terminate" through the admission of a very small number of flows.  
|Otherwise, the gradual admission of regular flows can be used to  
|determine whether admission should be terminated. In other words,  
|regular flows can be used to "probe" the domain.
|
|I question whether domains that require explicit probing mechanisms  
|fall under the current PCN charter, which focuses on domains that  
|multiplex a sufficient numbers of flows such that stateless,  
|statistical mechanisms are effective.
|
|Explicit probing adds a lot of complexity. The idea behind PCN is to  
|add a minimal set of mechanisms to a large and busy DiffServ domain,  
|which would allow this domain to protect the QoS of already-admitted  
|flows in a best-effort sort of way. Meaning that although flow  
|termination should be mostly prevented due to admission stop  
|decisions, there will be no guarantees that no flow will ever be  
|terminated. PCN is also not a mechanism to squeeze the last ounce of  
|utilization out of a domain - sufficient overprovisioning remains a  
|requirement. Because of this, I wonder if PCN needs to support ECMP -  
|which did not come up during the chartering phase - when it looks  
|likely that such support will significantly complicate the solution.
|
|Finally, let me say that I'm worried that the WG seems to have  
|settled on a mode of working where all the different proposals will  
|be merged into one. The result is not likely to be a very simple  
|solution. We have ample experience with solutions that have been  
|designed in this manner failing to gain any traction, due to their  
|complexities.
|
|I'd like to strongly encourage the WG to instead discuss which of the  
|proposals is the most *simple* one that will fulfill the chartered  
|goals. We'd like the outcome of the first phase of work to be  
|something that people will usefully deploy in some domains. Without  
|demonstrated deployability and use, the need for continued future  
|work on PCN is questionable.
|
|Lars
|
|
|

_______________________________________________
PCN mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pcn

---

From: "Hancock, Robert" <[email-masked]>
Date: Mon, 1 Oct 2007 10:55:02 +0100

hi,

a comment on probe messages: 

> -----Original Message-----
> From: Lars Eggert [mailto:[email-masked]] 
> Sent: 29 September 2007 07:31
> To: ext Steven Blake
> Cc: pcn
> Subject: Re: [PCN] PCN architecture, revised sub-section on probing
> 
> On 2007-9-28, at 21:15, ext Steven Blake wrote:
> > If a probe packet for a PCN-managed flow is intended to follow the 
> > same ECMP path as the flow, then the probe packet has to 
> look exactly 
> > like flow packets in the header fields usually hashed by routers to 
> > load split for ECMP: IP source/destination address, TCP/UDP 
> > source/destination port; maybe the protocol/next header 
> field as well.
> >
> > If you do this, you are essentially spoofing traffic for the host 
> > originating the flow.  I'm sure the security review 
> comments for this 
> > will make for entertaining reading.
> >
> > At the very least, you need to ensure that there is an *airtight* 
> > mechanism for ensuring that the probe traffic can never leak out of 
> > the PCN domain, and that no ICMP traffic can be sent back 
> towards the 
> > host.

Having been through a very similar question in the context of another
protocol, my feeling is that this level of 'faithfulness' in the 
probe messages at least is over-ambitious. There are conflicting 
requirements:

- you accept that ECMP can load share on nearly anything, i.e. the
interior packet classifiers can look far into the packet for fields 
to distinguish microflows, but
- you rely on egress packet classifiers to look into the packet to 
find fields to disambiguate probe and real traffic

It's hard to think of deployment approaches which would be robust
in meeting these requirements. Even in a single deployment, there
could be heterogeneous classification capabilities in the interior 
and edge routers. 

(For this circumstance at least, it would be nice if there was some
guidance or baseline assumption on exactly how fine-grained it is
sensible for things like ECMP to be. I'm not sure how practical
that is, however.)

robert h.

> 
> Let me add that probe traffic (other than for supporting 
> ECMP) seems only useful when a domain can go from "no 
> congestion" to "must terminate" through the admission of a 
> very small number of flows.  
> Otherwise, the gradual admission of regular flows can be used 
> to determine whether admission should be terminated. In other 
> words, regular flows can be used to "probe" the domain.
> 
> I question whether domains that require explicit probing 
> mechanisms fall under the current PCN charter, which focuses 
> on domains that multiplex a sufficient numbers of flows such 
> that stateless, statistical mechanisms are effective.
> 
> Explicit probing adds a lot of complexity. The idea behind 
> PCN is to add a minimal set of mechanisms to a large and busy 
> DiffServ domain, which would allow this domain to protect the 
> QoS of already-admitted flows in a best-effort sort of way. 
> Meaning that although flow termination should be mostly 
> prevented due to admission stop decisions, there will be no 
> guarantees that no flow will ever be terminated. PCN is also 
> not a mechanism to squeeze the last ounce of utilization out 
> of a domain - sufficient overprovisioning remains a 
> requirement. Because of this, I wonder if PCN needs to 
> support ECMP - which did not come up during the chartering 
> phase - when it looks likely that such support will 
> significantly complicate the solution.
> 
> Finally, let me say that I'm worried that the WG seems to 
> have settled on a mode of working where all the different 
> proposals will be merged into one. The result is not likely 
> to be a very simple solution. We have ample experience with 
> solutions that have been designed in this manner failing to 
> gain any traction, due to their complexities.
> 
> I'd like to strongly encourage the WG to instead discuss 
> which of the proposals is the most *simple* one that will 
> fulfill the chartered goals. We'd like the outcome of the 
> first phase of work to be something that people will usefully 
> deploy in some domains. Without demonstrated deployability 
> and use, the need for continued future work on PCN is questionable.
> 
> Lars
> 
> 
>

---

From: "Geib, Ruediger" <[email-masked]>
Date: Mon, 1 Oct 2007 12:29:39 +0200

Phil,
 
My replies in line, marked as [RG_later].
 
Regards,
 
Ruediger

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
Sent: Friday, September 28, 2007 7:30 PM
To: Geib, Rudiger
Cc: [email-masked]
Subject: RE: [PCN] PCN architecture, revised sub-section on probing

Ruediger,

Follow-ups (further questions!) in-line on each of your 3 points 

[RG _earlier] - the discussion on the decision when to apply probing
misses one important aspect, at least if 

  you look at a single domain: probing is only required if any
pre-congestion occurs within this 

  domain. If there's no pre congested link within a domain, no probing
is required within this 

  domain. 

[phil] I slightly re-phrased the first bullet (the second bullet is
unaltered), so the text now says:

Probing is useful or even essential under the following conditions:

   o  when an ingress-egress-aggregate carries no traffic (or too little
traffic for the PCN-egress-node to accurately make the "measurements of
PCN-traffic" that are required for an admission decision), but the
traffic levels on other ingress-egress-aggregates are so high that a new
flow shouldn't be admitted on the 'empty' ingress-egress-aggregate.
Probing is useful to check this.

   o  in the presence of multipath routing (ECMP) between the
PCN-boundary-nodes, when some paths are pre-congested there may be other
paths which aren't pre-congested.  Probing is useful to determine
whether the new flow would follow a path that isn't pre-congested and
hence can be admitted.

I think what's written implies that it may not always be possible to
know whether there's any (relevant) pre-congestion or none - so there's
an argument for probing in such times of doubt to discover what the
situation really is (if it turns out that there isn't any pre-congestion
then, as you say, in some ways the probing hasn't proved to be useful).
I think the tweaked text above captures this, but please suggest any
mods to make it clearer (or disagree!) 

[RG_later] suggestion last sentence first bullet point: "...Probing may
be useful to check this." Probing results in more complex edge system
implementations and I'm not convinced that it provides benefits. As
mentioned in another thread, if PCN provides the expected service
quality for admitted flows also without probing, I prefer not to probe
for the first round of PCN specs.

[RG _earlier } - given that probing is only applied within a domain, if
there's pre-congestion on any link, then 

  the only purpose of probing is to find out, whether traffic from the
particular ingress to a 

  desired egress would contribute to this pre-congestion. The probing
flow then should be 

  designed to ensure that packets of the probing flow are marked (and if
interpretation is 

  to be as simple as possible: marked probing packets mean no
admission). 

[phil] I'm not quite sure what you mean by "designed to ensure that
packets of the probing flow are marked". The section currently says that
probe pkts are handled in the same way as (ordinary data) PCN-pkts [*].
I think you may be suggesting something different, that probing pkts
should be preferentially PCN-marked? 

[*] incidentally, I'll tweak the text to add something like "in terms of
pkt scheduling/forwarding and PCN-marking" ; for pkt dropping it may be
arguable you should preferentially drop probes to ease DoS worries?

Re: "interpretation is to be as simple as possible: marked probing
packets mean no admission" - I think this is too solution-space for this
Architecture draft (presumably there may be different options, eg block
call if the single probe pkt is marked, if at least 2 out of 100 marked,
etc) 

[RG_later] Probing needs to be done at rate with a packet size and
during a determined probing intervall. The receiver needs to know, which
packet is the first and which is the last. What I want to say is,
"simple" probing works with a single rate/packet size/probing intervall.
These are not related the properties of to the flow for whose admission
the probing is applied. As the probe flow properties are not related to
the real flow properties, admission decisions should be rather simple
too. Ehat's the benefit if  a node starts to calculate rates of
pre-congestion marks for a probe flow whose properties aren't related to
the flow to be admitted?

To complete my statement: rather that having a measurement function
trying to simulate the flow requesting admission more exactly, I prefer
to have no measurement functionality. I prefer simple edge nodes.

[RG _earlier] - If there's no flow between an ingress and an egress and
ECMP is operational within a 

  domain, the ingress may have to signal its intention to probe to an
egress.

[phil] could you explain your reference to ECMP please?

At the moment the doc says that "(if required) Communicate the request
that probing is needed - the PCN-egress-node signals to the
PCN-ingress-node that probe traffic is needed". But you're suggesting
there are particular circumstances when something is needed in the
opposite direction? 

[RG_later]: Please forget the ECMP statement. The PCN ingress node first
signals the per flow reservation request to the egress node and then the
egress checks, whether any admitted flows from this ingress pass it
already. Then the egress requests the ingress to probe. If that's what
your text means, it should say so. Or at least it should say, that the
egress must learn that there's a new flow to be admitted prior to
requesting an ingress to probe. I'd like to avoid probing if there's no
need to probe.

Thanks!

phil

---

From: Lars Westberg <[email-masked]>
Date: Mon, 01 Oct 2007 14:08:43 +0200

hi Robert.

I think the task is to do something very simplistic and skip the 
compexity to other solutions.
 If you want to have something complex, one can use NSIS. I am very 
worried to go into the complexity swamp.

-Lasse

---

From: "Jozef Babiarz" <[email-masked]>
Date: Mon, 1 Oct 2007 21:25:40 -0400

Issue with probing.

First, I believe that the PCN architectural draft should discuss probing
as it pertains to PCN. I also think that it should articulate what
problems probing is trying to solve. The architectural draft should
position (which I think it does) that probing should be used when needed
as there are deployment scenarios where probing may not be needed. 

Yes, probing between PCN boundary nodes needs to be designed carefully
as there are security concerns that need to be addressed. The design of
the PCN probing mechanism needs to be document in a stand alone draft so
that it can be used in a PCN solution when needed. This means that not
every PCN domain needs to support probing but some PCN domains may
choose to support probing because it solves specific configuration
issues (ECMP, etc.) that are important to them. 

I believe that the probing issues needs to be address by the PCN group
in a detailed proposal document in the draft and present to the group.
Once we have a good understanding of the proposed probing solution then
we can pass judgment on complexity, security, and performance. 

Regards, Joe
email:[email-masked]
Telephone:613-763-6098

---

From: "Georgios Karagiannis" <[email-masked]>
Date: Tue, 2 Oct 2007 10:59:30 +0200

Dear all

I agree with the solution proposed by Jozef on the probing issue!

Best regards,
Georgios

> From: Jozef Babiarz [mailto:[email-masked]] 
> Sent: dinsdag 2 oktober 2007 3:26
> To: Lars Eggert; ext Steven Blake
> Cc: pcn
> First, I believe that the PCN architectural draft should 
> discuss probing as it pertains to PCN. I also think that it 
> should articulate what problems probing is trying to solve. 
> The architectural draft should position (which I think it 
> does) that probing should be used when needed as there are 
> deployment scenarios where probing may not be needed. 
> 
> Yes, probing between PCN boundary nodes needs to be designed 
> carefully as there are security concerns that need to be 
> addressed. The design of the PCN probing mechanism needs to 
> be document in a stand alone draft so that it can be used in 
> a PCN solution when needed. This means that not every PCN 
> domain needs to support probing but some PCN domains may 
> choose to support probing because it solves specific 
> configuration issues (ECMP, etc.) that are important to them. 
> 
> I believe that the probing issues needs to be address by the 
> PCN group in a detailed proposal document in the draft and 
> present to the group.
> Once we have a good understanding of the proposed probing 
> solution then we can pass judgment on complexity, security, 
> and performance. 
> 
> Regards, Joe
> email:[email-masked]
> Telephone:613-763-6098
> 
> 
>

---

From: Michael Menth <[email-masked]>
Date: Tue, 02 Oct 2007 11:21:28 +0200

I agree.

    Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

---

From: Lars Eggert <[email-masked]>
Date: Wed, 3 Oct 2007 11:05:51 -0400

Hi,

> as there are deployment scenarios where probing may not be needed.

adding probing functionality to PCN - even as an optional component -  
is a very significant step. It changes the model of PCN from a  
reactive one, where the _egress_ monitors congestion levels and  
responds to imminent congestion, to a proactive one, where the  
_ingress_ determines whether a path can support a new flow.  
Similarly, it changes PCN from an optimistic mechanism, where flows  
are admitted until congestion is imminent, to a pessimistic one,  
where flows aren't admitted until the path has been probed. These are  
fundamentally different approaches, and designing an architecture  
that supports both will add complexity. I personally remain  
unconvinced that these complications have been sufficiently motivated.

> as there are security concerns that need to be addressed.

It's not only security. Probing adds at least several roundtrips of  
delay, so that a sufficient volume of probe traffic can be sent to  
determine whether the path can support a subsequent flow. Also, it is  
unclear how the probe traffic is being prevented from negatively  
affecting regular flows. Without such mechanisms, what's the benefit  
over simply admitting the new flow? Etc. There are so many open  
questions around such a mechanism that I question whether the IETF is  
the right place for this discussion. As far as I'm aware there is no  
other Internet protocol that uses something similar to probing.

> The design of
> that it can be used in a PCN solution when needed.

It's too early to discuss where probing should be described.

> This means that not

The need for probing hasn't been sufficiently motivated. It remains  
unclear what the benefits are, and it it certainly isn't part of the  
PCN mechanisms that are outlined in the current charter.

> we can pass judgment on complexity, security, and performance.

I disagree. The WG should discuss the simplest possible solution that  
will fullfil the chartered goals.

I can't stress enough that simplicity is key. It's perfectly OK if  
the initial PCN solution cannot support domains that use ECMP or  
don't have a lot of statistical multiplexing. As long as it offers  
sufficient benefit to some domains such that it will see deployment  
and use, it can always be extended later. (Heck, if PCN offers  
sufficient benefit, domains may end up turning off ECMP to run PCN.)

Finally, it's not only whether or not PCN needs probing. As I said in  
my last email, I'm worried that the WG generally seems to have  
settled on a mode of working where all the different proposals and  
ideas will be merged into one. The result is not likely to be a very  
simple solution. We have ample experience with solutions that have  
been designed in this manner failing to gain any traction, due to  
their complexities.

Merging all proposals into one is a feel-good approach that  
guarantees that there won't be any losers - all proposals live on in  
some form - but it doesn't guarantee that the merged outcome will be  
simple, usable and deployable.

As I've said before, I'd like to strongly encourage the WG to instead  
discuss which of the proposals is the best basis for a *simple*  
solution that will fulfill the chartered goals. We'd like the outcome  
of the first phase of work to be something that people will usefully  
deploy in some domains. Without demonstrated deployability and use,  
the need for continued future work on PCN is questionable.

Lars

---

From: "Anna Charny (acharny)" <[email-masked]>
Date: Wed, 3 Oct 2007 12:51:27 -0400

Hi Lars and all,

Lars,  I do not really see that the WG is trying to come up with a
solution that encompasses all proposals.  While it is true that at the
moment the architecture draft lists a number of options,  a decision on
how to narrow down these options indeed needs to be made, IMHO.  

We have started work on the document that does functional and
algorithmic comparison of various options, with a vew to provide input
to the decision on which options would go into the initial PCN WG work,
and which could be delayed.  We hope to present a draft the the list
early November, and certainly in time for the new draft deadline.  It
would be ideal to also have a thorough performance comparison as well.
While a number of steps to that effect have been taken by various
simulation efforts,  it is not easy to compare simulation results done
in different environments and frequently under different assumptions.
We are trying our best to bridge that gap, but it remains to be the case
that some algorithms/approaches have been subject to more scrutiny than
others.

As to probing,  I agree that the initial solution/WG output may be
viable without probing at all. However, as there is a possibly
significant benefit (and indeed also a possibly significant additional
complexity) that probing might bring,  I do not see any harm discussing
probing in the WG - as long as it does not delay milestones further.

Anna 

> Sent: Wednesday, October 03, 2007 11:06 AM
> To: ext Jozef Babiarz
> Cc: pcn
> > First, I believe that the PCN architectural draft should discuss 
> > probing as it pertains to PCN. I also think that it should 
> articulate 
> > what problems probing is trying to solve. The architectural draft 
> > should position (which I think it does) that probing should be used 
> > when needed as there are deployment scenarios where probing 
> may not be 
> > needed.
> 
> adding probing functionality to PCN - even as an optional 
> component - is a very significant step. It changes the model 
> of PCN from a reactive one, where the _egress_ monitors 
> congestion levels and responds to imminent congestion, to a 
> proactive one, where the _ingress_ determines whether a path 
> can support a new flow.  
> Similarly, it changes PCN from an optimistic mechanism, where 
> flows are admitted until congestion is imminent, to a 
> pessimistic one, where flows aren't admitted until the path 
> has been probed. These are fundamentally different 
> approaches, and designing an architecture that supports both 
> will add complexity. I personally remain unconvinced that 
> these complications have been sufficiently motivated.
> 
> It's not only security. Probing adds at least several 
> roundtrips of delay, so that a sufficient volume of probe 
> traffic can be sent to determine whether the path can support 
> a subsequent flow. Also, it is unclear how the probe traffic 
> is being prevented from negatively affecting regular flows. 
> Without such mechanisms, what's the benefit over simply 
> admitting the new flow? Etc. There are so many open questions 
> around such a mechanism that I question whether the IETF is 
> the right place for this discussion. As far as I'm aware 
> there is no other Internet protocol that uses something 
> similar to probing.
> 
> > The design of

---

From: <[email-masked]>
Date: Tue, 16 Oct 2007 17:45:13 +0100

Rob

Thanks for the experience tip from nsis-land. I'd assumed that ecmp
worked on source and destination IP addresses and port numbers, the
protocol ID and the DSCP - therefore the pcn-egress-node can choose
other bit(s) in pkt to distinguish probe from data. However, I hadn't
thought about the general case where the pcn-egress doesn't know what
fields ecmp might be working on [what the pkt classifiers are looking
at] - then you're stuffed. 

phil

-----Original Message-----
From: Lars Westberg [mailto:[email-masked]] 
Sent: 01 October 2007 13:09
To: Hancock, Robert
Cc: pcn
Subject: Re: [PCN] PCN architecture, revised sub-section on probing

hi Robert. 

-Lasse

Hancock, Robert wrote:

hi,

robert h.