Subject: RE: [PCN] Fwd: I-D ACTION:draft-eardley-pcn-architecture-00.txt

From: <[email-masked]>
Date: Mon, 6 Aug 2007 12:09:04 +0100

One issue I wanted to flag from the comments in your PDF.

In S4, in the part that begins "The following are some high-level points
about how PCN works:"
You added the bullet: "o It is assumed that PCN marking is being applied
to traffic scheduled with the expedited forwarding [RFC3246] per-hop
behaviour."

Do people agree with adding this bullet?

Thanks
Phil/

> -----Original Message-----
> From: Briscoe,RJ,Bob,XVR9 BRISCORJ R
> Sent: 25 July 2007 00:44
> To: Eardley,PL,Philip,CXR9 R
> Cc: [email-masked]
> Subject: RE: [PCN] Fwd: I-D
ACTION:draft-eardley-pcn-architecture-00.txt
> 
> Phil and all the authors of this PCN architecture draft,
> 
> Like Lars, I'd like to congratulate you all on such a mature first
draft,
> and on managing to encompass many different views we have seen from
> various
> people without making the thing an uncontrollable monster full of
options
> and choices.
> 
> I just read it all through and my review comments are given below.
> 
> Also see
>
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/ipe2eqos/gqs/papers/dr
af
> t-eardley-pcn-architecture-00_rvw_bb.pdf>
> for suggested minor text changes.
> 
> I haven't supplied text for more major changes inline below until the
> authors agree with the need for change. But then I can supply text if
> required.
> 
> At 16:59 28/06/2007, [email-masked] wrote:
> >Hi all,
> >
> >Just a reminder that all comments on this draft would be great. It
aims
> >to describe the PCN architecture, in light of the PCN WG's Charter &
its
> >Milestone of an Info doc on 'Flow Admission and Termination
Architecture
> >within a Diffserv Domain' (due Nov 07).
> 
> General comments:
> 
> * WG process management
> A lot of WG process management text, esp in S.3, but also the list of
> possible flow termination mechanisms in S.4. All instances hilited in
> magenta in the review at the above URL.
> See separate mail about this to the PCN WG chairs (cc PCN list)
> Subject: How to discuss WG scoping decisions in an RFC?
> 
> * Choices vs decisions
> I believe an architecture doc should say where one decision is
required or
> where multiple options are reasonable. Currently, the arch is good at
> laying out the design space, but I'd like us to try to say which
choices
> have to be made by the IETF, which should be left to vendors and which
> left
> to operators.
> 
> >S1 Introduction.
> >This is quite short. If desired, it could be boosted with a general
> >explanation of where PCN fits into the picture of QoS and how it's
> >evolved (compare Section 1.1 of draft-chan-pcn-problem-statement-01)
> 
> * Clarify that a packet might encounter more than one PCN domain along
its
> path, each being a separate admission control hop.
> 
> * Clarify that a "Diffserv domain" may be multiple concatenated
autonomous
> systems, but in this draft they are assumed to trust each other and
have
> identical configuration.
> 
> >S2 Terminology
> >In the "Editor's note" are 3 alternative terms that some of the
authors
> >preferred.
> 
> * ECN _and_ PCN marking?
> 
> "   o  Configured-termination-rate: [...]
>                                 Normally it is configured to be less
than
>        the maximum rate at which PCN-traffic can be forwarded on the
>        link, so that termination-marking occurs before any significant
>        queuing, ECN-marking or loss of PCN-packets.
> "
> 
> The above text implies (to me) that a PCN-interior router might do PCN
> marking _and_ also ECN marking just before drop. That also imposes a
> requirement on the encoding to be able to carry either in the same
field.
> I don't think the authors meant either of these implications. So
> somewhere,
> the relation between PCN & ECN marking should be stated.
> 
> Although this is perhaps too detailed for the architecture, I suggest
that
> any need for ECN marking on the same path but outside the PCN
domain(s)
> should be handled by tunnelling across the PCN domain, with a non-PCN
DSCP
> in the inner header (that turns on the RFC3168 meaning of its ECN
field)
> and a PCN DSCP in the outer.
> 
> * Marking is when "current" rate is "above" a reference rate?
> 
> "   o  Admission-marking: the marking of PCN-packets by a PCN-node to
>        indicate that the PCN-traffic on a link is above the
configured-
>        admissible-rate.
> 
>     o  Termination-marking: the marking of PCN-packets by a PCN-node
to
>        indicate that the PCN-traffic on a link is above the
configured-
>        termination-rate.
> "
> 
> These marking definitions and other examples hilighted in magenta in
the
> above PDF prejudge marking mechanisms as rate measurements.
> 
> We don't just have marking when the _current_rate is _above_ the
reference
> rate. We can have marking when the _current_ rate is _below_, or when
the
> _past_ rate was _above_.
> 
> The configured rates are parameters of a (to be defined) mechanism.
These
> mechanisms will not have to mark packets when the rate is above these
> configured rates. That depends on the mechanism.
> 
> This sounds picky, but it's actually a big rant I have about all this
> loose
> discussion of rate (yes, the issue of what rate means is mentioned in
the
> draft, but we need to be careful about such sloppy usage). IMHO, it's
> important to scale our conception of time down to inter-arrival times,
to
> be precise. Then the rate is packet size/interarrival time. Therefore
seen
> at a small enough timescale, rate might be varying rapidly above and
below
> the configured rate.
> 
> A marking mechanism could be implemented by a virtual queue rather
than a
> rate measurement. When the offered instantaneous rate has been below
the
> configured-admission-rate for many packets (perhaps hundreds), marking
> will
> and should still occur,... if the virtual queue is still above a
marking
> threshold even if it is emptying rather than filling. This can be a
> deliberate design choice that says we still want admission marking
when
> there has been recent overload to allow time for the control loop to
work.
> It ensures a greater safety margin is automatically created by the
marking
> mechanism when recent traffic had higher variance. This was the
marking
> algorithm that was most studied and simulated in CL-PHB draft.
> 
> The unintended implication that all packets are either marked or not
> marked
> should also be carefully avoided. The draft needs to say explicitly
that
> "Admission (or Termination) marking will be an encoding of some
> combination
> of codepoints in the IP header to signal a varying fractional number
from
> a
> path of PCN-interior-nodes to a PCN-egress-node"
> 
> * inelastic
> Do you think we should define "inelastic" or is this a well-known IETF
> term? We could refer to [Shenker95], which I think was the first to
define
> the term. Or is there an RFC that defines this (hopefully with ref to
> Shenker)?
> 
> Scott Shenker, "Fundamental Design Issues for the Future Internet",
In:
> IEEE Journal on Selected Areas in Communications 13 (7) pp. 1176--1188
> (1995).
> 
> 
> >S3 Assumptions and constraints on scope
> >These are the 4 things mentioned in the Charter, plus some
explanation
> >of them. Are they clear? Also we mention some of the ways that a
future
> >revised Charter might look at overcoming some of the
> >constraints/assumptions; is this sub-section at the right depth?
> 
> * Are all the scoping assumptions equal relaxable?
> 
> Just a thought: might it be worth giving an indication of which
> assumptions
> are likely to be hard or impossible to relax and which not (e.g. I
think
> the aggregation assumption is implicit to all MBAC (measurement-based
Adm
> Ctrl). Also inelasticity is fairly fundamental - you don't need
admission
> control if flows aren't inelastic.
> 
> This would affect the last para, which implies all the assumptions
might
> be
> relaxed one day.
> 
> >S4 High-level functional architecture
> >We have tried to write this section (and the following ones) so that
it
> >fits all the various proposals there've been for PCN mechanisms. Does
> >this make the section too wishy-washy or too hard to understand?
Should
> >it include some comparison of the different mechanisms proposed
> >(PCN-interior-node marking algorithms & PCN-boundary-node reactions)?
> 
> * Functional or topological?
> Is it intentional that the high level functions are classified by
which
> type of node does them in 3 cases and what function is to be done in
the
> others? Would it be possible to define functions in the abstract
first,
> then say which functions have to be implemented on certain types of
nodes?
> Or does this just make it unnecessarily incomprehensible?
> 
> Gaps:
> 1/ Discuss which PCN node needs to know the IP address of which other
PCN
> node in different configurations. And state requirements this implies
on
> the signalling protocol, or on configuration.
> Minimally, only the egress needs to know the ingress address. In
> centralised admission control scenarios, either the egress or the
ingress
> or both might need to be configured with a central address.
> 
> 2/ Related to knowing addresses of other nodes: When a PCN node
receives a
> message claiming to be from another PCN node, to validate the msg is
> indeed
> from the node it claims to be, are there any different issues that PCN
> raises relative to typical reservation signalling systems?
> draft-lefaucheur-rsvp-ecn-01 (expired) contains a good deal of text on
> this
> issue that should be generalised into this arch doc.
> 
> >S5 Detailed Functional architecture
> >Is this a reasonable description of the extra functionality that PCN
> >requires on various nodes in the PCN-domain? Is it the right way to
> >split up the description? For clarity / help reader's understanding,
> >should there be some specific examples of how functionality might be
> >distributed (eg "if you followed the deployment model in
> >draft-briscoe-tsvwg-cl-architecture-04, then this measurement is made
at
> >the PCN-egress-node, communicated to the PCN-ingress-nodes which
makes
> >the admission decision; etc.")
> 
> S.5.5 Probing functions
> 
> If we are going to consider a case where the ingress has prior
knowledge
> of
> which egress is associated with which destination address (e.g. with a
> separate signalling routing table as discussed in NSIS, or if the
> signalling protocol goes receiver-sender-receiver), then the ingress
could
> know that it has no reservations with that egress as soon as a
reservation
> request arrives, so it can unilaterally start probling.
> 
> S.5.6 Flow Termination Functions
> 
> The text seems to assume the egress makes the decision on which flows
to
> lose and tells the ingress. In cl-architecture (for instance) the
egress
> meters, feeds back the measurement to the ingress, which meters the
load
> it
> is forwarding and then the ingress calculates how much traffic to
> terminate, and either chooses the flows itself, or asks a central node
to
> choose.
> 
> I think there has to be metering at both boundaries otherwise we can't
> work
> out how much traffic has been dropped (as well as termination marked).
The
> decision might be at either (but in cl-architecture we had good
reasons
> for
> choosing the ingress because it has the most recent knowledge about
load
> (given flows might be being abandoned by users very quickly and/or
added
> very quickly during a disaster).
> 
> Other more incremental mechanisms have now been defined that don't
> calculate how much load to shed in one pass, but keep trying. Have we
> abandonned the earlier approach?
> 
> Whatever, I think a feedback function is missing between the boundary
> nodes.
> 
> >S6 Design goals and challenges
> >This briefly describes some open issues, taken from
> >briscoe-tsvwg-cl-architecture. Are there other ones that should be
> >mentioned? Is the problem description at the right level of depth?
> >Should we discuss various possible solutions to these problems?
> 
> I get the feeling we need a doc specifically on ECMP solutions?
> 
> >S7 Deployment scenarios
> >Briefly describes some deployment scenarios for pcn? is this at the
> >right level of depth?
> 
> Just a thought - I felt this section would have been better earlier.
> 
> >S8 Operations and Management
> >This section was written in response to the Charter saying that the
> >architecture document should include security, manageability and
> >operational considerations. The draft addresses this by providing
some
> >thoughts under the FCAPS headings: OAM of Faults, Configuration,
> >Accounting, Performance and Security? Is this the right way of
> >structuring it - does it cover the right set of topics? Is the text
at
> >the right level? - eg should it also have a detailed set of
parameters
> >that would be available for configuration?
> 
> * Don't think this is Fault OAM?
> Echoing what someone else (I forget who) said on the list, Fault OAM
isn't
> really about how the control system recovers from failures (that's
> control), it's how to tell the management system/manual operator that
you
> have recovered from a failure (or not).
> 
> Also, we should cover faults other than node or link failures. E.g. a
> wrongly configured address in a node, or a wrong address given in a
> singalling protocol, or a wrongly configured parameter in a queueing
algo.
> Etc.
> 
> * Config OAM: Add:
> - level of AM that leads to denying new admissions.
> - config paramteres that control distribution of functions?
> 
> * Don't think this is Performance OAM?
> I think Perf OAM is about monitoring performance at run-time, not
> characterising performance at design time.
> 
> * Don't think this is Security OAM?
> Again, as someone said, security OAM is finding out about security
> breaches
> or near-misses at run-time, not identifying security flaws in the
design,
> which is security the considerations section.
> 
> * Security Considerations Gaps:
> - PCN marking selectively applied to certain flows, rather than
randomly
> by
> interior node.
> - The statement about DoS attacks is a requirement with no pointers to
the
> detailed DOS issues, or any discussion of best practice design. I
could
> supply text here (among many other things to do!)
> - ref to hop-by-hop signalling authentication [RFC2747 for RSVP] and
the
> new recently posted draft to tsvwg on group keying for RSVP
authentication
> <draft-behringer-tsvwg-rsvp-security-groupkeying-00.txt>. I haven't
read
> it
> yet, but on a scan it's as relevant as RFC2747.
> 
> 
> Phew... that's all for now.
> 
> 
> Bob
> 
> 
> >An overall question is whether the draft should have more comparison
of
> >the options (pros/cons) for various aspects.
> >
> >I aim to edit another version of the draft before the ietf (but maybe
> >not before the deadline as I'm on hols next week).
> >
> >Thanks!
> >Phil/
> 
>
________________________________________________________________________
__
> __
> Bob Briscoe, <[email-masked]>      Networks Research Centre, BT
> Research
> B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473
> 645196
> 

_______________________________________________
PCN mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/pcn

---

From: Tina TSOU <[email-masked]>
Date: Mon, 06 Aug 2007 19:52:48 +0800

Hi Phil,
I agree with adding it.

B. R.
Tina

----- Original Message ----- 
From: <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Sent: Monday, August 06, 2007 7:09 PM
Subject: RE: [PCN] Fwd: I-D ACTION:draft-eardley-pcn-architecture-00.txt

Thanks
Phil/

draft,
options
aims
its
Architecture
required or
choices
its
autonomous
have
authors
than
field.
that
domain(s)
DSCP
field)
configured-
to
configured-
the
reference
the
These
the
to
seen
below
than a
the
marking
when
work.
marking
marking
that
from
define
explanation
future
think
Adm
admission
might
it
Should
which
the
first,
nodes?
PCN
on
ingress
receives a
at
makes
knowledge
could
reservation
to
egress
load
to
The
reasons
load
added
some
at
parameters
isn't
you
algo.
design,
randomly
the
could
the
authentication
read
of

---

From: <[email-masked]>
Date: Tue, 7 Aug 2007 09:37:59 +0100

Bob, all

Re your comment
> Subject: How to discuss WG scoping decisions in an RFC?

There was no response, I think, on this point of your email. My plan is
to be lazy and ignore this comment until consensus that something should
be done! However, if others agree with bob please shout and I'll try and
handle it in the next update (ie after the one that'll be out later this
week). If you shout, please could you point to an rfc that handles this
successfully (ie talks about WG scope restrictions, how they might might
be loosened after re-chartering...) so I get an idea what to do!

Thanks,
phil
draft,
options
aims
its
Architecture
required or
choices
its
autonomous
have
authors
than
field.
that
domain(s)
DSCP
field)
configured-
to
configured-
the
reference
the
These
the
to
seen
below
than a
the
marking
when
work.
marking
marking
that
from
define
explanation
future
think
Adm
admission
might
it
Should
which
the
first,
nodes?
PCN
on
ingress
receives a
at
makes
knowledge
could
reservation
to
egress
load
to
The
reasons
load
added
some
at
parameters
isn't
you
algo.
design,
randomly
the
could
the
authentication
read
of

---

From: <[email-masked]>
Date: Wed, 8 Aug 2007 23:08:05 +0100

Bob,
Thanks for all the comments - tried to deal with them all. In-line
summarises what I did with the more minor[?] ones. 

Yes please on your offers of help:
[1] writing the OAM section
[2] supplying text about DoS for security considerations section ["The
statement about DoS attacks is a requirement with no pointers to the
detailed DOS issues, or any discussion of best practice design"]

thanks!
phil

draft,
options
aims
its
Architecture
> Subject: How to discuss WG scoping decisions in an RFC?

There was no response, I think, on this point of your email. So I've
been lazy and ignored this comment until consensus that something should
be done!

required or
choices
> have to be made by the IETF, which should be left to vendors and which

I think this is a fair point, however probably need to discuss specific
examples on the list. 

> >evolved (compare Section 1.1 of draft-chan-pcn-problem-statement-01)

I didn't move any of this text, or the Intro text in
draft-chan-pcn-encoding-comparison-00 (my understanding is that the
Intro text in the latter draft would basically vanish)

its
> path, each being a separate admission control hop.

This is mentioned in the Deployment scenario Section - which I move into
the INtro. Hoping this solves it. 
autonomous
have
> identical configuration.

This is kind of covered in the Deployment scenario Section. If you think
it isn't enough, please supply 'copy and paste' text (I don't feel
confident that I'll get it exactly right with ref to RFC2475 definition
of DS domain.)

authors
than
field.
> I don't think the authors meant either of these implications. 

Deleted such implications. 

> So somewhere,
that
domain(s)
DSCP
field)
> and a PCN DSCP in the outer.

Separate mail thread on this. 

configured-
to
configured-
the
reference
the
These
the
to
seen
below
than a
the
marking
when
work.
marking
marking
> algorithm that was most studied and simulated in CL-PHB draft.

See separate email thread. 

that
from
> path of PCN-interior-nodes to a PCN-egress-node"

I'm not sure this is accurate for either of the main categories of
marking algorithm (threshold marking [=step marking of cl-architecture],
or excess-rate-marking [where a mark means so many bits in excess]). (it
is true for ramp-marking of cl-architecture.)

define
> IEEE Journal on Selected Areas in Communications 13 (7) pp. 1176--1188

Added
explanation
future
think
Adm
admission
> control if flows aren't inelastic.

Whilst I personally agree, I'm not sure there's general agreement on
which assumptions are the most relaxable. If other people would like
text added on the lines bob suggests please shout (with specific
suggestions), otherwise I'll leave as is.

might
it
Should
which
the
first,
nodes?
> Or does this just make it unnecessarily incomprehensible?

[think your comment applies to S5, Detailed functional architecture]
it was intentional. I found it the easiest way to describe it!

PCN
on
ingress
> or both might need to be configured with a central address.

I've added a new Section 5.7 on addressing, see separate thread. 

receives a
> issue that should be generalised into this arch doc.

Security consids section covers hopefully. I didn't find any text on
security/authentication in that draft. 

at
makes
> >the admission decision; etc.")

In S5.4 where I listed various possibilities for how the measurement
/decision making functionality (for adm ctrl) is distributed
(egress/ingress/centralised node), you made the comment (in your PDF
comments) " Should we say how an implementation should know which option
is being used? Negotiation? Configuration? The IETF standardise only
one?"
I've added "(we assume the operator would configure which is used)"
Ok?

knowledge
could
reservation
> request arrives, so it can unilaterally start probling.

Added.
I actually re-wrote the probing section a bit. See separate thread. 

to
egress
load
to
The
reasons
load
added
> Whatever, I think a feedback function is missing between the boundary

Have clarified this section - re-wrote quite a bit. 
> I get the feeling we need a doc specifically on ECMP solutions?

I think there was some discussion on the list that a separate isn't
needed at the moment. I wrote some revised text on ecmp - separate
thread. 
> Just a thought - I felt this section would have been better earlier.

Moved to the end of the Intro.
some
at
parameters
isn't
you
algo.
design,
> which is security the considerations section.

Thanks for agreeing to (re)write the OAM section!!
> * Security Considerations Gaps:
I created the section by moving the old 'security oam' text, adding your
points below plus others comments. 

randomly
the
could
> supply text here (among many other things to do!)
yes please!

the
authentication
read

of

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 10 Aug 2007 18:42:34 +0100

Phil,

>is true for ramp-marking of cl-architecture.)

1/ threshold marking

As load builds, assuming traffic isn't pure CBR, a queue will very likley 
be oscillating up and down quite fast. Here's a picture at packet 
granularity that I did about a year ago to illustrate this:
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/ipe2eqos/gqs/papers/images/dmbac_queue.pdf>

Even if you have a step marking threshold at some queue length (any 
positive horizontal line across the top graph), only a proportion of packet 
will be marked.

I believe Anna's simulations of step vs ramp demonstrated they weren't that 
different, which is what I would expect. A ramp merely ensures there is a 
smooth transition from zero marking to full marking even if the traffic 
doesn't provide any randomness itself (pure CBR) or its variable but highly 
smoothed by statistical aggregation.

2/ Excess rate marking

Surely the whole point here is to mark a proportion of packets to signify 
the amount of excess rate. So I don't quite understand how it will either 
be 0% or 100% marking? Am I missing something?

Bob

____________________________________________________________________________
Bob Briscoe, <[email-masked]>      Networks Research Centre, BT Research
B54/77 Adastral Park,Martlesham Heath,Ipswich,IP5 3RE,UK.    +44 1473 645196

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 10 Aug 2007 19:03:14 +0100

Phil,

>security/authentication in that draft.

Some time after I said this, I went and looked myself and found it wasn't 
there. Sorry about this.

When I've looked at what you've done, if I can add any insight, I'll try to 
supply text on this when I do the other security txt I promised. I had 
written some stuff on this in an early pre-I-D we never got round to 
publishing (I knew I'd seen it somewhere).

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Fri, 10 Aug 2007 18:57:46 +0100

Phil,

I should have said in the last mail that I'm picking off individual 
responses as new threads as I get time...

>it was intentional. I found it the easiest way to describe it!

Yup sorry, I meant S5. Here's the contents:

    5.  Detailed Functional architecture . . . . . . . . . . . . . . . 12
      5.1.  PCN-interior-node functions  . . . . . . . . . . . . . . . 12
      5.2.  PCN-ingress-node functions . . . . . . . . . . . . . . . . 13
      5.3.  PCN-egress-node functions  . . . . . . . . . . . . . . . . 13
      5.4.  Admission control functions  . . . . . . . . . . . . . . . 14
      5.5.  Probing functions  . . . . . . . . . . . . . . . . . . . . 14
      5.6.  Flow termination functions . . . . . . . . . . . . . . . . 15

It might just be worth saying it isn't actually just a purely functional 
architecture even tho the title says it is (or some such excuse).

What's actually happening here is that the information and control 
necessary for a particular function naturally come together at a particular 
place in the directed topology of the domain. But many protocols are 
designed to carry information from the natural place of control to another 
place that wants to control the system.

In the data plane, traffic flows in one direction, so the natural places 
are not easy to change. But in the signalling plane, you get all these wars 
over where control should be (sender, receiver, PDP, etc), which is what 
signalling protocols are meant to do - allow control to be moved around.

In summary, these issues should be brought out explicitly in an arch doc, 
so something needs to be said about this rather than leaving it implicit.

Bob

---

From: Michael Menth <[email-masked]>
Date: Sat, 11 Aug 2007 08:58:21 +0200

Hi Bob,

Bob Briscoe wrote:
> As load builds, assuming traffic isn't pure CBR, a queue will very 
> likley be oscillating up and down quite fast. Here's a picture at 
> packet granularity that I did about a year ago to illustrate this:
> positive horizontal line across the top graph), only a proportion of 
> packet will be marked.
>
> I believe Anna's simulations of step vs ramp demonstrated they weren't 
> that different, which is what I would expect. A ramp merely ensures 
> there is a smooth transition from zero marking to full marking even if 
> the traffic doesn't provide any randomness itself (pure CBR) or its 
> variable but highly smoothed by statistical aggregation.
Threshold marking marks 0% of packets if the rate is significantly below 
the critical threshold and 100% of the packets if it is significantly 
above this threshold. Therefore, it also guearantees a fast transition 
from the 0% marked packets regime to the 100% marked packets regime when 
the rate increases. Certainly, there is a rate interval in between where 
only some packets are marked due to rate variability, but I hope that 
this does not create severe problems regarding under- or overadmission 
for large traffic aggregations.

>
> 2/ Excess rate marking
>
> Surely the whole point here is to mark a proportion of packets to 
> signify the amount of excess rate. So I don't quite understand how it 
> will either be 0% or 100% marking? Am I missing something?

I guess this applies for threshold marking only. Excess rate marking 
leaves an amount of traffic with the token bucket rate unmarked.

Michael

-- 
Dr. Michael Menth, Assistant Professor
University of Wuerzburg, Institute of Computer Science
Am Hubland, D-97074 Wuerzburg, Germany, room B206
phone: (+49)-931/888-6644, fax: (+49)-931/888-6632
mailto:[email-masked]
http://www3.informatik.uni-wuerzburg.de/research/ngn

---

From: <[email-masked]>
Date: Mon, 20 Aug 2007 08:43:48 +0100

> Sent: 10 August 2007 18:43
> Subject: [PCN] draft-eardley-pcn-architecture-00.txt on-off marking
> 
> Phil,
> 
likley
> granularity that I did about a year ago to illustrate this:
<http://www.cs.ucl.ac.uk/staff/B.Briscoe/projects/ipe2eqos/gqs/papers/im
ag
> es/dmbac_queue.pdf>
> packet

[phil] yes, agree. This is just interpretation of the phrase "signal a
varying fractional number"

> Surely the whole point here is to mark a proportion of packets to
signify

---

From: <[email-masked]>
Date: Mon, 20 Aug 2007 08:44:53 +0100

Bob, thanks will add clarification on lines you suggest
phil

> Sent: 10 August 2007 18:58
> Subject: RE: [PCN] draft-eardley-pcn-architecture-00 function v
topology

---

From: Bob Briscoe <[email-masked]>
Date: Mon, 20 Aug 2007 12:20:10 +0100

Michael,

At 07:58 11/08/2007, Michael Menth wrote:
>>>[BB]> The unintended implication that all packets are either marked or not
>>> > marked
>>>[PE] I'm not sure this is accurate for either of the main categories of
>>[BB] 1/ threshold marking
>>
>>that different, which is what I would expect. A ramp merely ensures there 
>>is a smooth transition from zero marking to full marking even if the 
>>traffic doesn't provide any randomness itself (pure CBR) or its variable 
>>but highly smoothed by statistical aggregation.
>[MM] Threshold marking marks 0% of packets if the rate is significantly 
>below the critical threshold and 100% of the packets if it is 
>significantly above this threshold. Therefore, it also guearantees a fast 
>transition from the 0% marked packets regime to the 100% marked packets 
>regime when the rate increases. Certainly, there is a rate interval in 
>between where only some packets are marked due to rate variability, but I 
>hope that this does not create severe problems regarding under- or 
>overadmission for large traffic aggregations.

[BB] The problem we were trying to solve with ramp marking was the reverse: 
we were looking for a marking algo that produced a smooth increase in 
marking over a wider range of loads. The edge nodes can always turn a 
smooth rise into a toggle at a specific threshold marking level.

A smooth increase would extend the operating range to lower levels of 
aggregation, where the bit rate of one additional flow is not tiny relative 
to all the flows already in an aggregate between two edge nodes. With sharp 
threshold marking, you don't any marking even if the total of all flows 
already in the system are just below the threshold. Even adding some 
probing packets for the next flow may not make enough difference to the bit 
rate to see any marking. But when you add the next flow, marking might 
toggle to 100% (whether this is overadmission or not depends on whether the 
configured admission rate was set lower to allow for such a possibility).

Often, at low aggregation, there's sufficient variability in the flows 
(e.g. silence suppression) to cause the virtual queue to rise and fall 
sufficiently to cross back and forth over the threshold and cause an 
intermediate marking level that could be used to avoid overadmitting the 
next flow. But there is still the pathological case of pure CBR flows. 
That's why we recommended ramp originally (or that's why I thought we 
did!): to produce a smooth (more controllable) transition, even if the 
traffic doesn't have any natural variability.

However, the intention in writing ramp as the canonical algo in the cl-phb 
draft was never to say that ramp should be mandated. It was just one of the 
better tradeoffs if you were only thinking in terms of functionality - if 
you didn't have to worry about implementation and management.

The wider tradeoff is that ramp requires one more parameter to be set and 
therefore one more parameter to be wrong some of the time. And not all 
current hardware can do a programmable marking "curve" anyway.

>>[BB] 2/ Excess rate marking
>>
>[MM] I guess this applies for threshold marking only. Excess rate marking 

[BB] Good - we're agreed on this then.

Cheers

Bob

---

From: "Lachlan Andrew" <[email-masked]>
Date: Thu, 23 Aug 2007 12:28:24 -0700

Greetings,

On 20/08/07, Bob Briscoe <[email-masked]> wrote:
> [BB]
> With sharp
> toggle to 100%

Hear, hear!

I believe that the marking scheme should carry as much information as
possible about the network state, and that thresholding should be done
as late as possible (at the decision-making end-points).  The more
information the decision-making entity has, better the decisions it
can make.

If things like thresholding are done where the admission decisions are
made, there is also a smoother upgrade path for changes in the
admission rules.  For example, an operator may want to apply different
thresholds applied to different types of incoming calls.  If routers
provide a single threshold, this can't be implemented at a later
stage.  Similarly, an operator may want to admit a certain percentage
of traffic dependent on the congestion level, rather than admitting
all or none.

Cheers,
Lachlan

-- 
Lachlan Andrew  Dept of Computer Science, Caltech
1200 E California Blvd, Mail Code 256-80, Pasadena CA 91125, USA
Phone: +1 (626) 395-8820    Fax: +1 (626) 568-3603

---

From: Lars Eggert <[email-masked]>
Date: Fri, 24 Aug 2007 09:04:59 +0300

> information the decision-making entity has, better the decisions it

I completely agree. A marking scheme that attempts to convey the  
maximum possible information about the network state is also much  
more likely to be future-proof, i.e., support uses or extensions of  
PCN that aren't currently on the plate for standardization.

> of traffic dependent on the congestion level, rather than admitting

Lars (with my individual contributor hat on)

---

From: "Geib, Ruediger" <[email-masked]>
Date: Fri, 24 Aug 2007 09:08:42 +0200

Lachlan, Bob,

you both seem to agree that ramp marking is the best solution. 
Picking up Lachlan's pretty sound arguments on different thresholds 
for different types of incoming calls - what you describe is 
estimating the available (not consumed) PCN bandwidth rather than 
just a "pre congestion notification", I think. 
If there's a simple solution to realise this idea, PCN should 
investigate it. This could be helpful if there's a new call to be 
admitted and no PCN flow exists between two gateways. The question 
then would be, whether a certain bandwidth is available, rather 
than "any congestion visible".

Regards,

Ruediger

|Lachlan wrote:
|I believe that the marking scheme should carry as much information as
|possible about the network state, and that thresholding should be done
|as late as possible (at the decision-making end-points).  The more
|information the decision-making entity has, better the decisions it
|can make.
|
|If things like thresholding are done where the admission decisions are
|made, there is also a smoother upgrade path for changes in the
|admission rules.  For example, an operator may want to apply different
|thresholds applied to different types of incoming calls.  If routers
|provide a single threshold, this can't be implemented at a later
|stage.  Similarly, an operator may want to admit a certain percentage
|of traffic dependent on the congestion level, rather than admitting
|all or none.

|Bob wrote:
|[BB] The problem we were trying to solve with ramp marking was the 
|reverse: 
|we were looking for a marking algo that produced a smooth increase in 
|marking over a wider range of loads. The edge nodes can always turn a 
|smooth rise into a toggle at a specific threshold marking level.

---

From: "Jozef Babiarz" <[email-masked]>
Date: Mon, 27 Aug 2007 10:17:08 -0400

I think there is a bit more to this issue than just different levels of
pre-congestion.

For admission control the challenge for PCN is to find out how much new
traffic can be added on to a specific link or path and not to determine
the level of congestion. The purpose of admission control is to prevent
congestion of the service class on link that forwards PCN traffic. PCN
is deployed in Diffserv networks, with the understanding that PCN
traffic is forwarded using priority scheduling.

Voice with silence suppression plus calls naturally ending as well new
calls being added generates highly variable traffic. The marking by
interior node to indicate that PCN admission threshold is exceeded or
not, needs to be quick to reflect true traffic levels. Analysis of the
marking density and control of response to the marking can be handled at
the edges. The proposed marking scheme in draft-babiarz-pcn-3sm-00.txt
for admission control, produces the following behavior, when PCN traffic
is fare below the admission stop threshold, no packets are marked. As
the variable traffic increases and some packets exceed the token bucket
rate (admission stop threshold) they are marked. If additional new
traffic is allowed to be admitted on to the link, more packets will
exceed the token bucket rate therefore marking density increases
possibly until all packets are marked. However, for pure and stable CBR
traffic you will get a step marking (on-off) where the transition from
no marking to all packets marked being quick.
 
My view is that PCN should work equally well regardless of the number of
flows in an ingress-egress aggregate from one to many flows, as well
should work without the need for ingress-egress aggregation concept. The
so called on-off marking approach is not that dependent on the rate of
ingress-egress traffic meaning that edge node will be able to determine
if new flow should be admitted based on low bit rate (probe or single
voice flow) or high bit rate flow(s) passing through the pre-congested
link.

Regards, Joe
email:[email-masked]
Telephone:613-763-6098
-----Original Message-----
From: Geib, Ruediger [mailto:[email-masked]] 
Sent: August 24, 2007 3:09 AM
To: [email-masked]; [email-masked]
Cc: [email-masked]; [email-masked]
Subject: RE: [PCN] draft-eardley-pcn-architecture-00.txt on-off marking

Lachlan, Bob,

If there's a simple solution to realize this idea, PCN should 

Regards,

Ruediger

|Lachlan wrote:

|Bob wrote:

---

From: Bob Briscoe <[email-masked]>
Date: Wed, 29 Aug 2007 09:55:18 +0100

Ruediger,

I was preparing an answer to both you & Joe yesterday, but the one to Joe 
needs to be a bit more involved, so below is my response to your point, and 
I'll get back to Joe next...

Pre-congestion notification is a form of estimating available bandwidth for 
the Diffserv class - they aren't mutually exclusive. I'll explain using an 
example of three aggregates crossing a link:

                   a) option | b) option
____________________________|______________________________________________
load from         available | percent PCN marking  share of
ingress           capacity  | PCN     threshold    available
                             |         (constant)   capacity
                             |
3.0M __        __ 495k      | 6%      15%          (15%-6%)x3.0M = 270k
        \      /             |
2.0M ---+----+--- 495k      | 6%      15%          (15%-6%)x2.0M = 180k
      __/      \__           |
500k              495k      | 6%      15%          (15%-6%)x500k =  45k
                                                                    ____
                                         Total available capacity = 495k

You get the same answer both ways but neither is meaningful. The question 
is, which is better:
a) telling all edge nodes 495k is available (even tho no more than one can 
use any part of it at a time)? or
b) telling all edge nodes they can each send 9% (=15%-6%) more than they 
are currently sending (when they are all unlikely to do so at the same time 
but they might)?

In both cases, no single edge node knows how much extra traffic it can send 
because it doesn't know how many other nodes might be about to do the same:
a) each node should not add 495k in case others do too
b) each node will probably think it can add more than its share of 
available capacity, because not all other nodes are likely to in the next 
round trip time.

In practice there may be anything from 0 to 10,000s of aggregates crossing 
any specific link, but the aggregation assumption is that a link not 
showing any marking can't be driven into overload by addition of a single flow.

Imagine now a path between two edge nodes that crosses one link shared by 2 
aggregates (a possible figure in BT's core) and another shared by 1000 
aggregates (a typical figure across BT's core). How meaningful would it be 
for the second link to say to 1000 aggregates that 20M is available? Or for 
the first link to say to both aggregates that 500k is available? How 
meaningful would it be for either link to say there is 1% pre-congestion? 
In any of these cases, the numbers are fairly meaningless because the edge 
nodes don't know how many other edge nodes they are up against.

This uncertainty is a design 'feature' of a distributed solution like PCN - 
because it's hard to know which situation you're in without a system-wide 
view (and we eventually want system-wide to mean Internet-wide):
- am I the only one asking?
- am I part of a flash crowd?
- am I part of normal flow arrivals and departures?
- what is normal anyway?

So, for admission control, I think the important thing about the marking is 
its shape wrt load. It should rise gradually then get steeper as the 
reference rate is approached and rapidly saturate at some point (the 
reference rate). Then edge nodes can set their thresholds somewhere between 
the two, so they trigger admission control close to the reference rate, but 
sufficiently under the reference rate to give contingency if they happen to 
be part of a flash crowd.

I also believe it is useful for the marking to be dimensionless (ie a 
fraction rather than meaning a certain rate). Then the system scales into 
the future as rates increase by orders of magnitude.

Bob

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 30 Aug 2007 20:41:16 +0100

Joe,

At 15:17 27/08/2007, Jozef Babiarz wrote:
>the level of congestion.

I agree that the AC challenge for PCN is to find how much new traffic can 
be added, but surely not just for a specific link. The full challenge is 
for any single ingress edge node to know how much it can add. But it has to 
accept the fact that it doesn't know how many other ingress nodes are 
competing to add traffic to the same link. (See previous response to 
Ruediger - although I don't think you're asking for the same thing, but it 
sounds like you are).

more...

>The purpose of admission control is to prevent
>not, needs to be quick to reflect true traffic levels.

Agreed

>Analysis of the

I'm not sure which you you are saying is more desirable:
- the outcome with variable traffic (a gradual transition from 0-100% marking)
- or the outcome for CBR traffic (a step change)?

I think you're saying that a gradual transition is desirable, but that even 
with on-off marking you get a gradual transition as long as there's 
variation in the traffic, which there usually is.

(See later for my view)

>flows in an ingress-egress aggregate from one to many flows,

Agreed.

>as well
>should work without the need for ingress-egress aggregation concept.

If you mean by this, that a new flow shouldn't need to use information from 
existing flows in an aggregate, I agree. That's a performance enhancement, 
not an essential function.

>voice flow) or high bit rate flow(s) passing through the pre-congested

I think I am agreeing with you in everything you say, except on one point 
(if I understand you right). So I'll restate everything I believe, tagged 
with whether we're agreeing or disagreeing, to try to reach closure, as 
we've been going round these issues for some time (pre-PCN w-g).

Agreement?: A gradual transition in interior admission marking is desirable 
as the boundary nodes can always turn this into a threshold or use it in 
innovative ways, to be determined.

Agreement?: Whether the transition is gradual is a serious issue, because a 
distributed measurement-based admission control (DMBAC) system like PCN 
admits traffic one round trip before it knows whether it should have done. 
So in a flash crowd (mass calling event) admissions can overshoot if the 
interior admission marking only flips from 0-100% at the last instant. 
Whereas if the interior nodes give a gradual increase, the boundary nodes 
can use the early warning contingency to be more conservative when necessary.

Agreement: We hope and expect that traffic levels will be variable because 
of silence suppression etc. Then step (on-off) marking on interior nodes 
will be sufficient. It will still lead to a gradual increase in admission 
marking as the token bucket level goes back and forth across the step, 
because it will stay increasingly on the marking side the closer the 
average rate gets to the 'PCN lower rate' ("admission stop threshold" in 
your terms).

Agreement: It would be sufficient to standardise step marking (on-off e.g. 
3sm) as a minimum requirement for admission control.

Agreement: This would leave one case of pure and stable CBR traffic that 
would not give a smooth rise in marking, but would toggle from 0 to 100% on 
the addition of the last flow.

Disagreement?: Stable CBR-like traffic won't necessarily be unusual in 
certain networks. It is certainly possible (tho unlikely) that a particular 
link may be fed by a large proportion of CBR codecs without silence 
suppression (e.g. where a network itself deploys a load of cheap codecs for 
PSTN-emulation). But much more likely are links where the total traffic is 
CBR-like even tho each flow is variable, due to statistical aggregation of 
very many variable flows.

Disagreement?: The algo shouldn't have to rely on the traffic being 
variable to behave in the way we want. Ramp marking (as in the body of 
draft-briscoe-twvwg-cl-phb-03.txt) creates a gradual increase in marking 
even for CBR-like traffic. Then the marking algo gives a more gradual 
transition WHATEVER the traffic (unless it's locally shaped to be absolutly 
pure CBR).

The following ASCII art illustrates this. Even for step marking, the curve 
of marking probability against mean arrival rate is smoother for traffic 
with more variance, but if the traffic doesn't have enough variance, you 
can push the curve smoother by using a ramp algo. The flatter the ramp, the 
smoother the curve.

^_
|p, mean marking probability                           |
|                                                      |
|                                                     !|
|                                                     !|
|                                                     !|
|                                                     ;|
|                                                    ,;|
|                                                    ,;|
|                                                    |;|
|                                                    |;|
|                                    variance        ;;|
|                                of total PCN       / ||
|                             traffic on link,      | ||
|                                           v       , ||
|                             OR             +---  .  ||
|                             slope of ramp  |\    ,  ||
|                                            | \  /   ||
|                                               _/    ||
|                                            __/ \    ||
|                                       ,--^^     \   ||           mean
|                         __,,,,----^^^^           \_/ |        arrival
|    /|    ______._,_=_=_=_____,,,,,,,,,,......----^\  |           rate,
+---/ | /----------------------------------------------+------------->_
      |/                                               X              x
                                              PCN lower rate

Agreement: A gradual increase in marking will sometimes lead to a flow 
being denied that could have been admitted. But that's unavoidable anyway 
if some flows are variable - we even had that trade-off to an extent even 
in Intserv-style admission control. However, ramp marking makes the range 
of rates over which you get under-admission wider than it was with just 
natural traffic variability.

Agreement?:
* For a link with a large number of flows with high statistical 
multiplexing, the benefit of artifically creating a gradual increase using 
ramp marking outweighs the cost. You get early warning of a flash crowd but 
miss the odd flow unnecessarily (when the system hits its admission limit, 
which should be rare anyway).
* For a link only carrying a small number of flows within its PCN lower 
rate, the cost of ramp marking in under-admissions outweighs its benefit. 
Note, this needen't be a low capacity link - it might be a high capacity 
link with a few large flows.
* For a link in between - with high number of flows but not high enough 
stat mux to smooth to CBR-like, the traffic variance may give enough of a 
gradual increase to handle flash crowds without needing to add more 
smoothness using ramp marking (ie cost of ramp under-admissions outweights 
benefit of more smoothness when you've got enough anyway).

Agreement?: Note: links with a few flows fall outside the PCN charter, but 
that shouldn't stop us prefering solutions that handle such links, all 
other issues being equal.

Agreement?: If I'm right in all this, I believe we should allow and 
encourage (but not mandate) vendors to implement ramp marking on machines 
targeted for PCN interior nodes (ie SHOULD implement ramp). Then ramp is 
available if it's needed, but can be turned off if it isn't.

Agreement?: But we should advise that step marking is the best solution (ie 
SHOULD deploy step) if there's enough variability in the traffic anyway (or 
only a few flows are expected on the link).

Agreement?: But ramp marking is hard to do on much current hardware. So 
it's useful that it's not a MUST. However, if an operator needs ramp 
marking (a risk of flash crowds and links with smoothly aggregated traffic) 
we should caution that inability to deploy ramp is NOT a good reason for 
not deploying it.

Disagreement: You imply that, with low packet probing rates, step (on-off) 
marking can determine whether to admit a new flow faster than ramp. Given 
both have a range over which they mark packets probabilistically, I don't 
believe this is the way to state the distinction between them. Certainly 
ramp marks packets probabilistically over a wider range of loads. But, for 
either algo, the ingress boundary node doesn't know whether an interior 
node is within this range or not when it probes. So it has to send as many 
probes in both cases in case an interior node might be in its probabilistic 
range of marking.

Disagreement?: In fact, ramp actually gives a much quicker probing time 
than step:
* Step: If the queue is above the step 5% of the time due to traffic 
variation, over a *long* time period it will indeed mark 5% of probes. BUT 
over a *short* period it will mark all the probes the same (queue length 
correlation over time).
* Ramp: If the queue is varying around the point on the ramp that is doing 
5% marking, it will mark each probe with ~5% probability, independently of 
how it marked previous ones.
So, with either algo you have to send the same number of probes, but with 
ramp you can send them back-to-back, while with step you get hardly any 
more information unless you send them spaced over a long time period (ie a 
few seconds). Hence ramp should be much quicker to probe.

Pls say whether you agree or disagree with each point, so we can home in on 
exactly why we keep both going round this loop.

Cheers

Bob

>than "any congestion visible".

ToDo:

* an edge ingress node needs to use its own knowledge of its own rate 
contribution to the problem (x).
0 <= p <= 1

Bob