Subject: [Ieprep] any commercial usage for "emergency QoS"?

From: Rohan Mahy <[email-masked]>
Date: Tue, 26 Nov 2002 13:18:22 -0800

Hi,

<soapbox>
One goal of IEPREP is to make applications used by emergency workers 
more likely to work in the event of an emergency. I am assuming that 
overall the group thinks disabling spam filters is out of scope  ;-). 
The core of the Internet would not be made faster at any time 
(especially during an emergency) by adding QoS there. In addition, most 
networks do not require QoS during ordinary usage, as their edges 
already work quite well by giving users their fair share of an ample 
amount of bandwidth.  During emergencies, there may be network outages 
and/or network congestion--specifically IP network congestion at access 
aggregation points such as DSLAMs, Cable Head End Routers, and Edge 
Aggregation Routers. If some very simple QoS scheme was applied at 
these points, this would improve the probability that applications with 
specific flows would continue to work well even under extraordinary 
congestion situations. For ISPs to deploy this type of QoS however, 
they need a financial incentive. Historically, demanding ISPs to deploy 
new services (such as QoS) by force or by moral suasion are unlikely to 
succeed. Therefore, a good way to test if a service is deployable is to 
see if there are multiple classes of consumer who might request that 
service.  In the case of emergency deployed QoS, certain governments 
may request such a service, but also possibly insurance companies, the 
press, and relief organizations.

In addition, any successful approach will probably start small and 
simple. If it is incrementally useful to get emergency QoS just at the 
first aggregation router from a handful of popular ISPs at the 
granularity of one week for one IP address or subnet, that is probably 
a good place to start.  That is much much much more likely to get 
deployed than per flow QoS end to end.
</soapbox>

I think it might be useful to ask ISPs how much they would have to 
charge (or how much it would cost them) to turn on something very 
simple at their aggregation routers only. You might be surprised by how 
much it will cost (a lot), and how far toward your goal that will 
probably get you (quite a long way).  I envision some large insurance 
company putting a few million dollars into an escrow account and paying 
a fixed price per IP address per week of usage, or prepaying every 
month or year with some "free" incidents each year and covering extra 
usage as needed (the cell phone model).

I also think it would be useful to evaluate the various qos tools in 
our rather dusty arsenal for *ease of implementation*. To put another 
way, considering one of Ran's messages from Nov 15th (snipped below), 
we probably want to only consider approach (B) where the actual 
mechanism the ISP would use to do this is already implemented in 
frequently deployed routers.

> Normally, (A) commercial customers tell equipment vendors that they 
> will purchase
> X million dollars of stuff if the vendor implements foo.  The vendor 
> figures out
> the cost (including opportunity cost) of implementing foo, compares 
> with revenue
> and ROI needs, then decides whether to implement foo for the customer.
>
> Normally, (B) commercial customers tell ISPs that they want to 
> purchase the
> foo service and ask what price it is available at.  The ISP then 
> decides
> whether they want to offer foo (or not) and provides a quote (iff they
> decide to offer foo).  @Home declined to offer a lot of the services 
> that
> we were asked to provide, normally because the opportunity cost was 
> too high.

thanks,
-rohan

_______________________________________________
Ieprep mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ieprep

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 16:47:59 -0500

On Tuesday, Nov 26, 2002, at 16:18 America/Montreal, Rohan Mahy wrote:
> simple at their aggregation routers only. You might be surprised by 
> how much it will cost (a lot), and how far toward your goal that will 
> company putting a few million dollars into an escrow account and 
> paying a fixed price per IP address per week of usage, or prepaying 
> every month or year with some "free" incidents each year and covering 
> extra usage as needed (the cell phone model).

Did you see the presentations in Atlanta ?

 From them, one comes away with the understanding that Genuity and 
Sprint permit
class-based-queuing to be enabled on access routers at the request of 
the access
customer.  I imagine their existing pricing for this is public data.

Note well that IP ToS bits are likely to be zero'd whilst transiting a 
commercial
backbone, so marking packets' ToS bits is unlikely to be a widely 
deployable
approach.

>> we were asked to provide, normally because the opportunity cost was 

Obviously I still take that posture.  However, I see that as a purely 
contractual
matter between USG and its (potential) suppliers -- not as an IETF 
matter.
It is a bad idea for IETF to get involved in business contractual 
issues like
this.

Ran

---

From: "Joel M. Halpern" <[email-masked]>
Date: Tue, 26 Nov 2002 16:59:58 -0500

I suspect I am misunderstanding the point you are getting at.  To the 
degree that we are talking about stuff in already deployed routers (which I 
think we are, and you suggest at the end of your note) there is nothing 
that IEPREP needs to do.

There are two different cases which need to be addressed in quite different 
ways:

1) The ability to enable Differentiated traffic treatment on an access link 
to a customer who wants it.  This is a well defined service.  It can be 
delivered with current technologies.  The user needs to negotiate with the 
access connected service provider, and may need some tech support from the 
CPE vendor (for directly purchased CPE).  Unless one can show a need for a 
new kind of treatment, we are done.

2) The ability to enable Differentiated traffic treatment on an arbitrary 
link at the time of an emergency.  While I can understand this 
conceptually, I can not imagine how it would be rolled out.  There is no 
contract between the emergency workers and the service provider controlling 
the access link.  There is no practical way to fund in advance the 
provisioning capabilities at every mom and pop ISP, wireless access 
provider, DSL provider, ... to provide this kind of capability.

I am missing how one could "start small".  There is no small set of folks 
who have access links that will happen to be in the vicinity of an 
emergency and probably not damaged by that emergency.  Someone probably has 
such links, but predicting who appears to be mathematically and financially 
intractable.

Note that all of this is about network level issues.  There may well be 
some useful application level enhancements for ubiquitous emergency services.

Yours,
Joel M. Halpern

At 01:18 PM 11/26/2002 -0800, Rohan Mahy wrote:
>Hi,
>
><soapbox>
>One goal of IEPREP is to make applications used by emergency workers more 
>likely to work in the event of an emergency. I am assuming that overall 
>the group thinks disabling spam filters is out of scope  ;-). The core of 
>the Internet would not be made faster at any time (especially during an 
>emergency) by adding QoS there. In addition, most networks do not require 
>QoS during ordinary usage, as their edges already work quite well by 
>giving users their fair share of an ample amount of bandwidth.  During 
>emergencies, there may be network outages and/or network 
>congestion--specifically IP network congestion at access aggregation 
>points such as DSLAMs, Cable Head End Routers, and Edge Aggregation 
>Routers. If some very simple QoS scheme was applied at these points, this 
>would improve the probability that applications with specific flows would 
>continue to work well even under extraordinary congestion situations. For 
>ISPs to deploy this type of QoS however, they need a financial incentive. 
>Historically, demanding ISPs to deploy new services (such as QoS) by force 
>or by moral suasion are unlikely to succeed. Therefore, a good way to test 
>if a service is deployable is to see if there are multiple classes of 
>consumer who might request that service.  In the case of emergency 
>deployed QoS, certain governments may request such a service, but also 
>possibly insurance companies, the press, and relief organizations.
>
>In addition, any successful approach will probably start small and simple. 
>If it is incrementally useful to get emergency QoS just at the first 
>aggregation router from a handful of popular ISPs at the granularity of 
>one week for one IP address or subnet, that is probably a good place to 
>start.  That is much much much more likely to get deployed than per flow 
>QoS end to end.
></soapbox>
>
>I think it might be useful to ask ISPs how much they would have to charge 
>(or how much it would cost them) to turn on something very simple at their 
>aggregation routers only. You might be surprised by how much it will cost 
>(a lot), and how far toward your goal that will probably get you (quite a 
>long way).  I envision some large insurance company putting a few million 
>dollars into an escrow account and paying a fixed price per IP address per 
>week of usage, or prepaying every month or year with some "free" incidents 
>each year and covering extra usage as needed (the cell phone model).
>
>I also think it would be useful to evaluate the various qos tools in our 
>rather dusty arsenal for *ease of implementation*. To put another way, 
>considering one of Ran's messages from Nov 15th (snipped below), we 
>probably want to only consider approach (B) where the actual mechanism the 
>ISP would use to do this is already implemented in frequently deployed routers.
>
>>Normally, (A) commercial customers tell equipment vendors that they will 
>>purchase
>>the cost (including opportunity cost) of implementing foo, compares with 
>>revenue
>>Normally, (B) commercial customers tell ISPs that they want to purchase the
>>foo service and ask what price it is available at.  The ISP then decides
>>decide to offer foo).  @Home declined to offer a lot of the services that
>>we were asked to provide, normally because the opportunity cost was too high.
>
>thanks,
>-rohan
>

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 17:20:10 -0500

On Tuesday, Nov 26, 2002, at 16:59 America/Montreal, Joel M. Halpern 
wrote:
> Note that all of this is about network level issues.  There may well 
> be some useful application level enhancements for ubiquitous emergency 
> services.

Everything in Joel's note makes sense to me.  The last bit also applies 
to me
and I should mention it more often.  I'm pretty indifferent about what 
useless
gorp folks choose to put into SIP or other applications -- provided 
there is not
an increased in implemented/deployed gorp at the layers below the 
Application Layer.

Ran

---

From: Rohan Mahy <[email-masked]>
Date: Tue, 26 Nov 2002 14:26:30 -0800

> Did you see the presentations in Atlanta ?

some of them, I had a conflict with SIMPLE.

> customer.  I imagine their existing pricing for this is public data.

that's not an opportunistic service.  the kind of service i am thinking 
of would be more like an emergency worker walking into a random 
internet cafe, convincing the manager to help him or her, and then 
somehow telling SBC, "PG&E will give you $10,000 if you configure your 
DSLAM nearest San Wherever, California to drop less traffic from ip 
address x.x.x.x for one week."  (for the curious, in california, SBC 
and PG&E offer residential DSL service and natural gas and electricity 
respectively).

i don't care if this happens because of a cellphone call to the SBC 
NOC, or because a few packets where exchanged.  however if a few 
packets are exchanged, we will end up with better interop (someone can 
contract with more providers more easily if they all use the same 
mechanism) if we decide what those packets are.

> backbone, so marking packets' ToS bits is unlikely to be a widely 

agree that setting ToS bits and doing nothing else isn't useful. are 
you familiar with the RSVP DCLASS object?  (ask for RSVP but instead 
get told what diffserv you can keep without getting re-marked) I'm not 
proposing that as the answer, but it seems like a reasonable candidate 
to consider.

> It is a bad idea for IETF to get involved in business contractual 

agreed 100%

thx,
-r

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 17:36:30 -0500

> proposing that as the answer, but it seems like a reasonable candidate 

Not deployable.  Issues with authentication.  Also issues with 
inter-domain
traffic.  Also issues with too much state inside a deployed router.  
Also
issues with relying on QoS over a path that changes with time (due to 
dynamic
routing).

Ran

---

From: Fred Baker <[email-masked]>
Date: Tue, 26 Nov 2002 14:43:20 -0800

At 04:47 PM 11/26/2002 -0500, RJ Atkinson wrote:
> From them, one comes away with the understanding that Genuity and Sprint 
> permit class-based-queuing to be enabled on access routers at the request 
> of the access customer.  I imagine their existing pricing for this is 
> public data.

As far as I know, that is correct. AT&T and BT both also have had publicly 
offered services of that type for several years. AT&T and BT, BTW, appear 
to be the principal contributors pushing the efforts in ITU to normalize IP 
QoS between service providers, as discussed in

     ftp://ftpeng.cisco.com/fred/ieprep/IP_QOS_Summary_Nov_2002.pdf
     ftp://ftpeng.cisco.com/fred/ieprep/IP_QOS_Summary_Nov_2002.ppt

(use the ppt if you support powerpoint; the pdf is Huge)

Which is to say, I think, that while it is quite correct that US service 
providers tend to use overprovisioning in their cores, it is quite 
incorrect to say that no ISP uses QoS facilities, that QoS facilities 
should be deprecated, or that no ISP is interested in end to end QoS 
guarantees in IP. There are some service providers that would appear to 
want to rebuild ATM's service classes in IP.

>commercial backbone, so marking packets' ToS bits is unlikely to be a 
>widely deployable approach.

So they not only don't choose to use them, they actively break other 
people's use of them. Now *that's* Clueful. That is really demonstrating an 
interest in providing robust services to their clients. Lovely.

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 18:14:47 -0500

On Tuesday, Nov 26, 2002, at 17:43 America/Montreal, Fred Baker wrote:
> Which is to say, I think, that while it is quite correct that US 
> service providers tend to use overprovisioning in their cores,

It is broader than that.  Most Asia/Pacific ISPs also over-provision.  
And
smd (who is Irish working in London) and roll (who is Swedish) have 
asserted
in the IEprep meeting that European ISPs and multi-continent ISPs also
routinely over-provision.

> it is quite incorrect to say that no ISP uses QoS facilities, that QoS 
> facilities should be deprecated, or that no ISP is interested in end 
> to end QoS guarantees in IP. There are some service providers that 
> would appear to want to rebuild ATM's service classes in IP.

Please suggest that those ISPs show up here so their views are 
presented directly.
Broader operator participation would have to be a good thing for IETF.

>> Note well that IP ToS bits are likely to be zero'd whilst transiting 
>> a commercial backbone, so marking packets' ToS bits is unlikely to be 
>> a widely deployable approach.
>
> people's use of them. Now *that's* Clueful. That is really 
> demonstrating an interest in providing robust services to their 
> clients. Lovely.

No, they are demonstrating an interest in protecting their own network 
from trivially
implemented DDOS attacks (as has been discussed recently on other lists 
that we are both
on) on the IP ToS bits.  This was explained quite nicely by Dave Meyer 
and others in
Atlanta in IEprep.  The issues also came up during the IEPG meeting 
before IETF in
Atlanta.  It seems like quite prudent system engineering to me and many 
other folks
present in Atlanta.

Ran
[email-masked]

---

From: David Meyer <[email-masked]>
Date: Tue, 26 Nov 2002 15:41:22 -0800

Fred,

>> > service providers tend to use overprovisioning in their cores,

	My experience is that anyone who can (i.e., who isn't
	capacity constrained) would approach the problem in this
	way. BTW, a point on terminology: The approach many
	carriers are taking (what you and many others are calling
	"over-provisioning") is no more over-provisioning than
	say, the SONET network was/is; in fact, very much less so
	(and even less so when you consider the fiber occupancy
	on something like 4 fiber BLSR). So if you're in the
	business of delivering all packets to all customers, you
	have to protect the traffic. If you do that, you get the
	QoS properties for "free" (where "free" means "without
	additional mechanism"). That is one of the beautiful
	things about the IP architecture, lest we forget...  

>> Broader operator participation would have to be a good thing for IETF.

	Most certainly. 

>> Atlanta.  It seems like quite prudent system engineering to me and many 

	To summarize the point, unless you want to touch every
	packet (*including* peers), there seems to be no straight
	forward (or any other) way to use the ToS bits in a way
	that can't be trivially DoS'ed. The crucial point here is
	that mandating the need to touch arbitrary parts of every
	packet will put fundamental limitations on the scalability
	of the network (along several dimensions), and I think we
	can all agree limiting our scalability isn't something
	that we want to do.

	Dave

---

From: Fred Baker <[email-masked]>
Date: Tue, 26 Nov 2002 15:47:36 -0800

At 06:14 PM 11/26/2002 -0500, RJ Atkinson wrote:
>on) on the IP ToS bits.

interesting. If they are ignoring the DSCP/TOS Flags, how would the 
settings of the DSCP/TOS Flags result in a DOS attack?

Pick one: they are ignoring the DSCP/TOS Flags, so that all IP traffic is 
equal, or they are not. Which is it?

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 19:59:01 -0500

On Tuesday, Nov 26, 2002, at 18:47 America/Montreal, Fred Baker wrote:

>> No, they are demonstrating an interest in protecting their own 
>> network from trivially
>> implemented DDOS attacks (as has been discussed recently on other 
>> lists that we are both
>> on) on the IP ToS bits.
>

Some ISP folks said that they use ToS bits internally to mark their own 
iBGP/IS-IS/OSPF/SNMP
packets and clear all ToS bits on all incoming packets at the 
administrative boundaries.
Some routers can provide routing protocol performance optimisations by 
having the
routing protocol packets appropriately marked.  I'm uninformed whether 
your routers
are among those.  I believe this was mentioned during IEprep, though 
maybe it was
during IEPG instead.

Ran

---

From: Bill Woodcock <[email-masked]>
Date: Tue, 26 Nov 2002 17:50:44 -0800 (PST)

> equal, or they are not. Which is it?

Both.  The Internet isn't monolithic.  Currently, the best practice is to
zero at the borders, which means that anyone following the best practice
is vulnerable only to attacks from their own customer base, if at all,
and they can use TOS bits inside their own infrastructure.  Everyone
who's not following the best practice can't use TOS bits because they're
being set from the outside.

If some sort of requirement were imposed which mandated inter-provider
carriage of un-zeroed TOS flags, that would be requiring everyone to
ignore the best-practice, and would make it unsafe for anyone to pay
attention to the TOS bits for anything.  It would also find us a lot of
bugs in a lot of people's router code, the painful way.  So that would be
completely self-defeating.

                                -Bill

---

From: Scott Bradner <[email-masked]>
Date: Tue, 26 Nov 2002 20:59:23 -0500 (EST)

> Not deployable.  Issues with authentication.  

I'm told that Windows includes DCLASS 
the assumption is that a *local* router tells the host what
diffserv DSCP to put on an application's packets

> Also issues with too much state inside a deployed router.

I would think this depends on what its being used for - apparently there 
is now enough experience with RSVP to signal MPLS to know that
it can scale to at least small # (100s anyway) of sessions -
if rsvp wer eto be used with the DSCP as the flowspec it might
scale well enough

btw - it would be good to get some *real* idea of what the load might be 
for a random router in various parts of an  ISP under what
conditions - for example, how many GETS calls were made in the NYC
area around sep 11? (and how many at the same time?)

that does not mean that the authentication and inter-ISP issues
are not signiicant issues

Scott

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 21:33:35 -0500

Bill,

	By the way, during the Nairobi VoIP demo in Atlanta, was there any use
of IP ToS bits, DiffServ, or other QoS ?

Ran

---

From: Bill Woodcock <[email-masked]>
Date: Tue, 26 Nov 2002 18:31:34 -0800 (PST)

> By the way, during the Nairobi VoIP demo in Atlanta, was there any
    > use of IP ToS bits, DiffServ, or other QoS ?

Nope.  We wanted it to work.  Demos which fail aren't as impressive.

                                -Bill

---

From: Bill Woodcock <[email-masked]>
Date: Tue, 26 Nov 2002 18:30:06 -0800 (PST)

On Tue, 26 Nov 2002, Rohan Mahy wrote:
    > much it will cost (a lot)...

Anyone who goes to the trouble to ask that question subsequently draws the
inescapable conclusion that spending less money to buy more bandwidth
solves the problem better.  With luck the guys with funny hats and guns
will ask the question soon, so we can get back to work.

                                -Bill

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 21:45:53 -0500

On Tuesday, Nov 26, 2002, at 20:59 America/Montreal, Scott Bradner 
> diffserv DSCP to put on an application's packets

Weird.  If the first-hop router is the source of the policy knowledge,
it would be much easier for the router to just modify the IP ToS bits.
A modern router or L3 switch ought to be able to do that rewriting
on every packet while still sustaining full line rate 
forwarding/bridging.
And the first-hop router/L3 switch would need to police the values
in any event.

> if rsvp wer eto be used with the DSCP as the flowspec it might

I'm a bit sceptical, but would be interested to learn more.  I'm unaware
of any commercial ISP with that much (RSVP-based MPLS) in operation in 
their
deployed network.  Is there a URL or paper supporting this claim ?

> area around sep 11? (and how many at the same time?)

Yes.  Having real measured data of how a router performed with RSVP
enabled in such a situation would be helpful.  Of course, one would
also want to subject the router to one of the trivial RSVP-based attacks
at the same time, since routers being under attack in times of stress
is something one should plan for.

> are not signiicant issues

Also, deploying RSVP itself creates new security issues.  Those are
a bit more tractable, since there is an MD5 authentication option
for RSVP, though key management remains its usual nightmare.  And
for an IP network based on commercial products, the sundry alternatives
used only in the military world aren't available to help with KM.

Ran
[email-masked]

---

From: RJ Atkinson <[email-masked]>
Date: Tue, 26 Nov 2002 21:48:38 -0500

On Tuesday, Nov 26, 2002, at 21:31 America/Montreal, Bill Woodcock 

> Nope.  We wanted it to work.  Demos which fail aren't as impressive.

So the demo might be an existance proof that VoIP works fine over
a set of best-effort IP networks across 2-3 continents ?

Do you happen to have a traceroute handy for that to document how
many ASs, hops, and continents that the demo traversed ?  (Feel free
to omit the last few IP addresses since this list is permanently 
archived;
we don't want to accidentally encourage bored folks to attack the
innocent user in Nairobi).

Ran

---

From: Bill Woodcock <[email-masked]>
Date: Tue, 26 Nov 2002 18:49:57 -0800 (PST)

> many ASs, hops, and continents that the demo traversed ?

We had the slide (which I'll put up permanently) showing how many hops,
AS path length, loss, and latency across a full-mesh of myself, Scott
Bradner, Patrik Faltstrom, and Brian Longwe.  We also have a traceroute
(attached) from the day before, showing the path from Nairobi to the
hotel.  Both times we were getting variable loss, up to about 40%.
Most of those outward-facing IPs seem to be ICMP-blocked, so that's a
little hard to replicate at the moment.

                                -Bill

  1     6 ms     3 ms     3 ms  192.168.14.1
  2    13 ms    15 ms    16 ms  64.86.232.121
  3    41 ms    54 ms    30 ms  64.86.231.9
  4    27 ms    70 ms    48 ms  64.86.231.1
  5   622 ms   636 ms   619 ms  212.49.70.105
  6   641 ms   618 ms   620 ms  216.129.151.150
  7   619 ms   593 ms   642 ms  216.129.151.149
  8   621 ms   627 ms   634 ms  216.129.150.249
  9   635 ms   613 ms   602 ms  66.133.0.45
 10   614 ms   602 ms   696 ms  209.244.160.241
 11   612 ms   645 ms   612 ms  64.159.17.38
 12   639 ms   609 ms   631 ms  209.244.160.150
 13   608 ms   621 ms   660 ms  129.250.3.48
 14   639 ms   620 ms   656 ms  129.250.5.99
 15   648 ms   641 ms   644 ms  129.250.2.180
 16   643 ms   628 ms   643 ms  129.250.2.49
 17   671 ms   636 ms   674 ms  129.250.26.227
 18   633 ms   641 ms   655 ms  157.238.161.202
 19   672 ms   639 ms   646 ms  204.42.70.246

---

From: Fred Baker <[email-masked]>
Date: Tue, 26 Nov 2002 23:34:19 -0800

>it can scale to at least small # (100s anyway) of sessions -

actually, tens of thousands per router. How many edge routers surround an 
ISP? We have some customers that go CPE-CPE.

I'm not an MPLS advocate, but it does actually work...

---

From: Scott Bradner <[email-masked]>
Date: Wed, 27 Nov 2002 07:16:03 -0500 (EST)

henning reported a 7% (if memory serves) loss right after the test

Scott

---

From: Ken Carlberg <[email-masked]>
Date: Wed, 27 Nov 2002 12:23:30 +0000

> little hard to replicate at the moment.

one caveat to keep in mind.  some ISPs place ICMP messages on separate
queues that can experience worse treatment (loss/delay) than other
flows. the motivation being to protect other traffic (eg, TCP)
considered more important, and/or constrain DoS attacks based on
icmp-echo.

hence, traceroutes can be a bit deceptive at times.

and yes, i'll conceded that the tier-1 ISPs in the states don't do this
between their peers :-)

-ken

---

From: RJ Atkinson <[email-masked]>
Date: Wed, 27 Nov 2002 09:10:44 -0500

On Wednesday, Nov 27, 2002, at 07:23 America/Montreal, Ken Carlberg 
> considered more important, and/or constrain DoS attacks based on

Hmm.  I'm not aware of any ISP that has done that.  And traceroute
uses UDP to probe, the ICMP is only used for the return of the
ICMP Port Unreachable from the router.

What is true is that some *CPU-based routers* de-prioritise the
generation of ICMP error messages within the router, as compared
with spending CPU forwarding non-errorred packets, which can sometimes
mean that traceroute results are *slightly* (not dramatically) worse
than normal traffic.

The other thing is that Henning measured *after* the demo, meaning
that any competing TCP/SCTP-based flows probably had time to back off
-- so the packet loss at start of demo was probably a bit higher
than the measured number.

Bottom line is that it all washes out and so the measured packet loss
is probably accurate within some reasonable error bound.

Ran
[email-masked]

---

From: Bill Woodcock <[email-masked]>
Date: Wed, 27 Nov 2002 10:15:39 -0800 (PST)

On Wed, 27 Nov 2002, Ken Carlberg wrote:
    > flows.

That's CAR.  Yes, however it's actually not a separate queue, it's a drop
algorithm applied to the same queue.  So it affects drop rate but not
delay.  And it's only applied over a certain threshold, typically one that
constitutes a DoS attack.  So it's pretty obvious when it's being applied.

    > hence, traceroutes can be a bit deceptive at times.

As Ran pointed out, it doesn't affect the results of a traceroute, though
it may cause some of the return packets to be lost.  The answer that you
get is unaffected.

    > and yes, i'll conceded that the tier-1 ISPs in the states don't do this

Actually, depending upon who you condier a tier-1, I believe that some do.
It's certainly a common practice generally, between peers more than facing
customers, usually.

                                -Bill

---

From: Henning Schulzrinne <[email-masked]>
Date: Wed, 27 Nov 2002 14:17:02 -0500

Let me try a few additional angles on this problem.

- I don't think we need to posit that any mechanism be universal for it 
to be useful. Since this is all about probabilities, any *helpful* 
mechanism is a net gain. (In other words, we should only look at the 
mechanisms that don't require some critical mass to operate.)

- I think it would be helpful if we more clearly distinguished two 
scenarios, namely the "controlled edge" and the "opportunistic edge" 
(since we're talking primarily about modifying edge properties). In the 
'controlled edge', an emergency responder would use a network controlled 
by some (say) government entity. There, lots of things become possible. 
If necessary, one can do congestion control by pulling rank.

The second scenario is the opportunistic edge: an authorized person 
takes over some existing commercial facility or asks for assistance. 
There, at best, the local owner can assist.

- I'm not convinced that a static traffic classifier is particularly 
helpful for edge prioritization needs. I can see three possible types of 
classification:

(P) by protocol; e.g., prioritize UDP (under the assumption that 
time-critical application use it);
(A) by source or destination address;
(N) by port number

or some combination of these. (P) seems too crude and too much subject 
to DDOS attacks, either inbound or outbound. The attacker has to guess 
nothing.

(A) doesn't work for our needs, since the destination of emergency 
communication is generally unknown and can be anywhere in the Internet. 
It does work pretty well for VPNs.

(N) looks more promising, except that it has similar problems to (P). If 
the number of ports is small, this requires extensive coordination. All 
applications would have to be 'trained' to use these ports. If everybody 
uses the same "emergency" port, any modest security-by-obscurity would 
be blown and the port would become an inviting DDOS target.

Thus, I don't think that COS-by-filter is particularly helpful here.

An alternate model might be worth considering: Assume for a moment that 
we have an NSIS protocol that is suitable for edge QoS control. (I 
realize that some have well-founded doubts whether this will ever get 
deployed, but that would be something to discuss with the NSIS working 
group, I presume.) One of the primary scaling problems with existing 
resource control approaches is not the protocol itself but rather the 
AAA backend. Among other things, when the network is under the weather, 
having to go and ask some faraway places for authorization is not 
exactly probability-of-completion-enhancing.

However, fortunately, the problem here is somewhat easier. Authorized 
emergency personnel could easily have a cert signed by the local 
authorities that authorizes them to use the resources, similar to the 
GETS mechanism. The local NSIS entity doesn't have to worry about that 
Officer Smith gets charged vs. Officer Jones nor may be particularly 
concerned whether Officer Smith still has an "account", as long as it 
trusts the issuing authority and certs get issued with some reasonable 
frequency. Thus, this requires only a single CA entry in the NSIS 
router, without the usual complexity. One can forego any CRL checking in 
all likelihood, given that rogue officers can presumably be dealt with 
in other ways after the fact. Since the volume of emergency calls in any 
particular place is likely to be low, the additional PK verification 
delay is not likely to matter much either - it doesn't have to scale to 
every single phone call or web access. (Rohan and I discussed this 
roughly in Atlanta; he may not agree with the details above.)

Using a dynamic filter avoids the DOS problem and should have no 
complexity impact on the rest of Internet.

Thus, if this makes sense, it might be worth asking the NSIS working 
group to consider such requirements, in the spirit of the SIPPING effort.

Henning

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 28 Nov 2002 07:38:07 -0500

One should also add to this demo that it would have supported roughly 5 
people making phone calls simultaneously, if I remember the link 
bandwidth description correctly. (The codec used was G.711, I believe, 
since that's the default Cisco 7960 codec.)

RJ Atkinson wrote:
> 
> On Tuesday, Nov 26, 2002, at 21:31 America/Montreal, Bill Woodcock wrote:
> 
> to omit the last few IP addresses since this list is permanently archived;

---

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 28 Nov 2002 07:39:45 -0500

I did ping tests very shortly after the demo. I measured 7% packet loss 
from the wireless IETF network and from Columbia, averaged over roughly 
200 packets. Sorry to say, even the Cisco phone doesn't sound like this 
with 40% packet loss.

---

From: RJ Atkinson <[email-masked]>
Date: Fri, 29 Nov 2002 17:39:13 -0500

On Wednesday, Nov 27, 2002, at 14:17 America/Montreal, Henning 
Schulzrinne wrote:
> An alternate model might be worth considering: Assume for a moment 
> that we have an NSIS protocol that is suitable for edge QoS control. 
> (I realize that some have well-founded doubts whether this will ever 
> get deployed, but that would be something to discuss with the NSIS 
> working group, I presume.) One of the primary scaling problems with 
> existing resource control approaches is not the protocol itself but 
> rather the AAA backend.

That was not the problem with RSVP deployments and is not the problem 
with RSVP MD5.
The problem with RSVP deployments included at least (1) too much state 
in routers,
(2) paths vary dynamically with normal IP routing which increases 
robustness/resilience
but means that the packets spend a portion of their time travelling 
paths where
no reservations are in place.  The issues with RSVP MD5 centre on the 
computational
expense of computing MD5, the computational expense of key management, 
and the
difficulty of deploying inter-domain scalable dynamic key management.  
Even the DoD
key management systems are single domain -- and they are probably 
better than what
the commercial sector is using.

>  Among other things, when the network is under the weather, having to 
> go and ask some faraway places for authorization is not exactly 
> probability-of-completion-enhancing.

True, but not the main reason why RSVP and NSIS aren't likely to be 
deployable.

> router, without the usual complexity. One can forego any CRL checking 
> in all likelihood, given that rogue officers can presumably be dealt 
> with in other ways after the fact. Since the volume of emergency calls 
> in any particular place is likely to be low, the additional PK 
> verification delay is not likely to matter much either - it doesn't 
> have to scale to every single phone call or web access. (Rohan and I 
> discussed this roughly in Atlanta; he may not agree with the details 
> above.)

How does the cert get to the router exactly ?
What are the details of your plan here (from the viewpoint of the 
router, please) ?

Validating a signature is computationally *expensive*; validating is 
less expensive
than signing if one is using RSA, but still expensive.

> complexity impact on the rest of Internet.

Please do explain more.  I don't understand the details of who gives 
whom a CERT
about what, when, and when whom needs to do what with it.

Thanks,

Ran

---

From: Henning Schulzrinne <[email-masked]>
Date: Sat, 30 Nov 2002 09:20:31 -0500

> The problem with RSVP deployments included at least (1) too much state

This might have been a problem a few years ago, but I'm having a hard 
time believing that a router that can handle on the order of 100,000 
route entries has trouble with, say, a few hundred reservation entries. 
As was pointed out by Fred, there are large MPLS path counts in 
operation today, with no noticeable state problems. (Not counting the 
pain of configuration.)

To be more precise, we have to distinguish three types of states:

(1) scheduling state; this only matters if you have per-flow queueing
(2) filtering/policing state, which polices flows;
(3) reservation state, one for each reservation, but only affecting the 
control processor

 From what I can tell, (3) is not a major issue. Given the hundreds of 
MB on a backbone router and the tens of MB on an edge router and the 
roughly 500 bytes/reservation, you'd need lots of reservations to even 
come close to mattering.

I don't think you need per-flow queueing. For DOS prevention, you'd want 
  filtering, so that only reserved flows get into the "reserved" queue. 
This seems similar to the type of reverse-address filtering and other 
firewall-style filtering that edge routers already do. I don't know if 
any router operates in this mode, but it wouldn't seem difficult to set 
up a series of rule that says "if <match reservation spec i>, set DSCP 
to X" (for i=1 through N) (if none match, set DSCP to 0). Would, say, an 
Extreme box support such filtering rules?

I had earlier cited publications indicating performance achievable on 
commodity computing platforms that are very similar to modern control 
processors. If you have other indications for modern routers, please let 
me know.

> key management systems are single domain -- and they are probably better
> than what

Web servers routinely handle large volumes of TLS connections. I found 
http://www.isoc.org/isoc/conferences/ndss/02/proceedings/papers/coarfa.pdf 
which measures performance on reasonable hardware. As one number, Apache 
on a 900 MHz PIII seems to do about 250 transactions/second, which seems 
more than enough for most reservation needs and far above what I can 
picture for emergency edge reservation needs. (Even if we assume that 
each reservation lasts only 30 seconds and no state is kept across 
refreshes, this would still be about 6000 reservations that you could 
maintain.)

> about what, when, and when whom needs to do what with it.

For concreteness, let's assume that FEMA is in charge of this 
hypothetical program. FEMA sends, in some secure fashion, a cert to each 
operator it cares to (based on whatever business relationships it might 
set up). Each operator installs the CA cert in the control processor, 
just like any other CA root cert. Obviously, this happens during 
non-emergency conditions and can be as simple as a signed S/MIME mail. 
There's no need for secrecy, just integrity, since this is a public key 
only. Alternatively, the operator can just download the .p12 file from 
the FEMA web server. There may even be automated ways of doing this; I 
haven't followed all the recent PKI-related WGs. This operation is 
repeated at cert lifetime intervals; these seem to be roughly yearly.

When a signed resource request comes along that claims to be signed by 
the FEMA CA, the control processor verifies this assertion. If this 
checks out, the holder of the cert (presumably, an authorized emergency 
worker) gets her reservation, if not, the attempt fails. No real-time 
interaction with outside entities is required.

(I've simplified this a bit. In reality, you'd want a slightly more 
expressive authorization certificate, but the basic operation is the same.)

This is no different than any of the standard PK setups that are used 
within closed user communities. (As one example I'm familiar with: A 
large European telecom company has issued each employee a smartcard 
ID-badge that has his or her public key on it and uses this for system 
logins and cafeteria meals. I've seen the same on laptops used by 
Microsoft employees.) Unlike a global PK system for random "civilians", 
this is much easier to deploy - the number of certifying entities is 
very small. Similarly, the number of certified users is relatively 
modest and each of them has a close relationship with the certifying entity.