Subject: Re: hop-by-hop and router alert options [Re: Question about use of RSVP in Production Networks]

From: "John Loughney" <[email-masked]>
Date: Fri, 13 Aug 2004 12:48:11 +0200

> Of course.  Then why this wasn't the first thing NSIS did after going
> for on-path signalling, or didn't I just manage to find it?  

NSIS was specifically charged to by the Transport ADs to work on on-path signaling. There is an analysis document being reviewed by the IESG right now ...

> I really really hope that there has been a problem statement...

Why? I've generally found that problem statement documents have limited usefulness.
> Further point where you can use path-coupled signalling, you mean?  
> Not really, as there seems to be something seriously broken if you
> need set up priorization except in well-defined points in the network.  
> And to achieve that, you could use a "bandwidth broker": either it's
> able to set up the required bandwidth (it's in the same site, or at a
> site your site has a roaming agreement with), or it isn't and the
> approach is going to fail in any case, because the sites out in the
> Internet don't want outsiders to request bandwidth allocations.

As you know from  IPv6 discussions, people's view of what constitutes a site 
is very different. It is not a well defined term. NAT & FW traversal is very 
difficult, especially for signalling incoming connections. We have documentation, if you are interested.

Also, there is no standard bw broker. Hoping that there was doesn't help. Currently deployed bw brokers have scalability problems, especially if they are meant to scale Internet-wide.

> True enough, I had my operator hat on ;-).  My mind just boggled when
> I started to realize that some part of the IETF is still in denial on
> why RSVP didn't go through, and is now in the process of reinventing
> it .. wasting a lot of time and effort that could be much better spent
> on making an operationally more feasible system to provide these
> reservations in a well-defined, constrained points of the network.

Interestingly  enough, there are operators involved in the NSIS discussions. 
Also, some of the protocoö mechanisms of RSVP are suprisingly robust, but not all of the design criteria of RSVP hold today.

> What I can't figure why this is happening.  I guess the IESG must have
> been asleep, or the most people just thought, "well.. let them waste
> their energy on that.. at least they don't bother us while they are
> bashing the heads in the wall.."

Before passing such judgements on the IESG, reading the charter would be recommended. If you find yourself nearby Helsinki, I'd be happy to explain things.

John

_______________________________________________
Ietf mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/ietf

---

From: Pekka Savola <[email-masked]>
Date: Tue, 24 Aug 2004 15:29:00 +0300 (EEST)

[[ I was hoping there would be more follow-ups on this thread, but 
apparently not... ]]

Combining two messages in one:

On Thu, 12 Aug 2004 18:00:50 -0500 "James M. Polk" <[email-masked]>:
> Humor me (and a few others here) with a few listed mistakes you believe
> RSVP made in its design, with how you would redo them (either with a BB or
> just a better RSVP design).

Well, here's a couple -- hope it helps. Correct me if I'm mistaken:

 - RSVP doesn't have "network indication of support": the nodes keep 
spewing RSVP messages to the network even if every router is happily 
ignoring them, and the destination node does not support them.

Possible fix: have the signalling engine contact the local network 
administrator with a protocol to confirm if it supports the 
reservations or not.

 - path-coupled messages require significant processing at the
routers.  This probably needs to happen in "slow path", in software.  
A draft mentioned an implementation being able to support a whopping
(NOT!)  7000 sessions.  That works for the first hop, and possibly
even to a degree inside an enterprise where you can have software
based routers and not too many hosts, but it's not a useful model for
larger routers, deployed at more central places in the network.

Possible fix: instead of on-path processing, contact a node off-path,
which performs all the processing, checks the policies, etc., and
ensures that appropriate QoS metrics are configured at the appropriate
routers (these could be bulk DiffServ guarantees, more specific
reservations and whatever).

 - the policy checks are distributed to all the routers: the real
policy, within an administrative domain, is however made in a
centralized fashion, so it would be useful not to have to distribute
the policy to every router, but rather just process it in one place.

Possible fix: BB model provides this capability.

 - doing anything that crosses administrative domains for bandwidth or
other reservations requires that the operators have incentive to
provide these capabilities to outsiders, and hence the applicability
is restricted to a single administrative domain or the first hop.  
But when you operate inside a single domain, path-coupled signalling
does not seem necessary in the first place, you could just contact an
entity which would be trusted to make the reservations on your behalf.
The only alterntive is if significant amounts of money is transferred
based on the reservations, but that would make the mechanism a
non-starter for the users.

Possible fix: do signalling just inside an administrative domain, and
using a path-decoupled mechanism (whether a direct messaging with the
1st hop router or BB depends on the deployment).

 - the model where hosts or apps can reserve bandwidth or other
characteristics doesn't sit well with the operational models of ISPs.  
Such a reservation by definition eats from the shared resource;  
multiple reservations result in fewer people being use the the
originally shared resource.  On the other hand, the reservations
aren't useful unless sufficiently large about of BW is allocated to
them, and then the rest will suffer.  So, this would only be feasible
as a premium paid service.  But to achieve that, one could set up more
static resource allocations as well, or use other form of signalling.

Clarification needed: is the signalling protocol envisioned to be used
for paid premium service only?  If so, it may have some marginal use,
but I doubt the users are willing to pay for it so it would be
marginal.  If not, the protocol needs to be highly resistant to
users/apps requesting too large reservations (consider a
virus-infected host requesting very large reservations), security
models, etc.

 - if an operator would provide paid "premium" service, it would have
to acknowledge that its regular service is bad.  The competitors just
state that all of their service is equally good (than the premium
service) due to sufficient over-provisioning.  And if your own regular
service is good enough, the customers have no incentive to request
better service and pay you. Hence, a model where the customer has to
pay extra to get quality doesn't seem to cut it in general.

 - the first-hop only case is considerably easier because the user can
just shoot himself (not the others) in the foot.  I.e., it allows the
user to manage *his* traffic preferences on the link, i.e., he cannot
reserve from the bandwidth used by others.

.....

A local administrative domain could allow the local hosts to set up 
reservations, yes.  But in most cases, these networks are already 
over-provisioned except for a few special links (e.g., between the 
geographical sites, or the first hop) -- a well-defined environment 
which wouldn't require path-coupled signalling.

Remote administrative domains (including the ISPs) don't want
"foreign" nodes to request any reservations, unless they get direct
income from allowing that.  A free service just doesn't work, as the
Internet users are greedy by definition.  Best effort service is the
fairest service available under such circumstances.

And if you constrain yourself to just solving the problem for premium,
paid service (so that the users would not have incentive to request
special treatment unless they want to pay big bucks for it), it won't
attract sufficient economic interest (because if it would have, we'd
have RSVP already deployed and used widely already).  The
economic/operational tradeoffs to both the ISPs and the users just
aren't right.

So, I think the main point is that a model which requires payment
doesn't fly economically, and a model which doesn't require payment
doesn't fly operationally or technically because then everyone would
send in reservations and the result would be worse than best effort
because there wouldn't be enough for everyone. The reason why BB model
is better is because it centralizes the processing and policing, and
makes it easier to allow adopt QoS because it's less costly
(considering state, processing, etc.) in the routers than path-coupled
signalling.

On Fri, 13 Aug 2004, John Loughney wrote:
> NSIS was specifically charged to by the Transport ADs to work on
> on-path signaling.

That'll certainly focus the work, but the question is whether that 
focus is useful or not.

> Why? I've generally found that problem statement documents have

That might help in understanding the problems better.  One part of
that is identifying what problems the earlier solutions aimed to
solved and why they did not succeed (at least to certain
expectations).  I saw some analysis in
draft-ietf-nsis-signalling-analysis-04.txt when I quick read it, but
it seemed to be lacking real, operational input about the operational
and deployment problems, and was looking just the issues with the
protocol on its own.

Based on the direction where NSIS seems to be heading, the result is
likely going to be a slightly fancier version of RSVP .. which isn't
going to be terribly useful, except for those few who are already
using RSVP.

> As you know from IPv6 discussions, people's view of what constitutes
> a site is very different. It is not a well defined term. 

I don't see why it would need to be well-defined, all you'd need to do
is ask a "bandwidth broker" at your site, by finding out its address
using one of many option (you don't need to know the actual definition
of the site to do that, or the span of the site).

> NAT & FW traversal is very difficult, especially for signalling
> incoming connections. We have documentation, if you are interested.

Why would the administrator of the firewall trust the user or an
application running on the user's laptop to "behave well", i.e., punch
holes in the firewall?

NAT traversal is more trickier esp. inbound, but we already have a 
large number of solutions aiming to achieve exactly that, among them 
ones which provide non-encumbered IPv6 connectivity through the NATs, 
and then the problem would be fixed automatically.
 
> Also, there is no standard bw broker. Hoping that there was doesn't
> help. Currently deployed bw brokers have scalability problems,
> especially if they are meant to scale Internet-wide.

I don't know if there are ones, but that certainly wouldn't be a
show-stopper if there was a decision that this was the way to go.

All the approaches have scalabililiyu problems, especially if they are 
meant to scale Internet-wise .. especially RSVP-like path-coupled 
techniques.  That's the point.  Any design which you'd like to scale 
Internet-wise is probably broken, and especially so if it requires 
processing at the intermediate nodes.
 
> Interestingly enough, there are operators involved in the NSIS

I wasn't referring to 3GPP operators ;-)  (who are probably interested
in this for the first hop purposes, or for the first+last hop
purposes, where other methods might work nicely as well)

-- 
Pekka Savola                 "You each name yourselves king, yet the
Netcore Oy                    kingdom bleeds."
Systems. Networks. Security. -- George R.R. Martin: A Clash of Kings

---

From: Iljitsch van Beijnum <[email-masked]>
Date: Tue, 24 Aug 2004 17:56:29 +0200

> [[ I was hoping there would be more follow-ups on this thread, but

Be careful what you wish for...

First of all, allow me to emphatically NOT support your quest for 
removal of the router alert option. While I agree with many of your 
points, removing something that is both in use now (however localized) 
and may be beneficial to have in the future is not the right thing to 
do.

I think that if you want to provide better / more scalable QoS across 
the internet, it's better to start from scratch rather than try to cram 
RSVP into a completely new architecture.

> characteristics doesn't sit well with the operational models of ISPs.

Things change. ISPs are used to that. (They may whine, though.)

> Such a reservation by definition eats from the shared resource;

That's certainly not a given. If I make a phone call, my 64 kbps isn't 
taken from anyone else using the phone network, but from the pool of 
available unused capacity. You could do the same in an IP network, and 
you probably should or your regular customers will walk away.

> as a premium paid service.

That's not necessarily true either. Sure, there are applications that 
want/need huge amounts of bandwidth that they will want to reserve. But 
there are also applications that need a certain amount of low-delay, 
low-jitter bandwidth, where the problem isn't the amount of bandwidth, 
but making it low-delay and low-jitter (which you can't do for ALL your 
bandwidth if you run close to capacity for even conservative 
definitions of "close"). (And remember: there is always a bottleneck. 
Always.)

> virus-infected host requesting very large reservations), security

What we see today is that average home users eat a lot of bandwidth. 
Even with today's prices that gets expensive. The problem is that users 
REALLY don't like to pay for volume or be restricted in any way. An 
interesting approach here would be to allow users to use as much 
capacity as the network can support (yes, at 100% utilization) so the 
file sharers are happy, but at the same time provide a premium service 
that bypasses the thus created congestion. This doesn't have to be a 
paid-for thing: you could simply make the first 100 kbps that a user 
uses premium and everything on top of that bulk.

This would require per-user policers (probably several) but we really 
need those anyway (and smart versions too) to be able to do something 
about denial of service.

(We also need to work on the "packet loss is congestion" assumption 
though, as even a little packet loss (unavoidable for a bulk service) 
kills TCP. Having to over-provision just because congestion management 
is too stupid is sub-optimal.)

> reserve from the bandwidth used by others.

But having whatever the access speed happens to be as your limit is 
often not the best choice.

> income from allowing that.

Well, every network in the path between two arbitrary endpoints is 
always directly or indirectly paid by one of the endpoints. So if both 
endpoints want this service, their respective ISPs (and ISPs of ISPs) 
have an incentive to provide it.

> Best effort service is the fairest service available under such 
> circumstances.

Life isn't fair.  :-)  You REALLY don't want to do something 
interactive on a small pipe that you share with some p2p file sharers 
and have everyone receive "fair" packet loss percentages and delay.

I do agree that it only makes sense to implement all of this QoS stuff 
if and when different service levels make sense. For a while, it looked 
like it didn't, but I don't think over-provisioning can go on forever, 
especially now that we have both more bulk traffic and traffic that 
really wants QoS constraints.

---

From: Pekka Savola <[email-masked]>
Date: Wed, 25 Aug 2004 10:02:45 +0300 (EEST)

> Things change. ISPs are used to that. (They may whine, though.)

Well, I have seen no evidence of that, but of course this may be
different with some who provide e.g. VoIP services across their
backbone (I just don't know).

> you probably should or your regular customers will walk away.

Certainly, but that presumes there is sufficient capacity to begin 
with.  Why exactly would you need to do an explicit, 
application-specific reservation then?

> definitions of "close"). (And remember: there is always a bottleneck. 

Seems awfully close to something what DSCP's are for.

> uses premium and everything on top of that bulk.

I see tendency towards another direction: ISPs implement rate-limiters
or traffic quotas to the heavy bandwidth users, for all of their
traffic.

An alternative, and IMHO probably more workable solution than
providing "premium service" is going towards another direction; you've
probably heard of academic networks' less-than-best-effort and
scavenger diffserv schemes, where the users can mark their traffic to
be discarded if necessary.

Let's assume that the traffic which would be marked by the user as
"discard if needed" would not be counted towards bandwidth quotas or
rate-limits, but would still be allowed.  Then the users could get
high quality for their regular traffic, get the available quality and
resources for their bulk traffic they don't care much about (specific
apps), or not mark any of it (and get rate-limited, or billed for
extra, or whatever).  That would make the users *care* about their
environment, or be forced to worse quality.

If it's several high-bandwidth bulk transfer apps that are causing
problems, why not fix the traffic (non-)reservations for *them*, not
for the rest of the traffic?

> often not the best choice.

Certainly not, but consider that in the context of e.g., 3GPP or 3GPP2
wireless (constrained) link which the customer might want to
police/prioritize in some manner.  The operator really doesn't care
how the user prioritizes its own traffic, but the user might.  But we
need no RSVP-like model to achieve that.
 
> have an incentive to provide it.

Certainly, but unless you get big bucks from the user (so that the
next people in the chain get more than just a couple of cents), the
ISPs won't bother to set it up.  And if it requires the user to pay
big bucks, the users won't bother, and the few who would be willing to
pay are not a sufficiently large user base for the first-hop ISP to
set up their infrastructure, so nothing gets done.  This is the state
we've been with for 10 years or so, since IntServ model was
introduced..
 
> and have everyone receive "fair" packet loss percentages and delay.

Small pipes have their problems of course (where a different set of
tools could be used rather than RSVP or path-coupled signalling), but
that's not an interdomain problem that needs to be fixed.  Just
rate-limit all the traffic of the bulk transferers, and provide a
means for them to mark their bulk traffic so that they can get better
quality for their non-bulk transfer.