Subject: RE: [Speermint] Secure RTP

From: "Geoff Devine" <[email-masked]>
Date: Tue, 8 Aug 2006 18:22:53 -0400

I agree 100% with this analysis of what can happen when you use 503.
There is a real risk that it can cause the network to overload.

That said, I think you guys are missing a use case:

I have a media gateway in New York.  My peer has a media gateway in San
Francisco.  One of my New York subscribers dials a San Francisco
telephone number.  This number is not a VoIP device.  I want to use the
peer media gateway in San Francisco whenever possible to avoid carrier
charges.

In this use case, PSTN gateway economics are certainly a consideration.
The media gateway controller in San Francisco has finite resources and
it would be a poor architecture to send INVITE messages to it from New
York when it is congested.

Geoff

-----Original Message-----
From: Michael Hammer (mhammer) [mailto:[email-masked]] 
Sent: Tuesday, August 08, 2006 2:22 PM
To: Brian Rosen; Geoff Devine; Patrick Melampy; Hadriel Kaplan
Cc: [email-masked]
Subject: RE: [Speermint] Secure RTP

Brian,

When you get to the point that you can't keep up with new offered calls,
you may send a 503 that causes another server to get the added load.  If
that server was also near it's thrashing point, it may also start
rejecting calls with 503 leading to yet another server getting
overloaded, and so on.  Retransmissions of the new call can oscillate
through your servers searching for one that can handle a new call.  But,
in the mean time it takes resources to reject a call.  The call gapping
tells upstream elements how to limit the new load   so that there is no
rejection of call resource consumption.

In your case, by suggesting the load balancer, you have merely moved the
problem into the receiving network, but now creating a new point of
congestion at the load balancer.

Yeah, they are just servers, but presumable they have software on them,
and presumable the software vendors don't write software for altruistic
reasons, but need to make money too.  And since revenue from voice is
being pushed lower, I don't see SPs buying into massive server farms
with 90% sitting idle.

I agree though that this shouldn't be based on PSTN GW economics, we are
trying to bypass that in Speermint.  However, the need to police borders
and actually charge for calls may have an impact anyway.

I was looking for the ability to dynamically direct calls to various
signaling border elements based on a variety of business and possibly
varying technical needs, congestion being only one aspect.  In some
ways, this may be akin to load balancing, since based on some
multifaceted criteria, it is making visible to any potential peer which
signaling peering point to direct traffic to.  Or deny access
altogether.

Mike

> -----Original Message-----
> From: Brian Rosen [mailto:[email-masked]] 
> Sent: Tuesday, August 08, 2006 12:45 PM
> To: Michael Hammer (mhammer); 'Geoff Devine'; 'Patrick 
> Melampy'; 'Hadriel Kaplan'
> Cc: [email-masked]
> Subject: RE: [Speermint] Secure RTP
> 
> I'm not so sure about that.
> 
> I do think that some of the folks that are designing VoIP 
> systems make them look like telephone systems and thereby 
> inherit the problems.  I think many of the architectures 
> being put forth have way too many elements, when only one or 
> two (replicated of course) need to be there.  I am always 
> amazed comparing statistics of two VoIP networks that started 
> accepting customers at around the same time, where one had a 
> cost to first customer north of 100M and the other was 
> something under 10.  The cost of the millionth customer ALSO 
> favored the latter, and it never turned around, while 
> customers perceived the services to be equivalent (I'm sure 
> the carriers didn't).
> 
> I'm not sure what you mean by "cascading failures" in this 
> context.  Please explain.
> 
> I'm also not sure that business economics don't allow you to 
> have servers waiting for peak traffic.  That depends on how 
> these systems are built only a little, it more depend on how 
> they are priced.
> 
> The "servers" here are the signaling elements, and they are 
> just computers.
> You can have a heck of a lot of computer sitting around doing 
> nothing waiting for peak traffic.  That isn't necessarily 
> true of the media, of course.
> 
> Brian
> 
> > -----Original Message-----
> > From: Michael Hammer (mhammer) [mailto:[email-masked]]
> > Sent: Tuesday, August 08, 2006 12:18 PM
> > To: Brian Rosen; Geoff Devine; Patrick Melampy; Hadriel Kaplan
> > Cc: [email-masked]
> > Subject: RE: [Speermint] Secure RTP
> > 
> > It is more like a telephone network than a Web service.
> > 
> > How do you explain cascading failures, then?
> > 
> > Also, the business economics don't allow you to have 90% of your 
> > servers sitting around doing nothing just waiting for peak traffic.
> > 
> > Mike
> > 
> > 
> > > -----Original Message-----
> > > From: Brian Rosen [mailto:[email-masked]]
> > > Sent: Friday, August 04, 2006 12:01 PM
> > > To: 'Geoff Devine'; 'Patrick Melampy'; Michael Hammer (mhammer); 
> > > 'Hadriel Kaplan'
> > > Cc: [email-masked]
> > > Subject: RE: [Speermint] Secure RTP
> > >
> > > This isn't a telephone network.
> > >
> > > We typically DON'T have overload conditions on well engineered 
> > > networks.  We presently have some mighty fine self 
> limiting issues 
> > > with media, but if we had proper QoS on the media, that 
> would be, I 
> > > think, very sufficient congestion control.  I've never 
> seen a well 
> > > engineered SIP based network where the signaling issues 
> were a big 
> > > problem.  It's almost all in the media.
> > >
> > > I know all about "Americal Idol" night.  The numbers just aren't 
> > > scary any more.
> > >
> > > Do the math.  A million BHCA is 278 calls per second.  How many 
> > > million do you want?  Rack up the servers (best make them 
> stateless, 
> > > but that's not a problem).
> > >
> > > Sure, you need a load balancer.  Sure, you need a 
> failover strategy 
> > > with a good database.
> > >
> > > Now, every system has limits.  We have an answer for those: busy.
> > >
> > > More importantly, we actually can use some telephone 
> engineering to 
> > > predict the traffic at the peering point.  We can 
> engineer systems 
> > > pretty straightforwardly to handle that traffic.  We DO need some 
> > > congestion control mechanisms, because the peering mechanisms we 
> > > have do have finite capacity.  However, I don't see much need for 
> > > telephone style call gapping and only trivial metering.
> > >
> > > Today, we can build peering systems where the actual 
> peering happens 
> > > at multiple places around the globe, where there is no effective 
> > > limits on signaling bandwith (or for that matter, media 
> bandwith) to 
> > > and from the peering points.  We can have SIP proxies 
> that perform 
> > > the actual peering function (take a call, do a dip to determine 
> > > where it's going, use available DNS and other mechanisms 
> to spread 
> > > load) that are similarly spread out and redundant enough 
> that we get 
> > > the kind of reliability we expect.  And we can engineer 
> them to take 
> > > load up to a static limit reliably.
> > >
> > > What can't we do?
> > >
> > > Brian
> > >
> > >
> > > > -----Original Message-----
> > > > From: Geoff Devine [mailto:[email-masked]]
> > > > Sent: Friday, August 04, 2006 11:35 AM
> > > > To: Brian Rosen; Patrick Melampy; Michael Hammer (mhammer); 
> > > > Hadriel Kaplan
> > > > Cc: [email-masked]
> > > > Subject: RE: [Speermint] Secure RTP
> > > >
> > > > Telephone networks don't and can't work that way.  They
> > > have overload
> > > > conditions and you absolutely must have a mechanism for
> > > dealing with
> > > > them.  In the legacy network, you can take SS#7 CICs 
> out of service.
> > > > You can implement call gapping and call metering.  We need
> > > equivalent
> > > > mechanisms and can likely do far better than best common
> > > practices in
> > > > the legacy network.
> > > >
> > > > Geoff
> > > >
> > > > -----Original Message-----
> > > > From: Brian Rosen [mailto:[email-masked]]
> > > > Sent: Friday, August 04, 2006 11:27 AM
> > > > To: Geoff Devine; 'Patrick Melampy'; 'Michael Hammer 
> (mhammer)'; 
> > > > 'Hadriel Kaplan'
> > > > Cc: [email-masked]
> > > > Subject: RE: [Speermint] Secure RTP
> > > >
> > > > There is no evidence that you can't do this with the 
> mechanisms we 
> > > > have, and some that we can.  I believe in a contractual,
> > > static, limit
> > > > on traffic between peers.  I don't think we need anything else.
> > > >
> > > > Brian
> > > >
> > > > > -----Original Message-----
> > > > > From: Geoff Devine [mailto:[email-masked]]
> > > > > Sent: Friday, August 04, 2006 11:01 AM
> > > > > To: Brian Rosen; Patrick Melampy; Michael Hammer
> > > (mhammer); Hadriel
> > > > Kaplan
> > > > > Cc: [email-masked]
> > > > > Subject: RE: [Speermint] Secure RTP
> > > > >
> > > > > I'm with Patrick on this one.  The requirements for a
> > > carrier-class
> > > > > telephone network are quite different from the 
> requirements for 
> > > > > a web server.  A web server doesn't have the moral 
> and ethical 
> > > > > obligation to ensure that the network is stable to 
> complete 911 
> > > > > calls.  If the
> > > > network
> > > > > floods a soft switch or media gateway controller with peering 
> > > > > traffic with no notion of congestion control, flow
> > > control, or fault
> > > > management,
> > > > > you could kill somebody.  In the real world, resources
> > > are finite,
> > > > > not infinite and the network architecture has to take
> > > that into account.
> > > > >
> > > > > Geoff
> > > > >
> > > > > -----Original Message-----
> > > > > From: Brian Rosen [mailto:[email-masked]]
> > > > > Sent: Friday, August 04, 2006 10:20 AM
> > > > > To: 'Patrick Melampy'; 'Michael Hammer (mhammer)';
> > > 'Hadriel Kaplan'
> > > > > Cc: [email-masked]
> > > > > Subject: RE: [Speermint] Secure RTP
> > > > >
> > > > > I don't think any of this is something we are 
> chartered to do.  
> > > > > I
> > > > don't
> > > > > think it's needed, at least for a long time, possibly forever.
> > > > >
> > > > > We have lots of experience building "farms" of servers that 
> > > > > handle
> > > > HTTP
> > > > > load, and I don't see any reason we shouldn't use all 
> of those 
> > > > > techniques to deal with SIP load.  I certainly don't see
> > > reasons to
> > > > > make the originator aware of congestion issues at the 
> > > > > terminating domain, beyond the contractual limits 
> that may be in place.
> > > > >
> > > > > I don't see any reason why peers should be aware of
> > > element failures.
> > > > > We
> > > > > clearly understand how to have a set of boxes that appear
> > > to be one
> > > > > domain name that are always available.
> > > > >
> > > > > I don't agree we need anything like a BGP for SIP any 
> time I can 
> > > > > foresee.
> > > > > The underlying network handles all the issues I am aware
> > > of.  Given
> > > > the
> > > > > name
> > > > > of a domain (or subdomain), normal SIP routing is quite
> > > sufficient I
> > > > > think.
> > > > > We attempted this for a really simple subset of the
> > > problem (gateway
> > > > > selection, see TRIP), and we found it was not needed (or at 
> > > > > least the complexity it introduced was not worth the 
> value it provided).
> > > > >
> > > > > Brian
> > > > >
> > > > >
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Patrick Melampy [mailto:[email-masked]]
> > > > > > Sent: Friday, August 04, 2006 9:59 AM
> > > > > > To: 'Brian Rosen'; 'Michael Hammer (mhammer)'; 
> 'Hadriel Kaplan'
> > > > > > Cc: [email-masked]
> > > > > > Subject: RE: [Speermint] Secure RTP
> > > > > >
> > > > > > Brian,
> > > > > >
> > > > > > I agree with you. However some aspects are protocol
> > > worthy. Path
> > > > > > determination -- or said another way -- the next 
> hop network 
> > > > > > to use
> > > > to
> > > > > > reach
> > > > > > a destination domain, must be partially provisioned,
> > > and partially
> > > > > > discovered. The discovery aspects are the dynamic aspects. 
> > > > > > This
> > > > would
> > > > > > include things like:
> > > > > > 	1.) Proxy Availability
> > > > > > 	2.) Reachable Domains
> > > > > > 	3.) Congested Domains
> > > > > >
> > > > > > In #1 -- proxy availability -- its very similar to how
> > > SS7 is used
> > > > to
> > > > > set
> > > > > > the service states on each and every DS0. SS7 is 
> the protocol
> > > > family,
> > > > > and
> > > > > > ISUP messages are sent to bring circuits in and out 
> of service.
> > > > > > The
> > > > > actual
> > > > > > phone numbers to send down the circuit are provisioned
> > > in SS7. In
> > > > our
> > > > > case
> > > > > > to have a well managed service, we need an equivalent
> > > to "circuit
> > > > > service
> > > > > > state", which might be embodied as a call arrival rate, 
> > > > > > maximum
> > > > > sessions
> > > > > > permitted, and maximum bandwidth to use.
> > > > > >
> > > > > > In #2 -- reachable domains -- Unlike the PSTN which 
> uses the 
> > > > > > NPA-NXX routing models which are hierarchical and well
> > > understood,
> > > > > > IP domains are amorphous and have very unpredictable
> > > intersection
> > > > > > points. This is why BGP was invented for routing IP
> > > packets. The
> > > > > > same is true for SIP based
> > > > > sessions.
> > > > > > If
> > > > > > there is to be peering, there will need to be a domain 
> > > > > > advertisement protocol that goes network to network. Thus a 
> > > > > > protocol is required
> > > > > here.
> > > > > >
> > > > > > In #3 -- congestion -- is dynamic and not 
> provisionable. It is
> > > > > directly
> > > > > > related to #2, and the congestion detection and prevention
> > > > mechanisms
> > > > > must
> > > > > > follow in reverse the paths established in #2.
> > > > > >
> > > > > > So I think we do have some protocol work here. I 
> also think we 
> > > > > > need
> > > > to
> > > > > > leave
> > > > > > to the individual networks, any and all decisions on how to 
> > > > > > run
> > > > their
> > > > > > networks, choose next networks, etc.
> > > > > >
> > > > > > -----Original Message-----
> > > > > > From: Brian Rosen [mailto:[email-masked]]
> > > > > > Sent: Friday, August 04, 2006 8:47 AM
> > > > > > To: 'Michael Hammer (mhammer)'; 'Hadriel Kaplan'
> > > > > > Cc: [email-masked]
> > > > > > Subject: RE: [Speermint] Secure RTP
> > > > > >
> > > > > > I think you have to be careful with the "p" word.
> > > > > >
> > > > > > I don't see the purpose in standardizing mechanisms 
> for peers 
> > > > > > within
> > > > a
> > > > > > peering federation to discover policies that apply to
> > > each other.
> > > > > That is
> > > > > > provisioning, and there is usually not a lot of value in
> > > > standardizing
> > > > > > provisioning.  It's not dynamic (in fact, it's usually
> > > specified
> > > > > > by contract).
> > > > > >
> > > > > > What there is value in is in defining what 
> mechanisms exist to
> > > > > provision.
> > > > > > So, for example, if one of the things peering means is that 
> > > > > > the rate
> > > > > of
> > > > > > calls is limited in some way, then we probably have to
> > > standardize
> > > > the
> > > > > > definition of limiting, but not the way to 
> determine what the 
> > > > > > limit
> > > > > is.
> > > > > > In
> > > > > > that way, the vendor who builds the proxy for Peer A
> > > will be able
> > > > > > to
> > > > > build
> > > > > > in it the ability to meet the policy for Peer B.  
> This is not 
> > > > > > a
> > > > > protocol
> > > > > > issue, but it is what you would expect to find in a BCP.
> > > > > >
> > > > > > Similarly, it's not clear to me that a federation will
> > > necessarily
> > > > > define
> > > > > > all of the potential policy mechanisms, or what 
> their values 
> > > > > > are
> > > > > limited
> > > > > > to.
> > > > > > It clearly WILL do that for some variables, and again, we 
> > > > > > don't need automatic provisioning mechanisms to discover 
> > > > > > these.  Even within a federation, I think it will be common 
> > > > > > that bilateral arrangements
> > > > > making
> > > > > > further policy limitations will exist.
> > > > > >
> > > > > > Brian
> > > > > >
> > > > > > > -----Original Message-----
> > > > > > > From: Michael Hammer (mhammer) [mailto:[email-masked]]
> > > > > > > Sent: Thursday, August 03, 2006 3:13 PM
> > > > > > > To: Hadriel Kaplan; Brian Rosen
> > > > > > > Cc: [email-masked]
> > > > > > > Subject: RE: [Speermint] Secure RTP
> > > > > > >
> > > > > > > The focus of Speermint should be centered around
> > > whether one can
> > > > > > > discover the policies of a potential peer and 
> whether those
> > > > policies
> > > > > can
> > > > > > > systematically describe the requirements on multiple
> > > levels that
> > > > > will
> > > > > > > need to be supported to have a successful 
> communication with 
> > > > > > > that
> > > > > peer.
> > > > > > >
> > > > > > > It is not about specifying X, Y, Z, or <name your
> > > favorite MUST
> > > > > here>
> > > > > > >
> > > > > > > Is is about whether the policy can be stated so one
> > > knows that
> > > > > > > X,
> > > > Y,
> > > > > ...
> > > > > > > is required to peer.
> > > > > > >
> > > > > > > I suspect that each federation will select 
> different MUSTs 
> > > > > > > for
> > > > each
> > > > > > > category of choices we come up with.
> > > > > > >
> > > > > > > Mike
> > > > > > >
> > > > > > >
> > >
> 

_______________________________________________
Speermint mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/speermint

---

From: "Michael Hammer \(mhammer\)" <[email-masked]>
Date: Tue, 8 Aug 2006 14:22:11 -0400

Brian,

Mike

---

From: Klaus Darilion <[email-masked]>
Date: Wed, 09 Aug 2006 14:28:34 +0200

> York when it is congested.

If you have 2 GWs, both under your control, distributing the load status 
of the gateways is rather easy. But it gets complicated if there are 
lots of peers, and in a federation peering scenario you do not even know 
each of your peers. It really gets complicated to send all possible 
peers the current load of your ingress point. That does not scale.

regards
klaus

---

From: "Geoff Devine" <[email-masked]>
Date: Wed, 9 Aug 2006 10:59:39 -0400

Klaus wrote:
> It really gets complicated to send all possible peers the current load
> of your ingress point. That does not scale.

Exactly.  You aren't going to build your peering network as a fully
interconnected mesh.  I suspect the most workable approach to solve this
is to have a hierarchy where higher points in the hierarchy understand
the congestion levels of the entities lower down in the hierarchy.  A
load balancer that understands congestion, as it were.  The whole point
is to create an architecture where you never have to retry INVITES since
the call rejection rate due to lack of resources is low enough to meet
legacy telco requirements.

Geoff

-----Original Message-----
From: Klaus Darilion [mailto:[email-masked]] 
Sent: Wednesday, August 09, 2006 8:29 AM
To: Geoff Devine
Cc: Michael Hammer (mhammer); Brian Rosen; Patrick Melampy; Hadriel
Kaplan; [email-masked]
Subject: Re: [Speermint] Secure RTP

regards
klaus