Subject: Re: E2E VoIP over MPLS ('VoMPLS') Header Compression

From: raymond zhang <[email-masked]>
Date: Sun, 16 Feb 2003 14:16:24 -0800

Hi Jerry,

The draft 
"http://www.ietf.org/internet-drafts/draft-ash-e2e-vompls-hdr-compress-00.txt" 
and 
"http://www.ietf.org/internet-drafts/draft-ash-e2e-crtp-hdr-compress-00.txt" 
indeed touch upon a practical requirement for SPs offering VoIP over their 
networks.  I'd support the suggestion of MPLS WG taking the lead on such 
efforts as discussed in this draft, perhaps in coordination with other WGs.

Regards,
Raymond

At 04:05 PM 2/12/2003 -0500, Ash, Gerald R (Jerry), ALABS wrote:
>All,
>
>I'd like to propose that the MPLS WG take on work regarding VoIP over MPLS 
>header compression.  This work was earlier accepted for consideration by 
>the MPLS WG (see the background summary below).
>
>Two I-D's related to this work are at:
>http://www.ietf.org/internet-drafts/draft-ash-e2e-vompls-hdr-compress-00.txt 
>(George is added as co-author in the next rev)
>
>http://www.ietf.org/internet-drafts/draft-ash-e2e-crtp-hdr-compress-00.txt
>
>Please review the documents and raise any issues/comments to the list.
>
>The work was presented at the IETF-55 PWE3 WG meeting (slides and meetings 
>notes at http://ietf.org/proceedings/02nov/index.html).  It was generally 
>agreed that the work did not fit well in PWE3, and that MPLS might be the 
>best overall fit.
>
>I propose a brief time slot at the IETF-56 MPLS WG meeting to very briefly 
>review the above proposals and issues, and get a sense of the room.
>
>Comments welcome
>
>Thanks,
>Jerry Ash
>
>Background:
>
>Some service providers (such as AT&T) have plans to migrate large amounts 
>of legacy voice traffic to VoIP, as well as to provide VPN services 
>enabling VoIP.  VoIP over MPLS ('VoMPLS') typically uses the encapsulation 
>voice/RTP/UDP/IP/MPLS.  For an MPLS VPN, the packet header is at least 48 
>bytes, while the voice payload is typically no more than 30 bytes.  VoIP 
>over MPLS header compression can significantly reduce the VoIP overhead 
>through various compression mechanisms.  This is important on access links 
>where bandwidth is scarce, and can be important on backbone facilities, 
>especially where costs are high (e.g., some global cross-sections).
>
>VoIP over MPLS header compression methods have been proposed earlier 
>(March, 2000) in the MPLS working group, however the relevant drafts have 
>expired.  In particular, George Swallow and Lou Berger proposed a 'simple' 
>end-to-end compression scheme in which all first-order differences in the 
>RTP/UDP/IP headers were transmitted, and in which the header compression 
>context was established through RSVP signaling.  The work was accepted 
>into the MPLS WG pending an addition to the MPLS WG charter.
>
>In the I-D's we propose 3 approaches to VoMPLS header compression: a) 
>re-use the methods in cRTP to determine the context and MPLS to route 
>packets, b) re-use the methods in Swallow's and Berger's 'simple' approach 
>to determine the context and MPLS to route packets, and c) re-use the 
>methods in cRTP to determine the context and the SCID (session context ID) 
>to route packets.
>
>Issues to discuss:
>
>1. WG charter -- The VoMPLS header compression work is not within the 
>current charter of the MPLS WG, so a charter extension is needed.  George 
>said he 'would have no objection to the work being done in MPLS'... but 
>suggested that the 'work fits more closely in the Transport Area/PWE3 WG' 
>(which is why it was brought there first).
>
>2. Protocol extensions -- As detailed in the I-Ds, extensions are proposed 
>to [cRTP] and [cRTP-ENHANCE], which involve a new packet type field to 
>identify FULL_HEADER, CONTEXT_STATE, etc. packets.  New objects are 
>defined for [RSVP-TE].  Extensions are also proposed to RFC2547 VPNs, 
>which create 'SCID routing tables' to allow routing based on the session 
>context ID (SCID).  These extensions need coordination with other WGs 
>(MPLS, CCAMP, AVT, ROHC, PPVPN, etc.).
>
>3. Resynchronization -- E2E VoMPLS using cRTP header compression might not 
>perform well with frequent resynchronizations.  The applicability and 
>performance need to be addressed (the 'simple' header compression approach 
>would avoid the need for resynchronization, but achieves less efficiency 
>than cRTP).
>
>4. Scalability -- E2E VoMPLS applied between CE-CE would perhaps require a 
>large number of LSPs to be created.  There is concern for CE ability to do 
>the necessary processing and the scalability of the architecture.
>
>4. LDP application -- It would be desirable to signal the VoMPLS tunnels 
>with LDP, since many RFC2547 VPN implementations use LDP as the underlying 
>LSP signaling mechanism.

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 18 Feb 2003 12:42:26 -0500

In message <[email-masked]>, raymond zhang w
> efforts as discussed in this draft, perhaps in coordination with other WGs.

Raymond,

Since you spoke in support of this and mention "practical requirement
for SPs offering VoIP", I'd be interested to hear 1) which service
provider is using gear that sends 30 byte payloads, 2) which gear is
sending 30 byte payloads.

I just spoke to someone at another SP yesterday about other things and
asked about packet sizes in the VOIP deployment.  The gear is Sonus
and the packet sizes were said to be "oh I don't know, I think about
200 on average but might depend on content due to silence suppression
and compression".  That is a far cry from 30 byte payloads.

It may be that some voice over ATM gear sends a 30 byte payload in a
55 byte cell.  That's stupid to start with.  Ten cells would get you
480 bytes minus some for LLC/SNAP if used plus AAL trailer for at
least 450 payload.  That's 15 times the payload for 10 times the cells
allowing 50% more capacity on the same network.  A more moderate 224
byte payload could be fit in 5 cells still yielding close to a 50%
gain.  But voice over ATM gear didn't do it that way.

Some voice over ATM product did things in a stupid way.  There is no
point in perpetuating this.  If the problem is migrating from these
then a better way to do the migration might be to put new VOIP gear on
the edges to replace the voice over ATM gear.  Use voice over
RTP/UDP/IP/MPLS/ATM temporarily.  This will still be more efficient
that using 30 byte payloads in ATM or 30 byte payloads in
RTP/UDP/IP/MPLS or RTP/UDP/MPLS with header compression.  Then
gradually get rid of the ATM.

Alternately if the SP needs to keep the voice/ATM gear the SP will
need some sort of gateway to the modern world.  Whether doing this or
not it would be good to get the voice/ATM vendor to change their gear
to use a larger frame rather than voice in individual cells, but that
may be unlikely to happen.  If anyone is going to make a voice/ATM to
voice/*/MPLS gateway (or IWU if you prefer that term), and wants to
improve efficiency, it would make more sense to go RTP/UDP/IP/MPLS and
don't bother leaving out IP and UDP or doing RTP header compression
but collect multiple ATM cells and build a bigger payload.

Curtis

---

From: raymond zhang <[email-masked]>
Date: Tue, 18 Feb 2003 16:03:55 -0800

Curtis,

>sending 30 byte payloads.

The practicality in my previous email pertains to the requirement for e2e 
compressed path for VoIP packets, not about the feasibility of offering 
"VoIP" over SP's network.

Regarding payload size, firstly the voice frame size varies with the ITU 
codecs selected on the CPE.  For example, G.729a has 10ms per voice 
sampling frame and one could configure to stuff either two or three frames 
per VoIP packet... Hence 20 to 30 byte payload.  Larger payload size may 
present an issue in terms of loss, esp for a global network since loss of 
one packet would result in multiple voice frame losses.

As for vendors, I'd say any CPE vendors should have this capability built in...

>voice/*/MPLS gateway (or IWU if you prefer that term),

Not likely

>and wants to

In the cases I am thinking of, voice frames are encaped in RTP/UDP/IP and 
transported over a MPLS network directly in frame mode...

Regards,
Raymond

---

From: Curtis Villamizar <[email-masked]>
Date: Tue, 18 Feb 2003 22:48:08 -0500

In message <[email-masked]>, raymond zhang w
> As for vendors, I'd say any CPE vendors should have this capability built in.

I think the SPs should speak as to whether the bandwidth inefficiency
of 30 byte payloads or the potential to lose a packet with a 200 byte
is a greater problem.  All of the SP I've spoken to that are doing
VOIP are using diffserv EF service or otherwise insuring that voice
traffic is rarely if ever dropped.

Curtis

---

From: raymond zhang <[email-masked]>
Date: Tue, 18 Feb 2003 21:55:45 -0800

At 10:48 PM 2/18/2003 -0500, Curtis Villamizar wrote:
Hi Curtis,

> > As for vendors, I'd say any CPE vendors should have this capability 
> built in.
> > ..
>
>

Surely...  bandwidth inefficiency is certainly of a great concern to 
SPs.   In areas where ample capacity can absorb any kind of network 
abnormality, some SPs has choosen not to even have diffserve provisioned at 
each HoP which would warrant larger payload size.  There are two areas, 
where EF is essential to provide a consistent level of statistical 
performance targets for voice traffic which are prone to congestion - PE-CE 
links and where SP's network spans over capacity scarce or expensive 
regions...  One example, if GK is not used and calls are made directly 
between two CEs across the network, there is basically no bandwidth 
admission mechanism to regulate the call request...  If EF bandwidth is 
allocated for 100 calls, when the 101st call comes in, EF would police 
across all voice packets of all calls hence packet drops may occur (the ash 
draft using rsvp signalling may resolve this issue).  In cases such as 
these, smaller payload size may help minimize the frame loss ratio while 
bandwidth efficiency may then be achieved via header compression...  These 
are just a couple of reasons I would like to voice support the two Ash drafts.

Regards,
Raymond

---

From: Dave Cooper <[email-masked]>
Date: Tue, 18 Feb 2003 20:50:13 -0800

>is a greater problem.  All of the SP I've spoken to that are doing
Bandwidth inefficiency is a much greater problem.

>traffic is rarely if ever dropped.
Correct.

---

From: Curtis Villamizar <[email-masked]>
Date: Wed, 19 Feb 2003 10:46:29 -0500

In message <[email-masked]>, raymond zhang w
> are just a couple of reasons I would like to voice support the two Ash drafts

Let me see if I understand this.  If you have 101 calls you have some
(misguided) notion that you drop less with tiny payloads and lots of
overhead.

If you had large payloads the same link would hold 120-150 calls
depending on how large the payload was.

Next question - is your notion of drop more with larger payloads only
applicable to ATM when packet shredding is being done rather than
discarding entire frames?  Don't all modern ATM switches support PPD?
If this is an ATM only issue and furthermore an old ATM switch issue
its probably not one for the IETF to address.

The way things have been explained to me is an attempt is made to keep
no more than about 30% of the bandwidth on a link for EF (or pick a
number) and leave the rest for IP.  The actual EF reservation might be
40% to 70% of the link so if you went to 101 calls and only 100 fit,
IP would be squeezed out of some more bandwidth than you'd like it to.

Are you talking about EF on an IP or MPLS diffserv network or a VC on
an ATM network or TDM where you have to reserve exactly what you
expect to use and get hammerred if you use ever so slightly more?

This is what has been recommended in the diff-serv and MPLS WGs almost
since the diff-serv WG started.  The technique is certainly no secret.
A few people at SPs have hinted that their employer may consider the
fact that they are using it to be some sort of secret.

Curtis

---

From: Dave Cooper <[email-masked]>
Date: Tue, 18 Feb 2003 20:50:13 -0800

Correct.

---

From: Dave Cooper <[email-masked]>
Date: Tue, 18 Feb 2003 20:50:13 -0800

Correct.

---

From: raymond zhang <[email-masked]>
Date: Thu, 20 Feb 2003 04:59:49 -0800

At 10:46 AM 2/19/2003 -0500, Curtis Villamizar wrote:

Hi Curtis,

Sorry I went off-line yesterday...  Pls see further comments in line below.

> > > > sampling frame and one could configure to stuff either two or three 
> frame
> > s
> > are just a couple of reasons I would like to voice support the two Ash 
> drafts
> > .
>
>

Good point...my brief description above is not the best example to 
illustrate this.  However issue remains in such situations.  Upper bounds 
in terms of delay and loss need to be maintained in a much stricter sense 
for such real-time class as voice...  Also, as Bur and Jerry pointed out, a 
payload size of 20 to 30 bytes enables us to do that much better across the 
voice path in delay budget planning  in terms of queue efficiency (per HoP 
delay and variance), esp across the lower speed local-access links at lower 
sub-T1/E1 rates in an integrated data/voice environment.  It may be more 
pronounced when the voice path spans across multiple continents with low 
speed last mile connections.

>depending on how large the payload was.

You meant muxing voice frames from multiple calls into one packet ?  Now I 
understand a bit what you are referring to, altho not all customer 
locations have same number of concurrent calls, many remote locations with 
very small # of concurrent calls.  That means we would either have voice 
packets with variable frame-sizes on EF queues or stuff more voice frames 
from the same call into one packet ?

>its probably not one for the IETF to address.

No.  It is over a MPLS packet network

>IP would be squeezed out of some more bandwidth than you'd like it to.

A larger EF allocation on an egress interface may starve the rest of the 
data queues... That's why there is upper bound for EF allocation.  Of 
course, the higher the link speed, the higher EF bandwidth can be allocated 
(e.g. local-access link).  In addition, EF traffic is policed which will 
not erode into bandwidth space allocated for other classes.

>expect to use and get hammerred if you use ever so slightly more?

EF on MPLS diffserv network...

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 20 Feb 2003 11:01:17 -0500

Raymond,

I trimmed out part of your response.

In message <[email-masked]>, raymond zhang w
> speed last mile connections.

Tiny payload is needed for queue efficiency is a myth left over from
the ATM days.  It is simply not true.

Lose overhead and queues are smaller.  Do some delay variation
buffering and jitter is a total non-issue.

This is like the IP vs ATM discussions of the mid-1990s.

> from the same call into one packet ?

No I meant increasing payload to 200 bytes to decrease overhead so
that there was 50% more payload capacity on the same link.

> not erode into bandwidth space allocated for other classes.

This is what the ISPs are doing.  TCP is compressible and some 95-98%
of Internet traffic is TCP, much of it large enough flows to compress
in the event of congestion.

This is what ISPs are doing.  They could favor SLA traffic over
non-SLA traffic (at least one way) but it hasn't become a practical
problem for most ISPs to the point where they've bothered to do so.

For the most part backbone links are sufficiently overprovisioned to
absorb the variation in traffic load without special queueing except
for the EF service.  Some of the aggregation links (regional) might be
a little more problematic for some SPs.

The statement that EF traffic is policed in this case is for the most
part not accurate.  SPs would like the VOIP gear to have limits on
calls going out but most (all?) don't.  Failing that SPs would like to
be able to use policing but it is impractical.  Ideally, VOIP would
use LSPs with different EXP and setup/resv priority with a bandwidth
allocation that reflected the traffic volume being offered.
Currently, the only practical way to get the bandwidth allocation
right or close to right is either configured based on history (usually
close to right, but some argue maybe not close enough) or based on
filtered measurements (and the filters may need some work here).
Measuring the bandwidth allows the LSPs to be rerouted but does not
handle the case were the bandwidth is no where to be found and the
VOIP equipment needs to block calls.  So far the practical answer has
been to ignore that case and ask VOIP gear for a configured limit on
calls.

There is some work on adaptive bandwidth techniques were the payload
itself is changed (lower bandwidth encoding) when congestion is
detected (some level of loss occurs).  An interesting change that any
VOIP gear could make without any changes to standards would be to move
the playout point ahead in time by 10-50 msec as congestion is
detected.  In the absense of congestion, you'd have tiny packets and
as small a delay as practical.  On the onset of congestion, you'd have
small loss and a switch from 30 msec to 40 msec playout delay.  The
improvement for G.709 would be from 30/(30+40) to 40/(40+40) percent
of the bandwidth being used for payload (43% to 50%).  You would
immediately get 16+% more capacity for a 10 msec additional delay.  If
congestion persists, you can push this all the way to 80 msec, and get
55% more traffic over the same link.  If you could also just compress
the RTP and UDP headers (yielding 24 byte overhead rather than 40),
this could become quite viable (55-76% payload for 30-80 msec).  If
the network becomes congested, it certainly makes more sense to relax
the delay criteria than to introduce loss.

Curtis

---

From: raymond zhang <[email-masked]>
Date: Thu, 20 Feb 2003 09:44:20 -0800

Curtis,

Thanks for the response and see some further comments in line...   Based 
upon discussions we have thus far including your response to my comments in 
response to Uzelac's email, maybe I should make some clarifications first 
which may help my understanding of your input as well in any further 
discussions:

The environment in which I am referring to is a global IP/MPLS packet-based 
network spanning all major and sub-continents where it is not always 
possible to get ample bandwidth (over-provisioning) economically (very 
expensive in fact)...  The issues of delay or packet loss do not normally 
present problems at all as you pointed out, in some continental or national 
networks of ISPs or SPs but to offer similar level of integrated services 
(voice/data) for customers with world-wide locations, it is in these remote 
regions and over last mile local loops that we spend a bit more efforts on 
to optimize.

On low speed local-loops (PE-CE links), IP/UDP/RTP compressions is 
needed.  I think we both agree on that.  As for PE doing the comp/depcomp, 
I'd like to hear a bit more from the Vendors...  My experience has been 
that hardware based compression at PE levels on the edge of the network 
with large # of aggregating diffserve-enabled customer links presents 
complexity that may become unmanageable to SPs and with amount of edge 
features already processed through hardware, would adding 
compression/decompression be even feasible ?  Besides,   the upgrade 
path  may be very costly for these line cards which maybe deployed in large 
numbers across the global network as we add new feature sets/hardware 
capabilities.  So I am not in favor of it as an option for PE level 
routers.   Therefore the concept of distributing compression/decompression 
into CE levels, not in the network appeals in a great deal.  If PE is not 
doing the comp/decomp, then the compressed path needs to be e2e...

Further comments in lines ....

Regards,
Raymond

>the ATM days.  It is simply not true.

I think there are two aspects in this...  Firstly traffic flows with 
constant packet size would improve queue efficiency (M/D/1) and smaller 
packet payload would reduce delay across low speed links (not an issue once 
again in areas of network with large pipes since no or little qeueuing) 
which is required as part of the budge delay planning.  The voice quality 
is provided on a per path basis not just over segments of the path inside 
SP's major portion of the backbone.   ( I don't mean to spur another around 
of discussions on ATM vs. IP at all ! I thought issues are well understood 
by now).

>that there was 50% more payload capacity on the same link.

So if not muxing voice frames from multiple calls between same pair of GWs, 
would then stuff more voice frames of the same call into the RTP payload 
?   If CE is the gateway (oftentimes it is in today environment), then as 
mentioned before larger packet payload would present delay issues in 
queuing/serialization over low speed local loops.  I don't think PE is the 
right place to do that this.

>part not accurate.

But it is, at least over the CE-PE link.

>SPs would like the VOIP gear to have limits on
>allocation that

But on the edge (CE-PE links) and inside network, voice traffic is treated 
via strict priority over other types of traffic which would begin to starve 
other classes of traffic should its resource occupation exceeds a threshold 
(varies with link speed)...  You would need CAC on this which there is not 
one over a MPLS/IP packet backbone...  TE LSP only signals the reservation, 
not CACing.

>  reflected the traffic volume being offered.

Right... We have to understand the per-class (traffic type) traffic mix in 
the network for primary path placements planning.  But it is somewhat 
difficult when over-provisioning is not a good option in some areas of the 
networks and during node/link events.

---

From: Curtis Villamizar <[email-masked]>
Date: Thu, 20 Feb 2003 14:35:14 -0500

In message <[email-masked]>, raymond zhang w
> Further comments in lines ....

The applicability statement part of this is embedded in the draft
itself, rather than following a requirement document.

You are citing PE-CE as the primary problem.  Others have cited PE to
PE within the metro.

If the problem is PE-CE, lets stick with that problem.

If PE-CE is the problem you are trying to solve, then RTP/UDP/IP/PPP
compression should do wonders.  I agree with you entirely that the
current practical problem there is that the PE can't handle the
comp/depcomp at anywhere close to line rate on even relatively slow
interfaces.  The PPS rate for these tinygrams is enormous.  (Of course
making them bigger would help reduce the PPS rate in addition to
improving the encaps efficiency).

At one packet per 30 msec, you have 30 pps.  At 8kbit/s a T1 holds not
quite 200 of these if the PPP encaps was 4 bytes or less.  Times 200
calls you have 6000 pps.  Nothing at all for a line card, but not many
of these will start to swamp a route processor if that is how the
comp/decomp is implemented.

Putting the compression in the CE might on the onset sound appealing.
If you think you have problems with the PE handling the comp/decomp,
wait til you try to bring up a full mesh of LSPs to all the CE for
this sort of scale of global networks.  No sense trading one practical
limitation for another of much more enormous magnitude.

If there is a huge number of CE then at some point you'll have to
partition the network to make this continue to work.  If so you might
get CE to some voice/VOIP aggregation box/gateway and have these
terminate the LSPs and do comp/decomp, then communicate among
themselves.  This is likely to be a huge comp/decomp load.  You can
concentrate the comp/decomp at some specialized processor or try to do
the processing on the line cards (which may mean new line cards).

If a CE is supporting on the order of at least 100 (to 200 if encaps
efficiency can be improved), then muxing might be easier on the PE
router.  For example, separating 300 pps of 600 byte packets is likely
to be a lot easier (even if 6000 pps are sent out) than compressing
and decompressing 6000pps and would add zero delay.  Either way,
moving processing to the PE line card seems essential.

I'm just trying to come up with something that will actually work.

Regards,

Curtis

ps - wrt queueing and serialization delay.  30bytes into a T1 incurs a
serialization delay of 240bits/1.5mbits or 0.16 msec.  200 bytes is 1
msec serialization.  If CE means customer handset then we're e2e MPLS
becomes completely absurd so I think it should be safe so assume at
least T1 and it not you get delay.  With serialization times on the
order of 1 msec edge and these days under a microsecond in cores, the
queueing argument has never held water for priority queued traffic
that is not congested.

That's a moot point for a single flow if you have to hold the playout
time for 200 msec to get 200 bytes packets because that is already an
excessive delay.  For multiplexing, the argument against large packets
based on serialization and queueing simply doesn't hold up.

We're not talking IP over barbed wire in the third world, right?  Its
at least T1?

---

From: Kireeti Kompella <[email-masked]>
Date: Mon, 24 Feb 2003 00:42:54 -0800 (PST)

Hi Curtis,

On Tue, 18 Feb 2003, Curtis Villamizar wrote:

> is a greater problem.

Two questions (more probably to others than to you):
a) what is the relative bandwidth of voice?
   (i.e., if voice accounts for <10% of a link's capacity, it
   may not be worth worrying about even a 100% overhead); and
b) if a 30 byte voice packet got dropped, would you expect that
   6 other voice packets that were temporally close would escape
   the hatchet?  I.e., how big really is the penalty of a 200 byte
   packet?

Kireeti.

---

From: Curtis Villamizar <[email-masked]>
Date: Mon, 24 Feb 2003 08:58:43 -0500

>    the hatchet?  I.e., how big really is the penalty of a 200 byte

I'm looking forward to seeing a review of the requirements doc.
Actually not really.  But it will be more sane than the current
requirements moving target.

Curtis