Subject: [XCON] On encodings

From: Henning Schulzrinne <[email-masked]>
Date: Thu, 16 Mar 2006 23:19:27 -0500

Since we're on the perennially fun topic of protocol encodings, maybe  
we can get past the stereotypes and try to do an engineering  
comparison, or at least an attempt at an approximation. Roughly  
speaking, an XML element is equivalent in functionality to a TLV  
object: it contains a label (element tag and the 'type' field),  
length or delimiter fields to figure out the boundaries, and the  
actual data (value). ASN.1 BER is close enough to TLV for this apples- 
to-pears comparison.

For TLV, you have a type value of 16 to 32 bits, depending on the  
predictions on the need to extend the protocol. For XML, you have  
<tag  </tag>, thus, 4 bytes plus whatever length you make the tag  
from 'X' to  
'WeLikeToPutTheWholeRFCIntoTheTagSoThatTheImplementerDoesntHaveToReadThe 
RFC'.

For length, this is already counted above for XML; for TLV, it would  
presumably be again 16 or 32 bits.

Thus, the basic overhead is fairly similar if you choose small  
element names. Since numeric labels aren't self-describing at all,  
this only seems like a fair comparison.

The big difference is in the datatypes. Clearly, if you have long  
binary opaque data, base64 encoding typical for XML costs you about 30%.

Integers and floats are trickier: Unless you believe in ASN.1-style  
variable-length encoding, it is quite possible that in many practical  
cases, XML integers will be shorter since TLV always has to allocate  
the maximum range, presumably 32 bits for integers and 32 (single  
precision) or 64 bits for floats. You can build custom fixed-point  
types in binary encodings, but they are a pain to get right (All  
kinds of funny things happen because of signed vs. unsigned issues,  
for example. Java programmers will hate you if you try this...)

Strings are obviously the same size in either TLV or XML, except for  
the use of delimiters (such as ") or length fields, but that's a wash  
at roughly 2 bytes each.

The one big difference for XML is the namespace declaration. The  
impact of that depends on the number of namespaces and the size of  
the overall document. This is also hard to compare since they give  
you proprietary extensions without the kludges like vendor IDs that  
binary protocols have to go through (unless they choose similar Java- 
style or domain-style tags). Indeed, for namespace-style extensions  
that aren't IANA registered, TLV is usually a pain. You end up with  
either per-TLV long labels or re-invent the XML-style indirection  
approach.

This is a rough comparison, but it indicates that for the same  
functionality, the cost isn't all that different. I'm guessing that  
for protocols that have a common application mix of text and mostly- 
integer numeric values, that we're talking about 10 to 30%.

You do pay a price for very long tags, but only if you can't use gzip- 
style compression, which essentially does the text-to-code  
translation automatically and without penalty.

I'm not comparing other textual, non-XML approaches here. Frankly, I  
don't think they have a chance in the market place and the size  
difference for the ones I have seen isn't all that large.

A separate issue is the functionality. This is left as an exercise  
for another post.

Henning

_______________________________________________
XCON mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/xcon

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 13:09:45 -0000

As mentioned in the other post, this is interesting analysis, so I thought 
I'd try to add a bit more detail to it.

I've compared XML encodings with long tag names with a Megaco like encoding 
(in this case Lumas*) and a binary encoding (Lumas again).

*To justify the choice of Lumas encoding here, the original Lumas was an 
attempt to formalise the Megaco text encoding so that it could work for both 
text and binary.  So Lumas has a lot in common with Megaco encoding. 
Indeed, change "context = 123 { ..." to "context = { 123 ..." and you're 
there!

Note that although Megaco is a text encoding (one of them anyway!), it 
probably has more similarities in terms of message length with a binary 
encoding than XML encoding.  This is because in a number of cases Megaco has 
no tag, or short tag.

Tagging is obviously a big issue.  To get an idea of the length of typical 
XML tags I grepped (well Perled really) out the element and attribute names 
defined in the CPCP draft.  When doing this I got an average tag length of 
12.7 characters.  I noted that many of the names were of the form 
"a-long-tag-name", so I derived short tag names by taking the first letter 
of each sub-word.  The average of this was 2.08 characters.  (I've attached 
the Perl code used so that you can replicate my analysis if desired.)

As noted in the other mail, XSD does not lend itself to designing XML with 
small tags.  If everything is called X (or similar) the XSD doesn't describe 
the message very well.  So, although you can define an XSD with small tags, 
I think in practice you wouldn't!  (Aside: This is why Lumas allows for 
specifying a long descriptive name in the message definition, and specifying 
a short name for tagging.)

For the binary encoding I used Lumas's binary rules.  This uses variable 
length tag and length fields similar to how UTF-8 is variable length.  As 
UTF-8 is commonly part of XML I can't see that this is an issue.

So here's the results I came up with:

Bool:
XML - Typical name: 5(<></>) + 2*12(tag) + 1(data) = 30
XML - Short name: 5(<></>) + 2 * 2(tag) + 1(data) = 10
Lumas - Text Long Name: 2(note 1) + 12(tag) + 1(data) = 15
Lumas - Text short name: 2(note 1) + 2(tag) + 1(data) = 5
Lumas - Text untagged: 1(note 2) + 1(data) = 2
Lumas binary: 2(tag+len) + 0(data note 3) = 2
Ratio XML Long/Lumas long: 2
Ratio XML long/Lumas short: 6
Ratio XML short/Lumas short: 2
Ratio XML long/binary: 15
Ratio XML short/binary: 15

16 bit int:
XML - Typical name: 5(<></>) + 2*12(tag) + 5(data) = 34
XML - Short name: 5(<></>) + 2 * 2(tag) + 5(data) = 14
Lumas - Text Long Name: 2(note 1) + 12(tag) + 5(data) = 19
Lumas - Text short name: 2(note 1) + 2(tag) + 5(data) = 9
Lumas - Text untagged: 1(note 2) + 5(data) = 6
Lumas binary: 2(tag+len) + 2(data) = 4
Ratio XML Long/Lumas long: 1.78
Ratio XML long/Lumas short: 3.7
Ratio XML short/Lumas short: 1.5
Ratio XML long/binary: 8.5
Ratio XML short/binary: 3.5

32 bit int:
XML - Typical name: 5(<></>) + 2*12(tag) + 10(data) = 39
XML - Short name: 5(<></>) + 2 * 2(tag) + 10(data) = 19
Lumas - Text Long Name: 2(note 1) + 12(tag) + 10(data) = 24
Lumas - Text short name: 2(note 1) + 2(tag) + 10(data) = 14
Lumas - Text untagged: 1(note 2) + 10(data) = 11
Lumas binary: 2(tag+len) + 4(data) = 6
Ratio XML Long/Lumas long: 1.6
Ratio XML long/Lumas short: 2.7
Ratio XML short/Lumas short: 1.3
Ratio XML long/binary: 6.5
Ratio XML short/binary: 3.1

string: http://example.com/conference/on (length = 32)
XML - Typical name: 5(<></>) + 2*12(tag) + 32(data) = 61
XML - Short name: 5(<></>) + 2 * 2(tag) + 32(data) = 41
Lumas - Text Long Name: 2(note 1) + 12(tag) + 2(quoting) + 32(data) = 48
Lumas - Text short name: 2(note 1) + 2(tag) + 2(quoting) + 32(data) = 38
Lumas - Text untagged: 1(note 2) + 2(quoting) + 32(data) = 35
Lumas binary: 2(tag+len) + 32(data) = 34
Ratio XML Long/Lumas long: 1.27
Ratio XML long/Lumas short: 1.6
Ratio XML short/Lumas short: 1.07
Ratio XML long/binary: 1.79
Ratio XML short/binary: 1.2

string: Sally (length = 5)
XML - Typical name: 5(<></>) + 2*12(tag) + 5(data) = 34
XML - Short name: 5(<></>) + 2 * 2(tag) + 5(data) = 14
Lumas - Text Long Name: 2(note 1) + 12(tag) + 2(quoting) + 5(data) = 21
Lumas - Text short name: 2(note 1) + 2(tag) + 2(quoting) + 5(data) = 11
Lumas - Text untagged: 1(note 2) + 5(data) = 6
Lumas binary: 2(tag+len) + 5(data) = 7
Ratio XML Long/Lumas long: 1.6
Ratio XML long/Lumas short: 3.0
Ratio XML short/Lumas short: 1.27
Ratio XML long/binary: 4.8
Ratio XML short/binary: 2

note 1: Lumas needs an = sign and a minimum of 1 white space char to 
separate values.

note 2: Lumas needs a minimum of 1 white space char to separate values.

note 3: The Lumas binary encoding encodes a bool into 2 bytes using a 
special length code.

Of course the overall message saving depends on how often each of these 
types comes up.

To me the analysis shows that a binary encoding is typically very much 
shorter than a typical XML encoding (with long tag names etc.).

BUT: Most of this gain is realised by using a Megaco like text encoding. 
(This is captured by the XML long/Lumas short ratio.)  As this is what Lumas 
offers, rather than inventing yet another message encoding method, I would 
say that the debate should be between an XML encoding and a Lumas text 
encoding rather than between an XML encoding and a binary encoding!

Pete.
--
=============================================
Pete Cordell
Tech-Know-Ware Ltd
                         for XML to C++ data binding visit
                         http://www.tech-know-ware.com/lmx
                         (or http://www.xml2cpp.com)
=============================================

----- Original Message ----- 
From: "Henning Schulzrinne" <[email-masked]>
To: "XCON-IETF" <[email-masked]>
Sent: Friday, March 17, 2006 4:19 AM
Subject: [XCON] On encodings

> Since we're on the perennially fun topic of protocol encodings, maybe  we 
> can get past the stereotypes and try to do an engineering  comparison, or 
> at least an attempt at an approximation. Roughly  speaking, an XML element 
> is equivalent in functionality to a TLV  object: it contains a label 
> (element tag and the 'type' field),  length or delimiter fields to figure 
> out the boundaries, and the  actual data (value). ASN.1 BER is close 
> enough to TLV for this apples- to-pears comparison.
>
> predictions on the need to extend the protocol. For XML, you have  <tag 
> </tag>, thus, 4 bytes plus whatever length you make the tag  from 'X' to 
> Thus, the basic overhead is fairly similar if you choose small  element 
> names. Since numeric labels aren't self-describing at all,  this only 
> seems like a fair comparison.
>
> The big difference is in the datatypes. Clearly, if you have long  binary 
> opaque data, base64 encoding typical for XML costs you about 30%.
>
> cases, XML integers will be shorter since TLV always has to allocate  the 
> maximum range, presumably 32 bits for integers and 32 (single  precision) 
> or 64 bits for floats. You can build custom fixed-point  types in binary 
> encodings, but they are a pain to get right (All  kinds of funny things 
> happen because of signed vs. unsigned issues,  for example. Java 
> programmers will hate you if you try this...)
>
> Strings are obviously the same size in either TLV or XML, except for  the 
> use of delimiters (such as ") or length fields, but that's a wash  at 
> roughly 2 bytes each.
>
> The one big difference for XML is the namespace declaration. The  impact 
> of that depends on the number of namespaces and the size of  the overall 
> document. This is also hard to compare since they give  you proprietary 
> extensions without the kludges like vendor IDs that  binary protocols have 
> to go through (unless they choose similar Java- style or domain-style 
> tags). Indeed, for namespace-style extensions  that aren't IANA 
> registered, TLV is usually a pain. You end up with  either per-TLV long 
> labels or re-invent the XML-style indirection  approach.
>
> functionality, the cost isn't all that different. I'm guessing that  for 
> protocols that have a common application mix of text and mostly- integer 
> numeric values, that we're talking about 10 to 30%.
>
> style compression, which essentially does the text-to-code  translation 
> automatically and without penalty.
>
> A separate issue is the functionality. This is left as an exercise  for 
> another post.
>
> Henning
>

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 12:39:59 -0000

This is interesting analysis, so I thought I'd try to add a bit more detail 
to it.

I've got some minor observations on the original post here, and some more 
detailed analysis in a separate post.

----- Original Message From: "Henning Schulzrinne" <[email-masked]>> 
For XML, you have

> <tag  </tag>, thus, 4 bytes plus whatever length you make the tag

Should be <tag></tag> e.g. 5 bytes.

> seems like a fair comparison.

My observation on this is that XSD does not lend itself to designing XML 
with small tags.  If everything is called X (or similar) the XSD doesn't 
describe the message very well.  So, although you can define an XSD with 
small tags, I think in practice you wouldn't!  (Aside: This is why Lumas 
allows for specifying a long descriptive name in the message definition, and 
specifying a short name for tagging.)

> maximum range,

This depends significantly on the distribution of the numbers within the 
range.  If you have a 16 bit number, only the numbers 0-9 will be shorter 
than the binary encoding (not that I'm pushing binary, as you will see. 
Just trying to get the facts straight.)  This is 0.01% of the time _if_ the 
numbers are evenly distributed.  If they are evenly distributed (which is a 
big assumption) text will be longer in 98% of the cases.  For 32 bit values, 
text will be longer in 99.999% of the cases.  (again, distribution is a big 
issue here.)

> automatically and without penalty.

gzip isn't free.  It costs on small devices and servers handling a lots of 
connections.  Hence I don't think you can say it is without penalty.

Pete.

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 17 Mar 2006 10:23:38 -0500

> This depends significantly on the distribution of the numbers  
> within the range.  If you have a 16 bit number, only the numbers  
> 0-9 will be shorter than the

Certainly. In most cases of interest here, numbers tend to be small,  
with exceptions.

> binary encoding (not that I'm pushing binary, as you will see. Just  
> trying to get the facts straight.)  This is 0.01% of the time _if_  
> the numbers are evenly distributed.  If they are evenly distributed  
> (which is a big assumption) text will be longer in 98% of the  
> cases.  For 32 bit values, text will be longer in 99.999% of the  
> cases.  (again, distribution is a big issue here.)
>
>> You do pay a price for very long tags, but only if you can't use  
>> gzip- style compression, which essentially does the text-to-code   
> gzip isn't free.  It costs on small devices and servers handling a  
> lots of connections.  Hence I don't think you can say it is without  
> penalty.
>

Sure. Let's design systems that work on vacuum-tube computers. This  
is getting silly - the rest of the world does just fine with TLS and  
three-tier web service architectures, and we worry about gzip  
overhead on a server that handles echo cancellation and video  
transcoding. Maybe there is a corollary here to nethead vs. bellhead:  
namely those who have last programmed on a large scale around 1980  
vs. the rest of the world that actually worries about programmer  
productivity and fitting into the computing infrastructure of the  
21st century.

Henning

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 17 Mar 2006 10:33:56 -0500

Pete,

Thanks for the summary. I'll just add that having the IETF, in a  
small niche of its portfolio, go out and invent a new text encoding  
that nobody else uses and is likely to use (sorry...) seems just  
plain hubris. We're no longer in the world of one-off systems where  
we build every system from scratch as if the only tool in our tool  
kit was a C compiler (I suspect some people here would actually  
prefer assembler...).

Every new encoding means new libraries, means new possibilities for  
buffer overflows, new porting for every language from C to C# to PHP  
to Java to Perl to Ruby, new training, new debugging tools and new  
interoperability testing. I can choose among dozens of XML and SOAP  
libraries, maintained by the largest IT corporations on the planet if  
that's my choice, and a corresponding number of tools, from open  
source to XMLspy and kin. Why in the world would a sane developer  
create their own new niche encoding when the rest of their system is  
going to be built using modern software tools and will already have  
to support XML anyway?

I'm afraid I'm not going to be spending time on Lumas analysis, I'm  
sorry to say. The IETF is a tiny part of the software universe and  
needs to learn to play nice with the 99.9% of the developers that  
don't show up at IETF meetings.

Henning

On Mar 17, 2006, at 8:09 AM, Pete Cordell wrote:

> As mentioned in the other post, this is interesting analysis, so I  
> thought I'd try to add a bit more detail to it.
>
> I've compared XML encodings with long tag names with a Megaco like  
> encoding (in this case Lumas*) and a binary encoding (Lumas again).
>
> *To justify the choice of Lumas encoding here, the original Lumas  
> was an attempt to formalise the Megaco text encoding so that it  
> could work for both text and binary.  So Lumas has a lot in common  
> with Megaco encoding. Indeed, change "context = 123 { ..." to  
> "context = { 123 ..." and you're there!
>
> Note that although Megaco is a text encoding (one of them anyway!),  
> it probably has more similarities in terms of message length with a  
> binary encoding than XML encoding.  This is because in a number of  
> cases Megaco has no tag, or short tag.
>
> Tagging is obviously a big issue.  To get an idea of the length of  
> typical XML tags I grepped (well Perled really) out the element and  
> attribute names defined in the CPCP draft.  When doing this I got  
> an average tag length of 12.7 characters.  I noted that many of the  
> names were of the form "a-long-tag-name", so I derived short tag  
> names by taking the first letter of each sub-word.  The average of  
> this was 2.08 characters.  (I've attached the Perl code used so  
> that you can replicate my analysis if desired.)
>
> As noted in the other mail, XSD does not lend itself to designing  
> XML with small tags.  If everything is called X (or similar) the  
> XSD doesn't describe the message very well.  So, although you can  
> define an XSD with small tags, I think in practice you wouldn't!   
> (Aside: This is why Lumas allows for specifying a long descriptive  
> name in the message definition, and specifying a short name for  
> tagging.)
>
>
> For the binary encoding I used Lumas's binary rules.  This uses  
> variable length tag and length fields similar to how UTF-8 is  
> variable length.  As UTF-8 is commonly part of XML I can't see that  
> this is an issue.
>
> note 1: Lumas needs an = sign and a minimum of 1 white space char  
> to separate values.
>
> note 3: The Lumas binary encoding encodes a bool into 2 bytes using  
> a special length code.
>
> Of course the overall message saving depends on how often each of  
> these types comes up.
>
> To me the analysis shows that a binary encoding is typically very  
> much shorter than a typical XML encoding (with long tag names etc.).
>
> BUT: Most of this gain is realised by using a Megaco like text  
> encoding. (This is captured by the XML long/Lumas short ratio.)  As  
> this is what Lumas offers, rather than inventing yet another  
> message encoding method, I would say that the debate should be  
> between an XML encoding and a Lumas text encoding rather than  
> between an XML encoding and a binary encoding!
>
> Pete.
> --
> ----- Original Message ----- From: "Henning Schulzrinne"  
> <[email-masked]>
>> Since we're on the perennially fun topic of protocol encodings,  
>> maybe  we can get past the stereotypes and try to do an  
>> engineering  comparison, or at least an attempt at an  
>> approximation. Roughly  speaking, an XML element is equivalent in  
>> functionality to a TLV  object: it contains a label (element tag  
>> and the 'type' field),  length or delimiter fields to figure out  
>> the boundaries, and the  actual data (value). ASN.1 BER is close  
>> 'WeLikeToPutTheWholeRFCIntoTheTagSoThatTheImplementerDoesntHaveToRead 
>> The RFC'.
>>
>> For length, this is already counted above for XML; for TLV, it  
>> would presumably be again 16 or 32 bits.
>>
>> element names. Since numeric labels aren't self-describing at  
>> all,  this only seems like a fair comparison.
>>
>> Integers and floats are trickier: Unless you believe in ASN.1- 
>> style variable-length encoding, it is quite possible that in many  
>> practical cases, XML integers will be shorter since TLV always has  
>> to allocate  the maximum range, presumably 32 bits for integers  
>> and 32 (single  precision) or 64 bits for floats. You can build  
>> custom fixed-point  types in binary encodings, but they are a pain  
>> to get right (All  kinds of funny things happen because of signed  
>> vs. unsigned issues,  for example. Java programmers will hate you  
>> if you try this...)
>>
>> Strings are obviously the same size in either TLV or XML, except  
>> for  the use of delimiters (such as ") or length fields, but  
>> that's a wash  at roughly 2 bytes each.
>>
>> impact of that depends on the number of namespaces and the size  
>> of  the overall document. This is also hard to compare since they  
>> give  you proprietary extensions without the kludges like vendor  
>> IDs that  binary protocols have to go through (unless they choose  
>> similar Java- style or domain-style tags). Indeed, for namespace- 
>> style extensions  that aren't IANA registered, TLV is usually a  
>> pain. You end up with  either per-TLV long labels or re-invent the  
>> XML-style indirection  approach.
>>
>> functionality, the cost isn't all that different. I'm guessing  
>> that  for protocols that have a common application mix of text and  
>> mostly- integer numeric values, that we're talking about 10 to 30%.
>>
>> I'm not comparing other textual, non-XML approaches here. Frankly,  
>> I don't think they have a chance in the market place and the size  
>> A separate issue is the functionality. This is left as an  
>> exercise  for another post.
>>
>> Henning
>>
>> <get-xsd-names.pl>

---

From: Adam Roach <[email-masked]>
Date: Fri, 17 Mar 2006 09:52:52 -0600

Henning Schulzrinne wrote:

> transcoding.

Mobile phones.

/a

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 16:35:16 -0000

----- Original Message ----- 
>> gzip isn't free.  It costs on small devices and servers handling a  lots 
>> of connections.  Hence I don't think you can say it is without  penalty.
>
> Sure. Let's design systems that work on vacuum-tube computers. This  is 
> getting silly - the rest of the world does just fine with TLS and 
> three-tier web service architectures,

The majority is without TLS so this is bogus.

> and we worry about gzip  overhead on a server that handles echo 
> cancellation and video  transcoding.

Not necessarily on the same hardware.

> Maybe there is a corollary here to nethead vs. bellhead:  namely those who 
> have last programmed on a large scale around 1980  vs. the rest of the 
> world that actually worries about programmer  productivity and fitting 
> into the computing infrastructure of the  21st century.

Are you talking about yourself here?  Someone proposing XML + SOAP + TLS + 
gzip + whatever else they can wrap things into doesn't sound like someone 
who is worried by programmer productivity to me.  Sounds like design with 
PowerPoint to me!

Pete.

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 16:32:02 -0000

----- Original Message From: "Henning Schulzrinne" <[email-masked]>

> Thanks for the summary. I'll just add that having the IETF, in a  small 
> niche of its portfolio, go out and invent a new text encoding  that nobody 
> else uses and is likely to use (sorry...) seems just  plain hubris. We're 
> no longer in the world of one-off systems where  we build every system 
> from scratch as if the only tool in our tool  kit was a C compiler (I 
> suspect some people here would actually  prefer assembler...).
>
> buffer overflows, new porting for every language from C to C# to PHP  to 
> Java to Perl to Ruby, new training, new debugging tools and new 
> that's my choice, and a corresponding number of tools, from open  source 
> to XMLspy and kin. Why in the world would a sane developer  create their 
> own new niche encoding when the rest of their system is  going to be built 
> using modern software tools and will already have  to support XML anyway?
>
> I'm afraid I'm not going to be spending time on Lumas analysis, I'm  sorry 
> to say. The IETF is a tiny part of the software universe and  needs to 
> learn to play nice with the 99.9% of the developers that  don't show up at 
> IETF meetings.

But people still seem to be happy to invent new binary encodings such as 
BFCP, AVPs, and new ABNF grammars such as MRCP that have all the problems 
that you mention above.  Why is that?

People are even pushing JSON, when they have access to all the XML tools 
that you mention (http://en.wikipedia.org/wiki/JSON).  This is used 
extensively by the likes of Google and Yahoo even though they have all the 
resources to run XML services.

Like you, I would like the encoding debate to go away as well.  Your 
approach is to pretend that there's no problem.  My approach is to design a 
system that has the high-level design benefits of XSD, the debuggability of 
a text protocol, and the compactness of a binary protocol.  With Lumas there 
would be nothing to debate, and we could get on with more interesting 
things.

It seems I also have more faith in the development capabilities of 
developers.  If Microsoft (and Borland) can knock out C# then they're not 
going to be bothered by Lumas.  Heck they can get the C and Perl versions 
from my web site now.

Don't get me wrong.  XML is great.  But it is not the be all and end all. 
In programming I use C++ for some things, Perl for other things, PHP/SQL for 
yet more things, C# for other other things.  It depends on the problem I'm 
tackling.  Why shouldn't I have the same choices for message definition?

Pete.
.

---

From: "Sean Olson" <[email-masked]>
Date: Fri, 17 Mar 2006 08:50:26 -0800

To be fair, JSON is pushed because of its ubiquitous support in
Javascript-enabled browsers. LUMAS doesn't have the same appeal because
it would be yet another new encoding. XML is not perfect and not
something I would try to use to solve all problems. Yet, it's use in the
conferencing event package, presence, and other SIP/SIMPLE/SIPPING
protocols makes it an attractive candidate for re-use. 

-----Original Message-----
From: Pete Cordell [mailto:[email-masked]] 
Sent: Friday, March 17, 2006 8:32 AM
To: Henning Schulzrinne
Cc: XCON-IETF
Subject: Re: [XCON] On encodings

prefer assembler...).
to support XML anyway?

But people still seem to be happy to invent new binary encodings such as
BFCP, AVPs, and new ABNF grammars such as MRCP that have all the
problems that you mention above.  Why is that?

People are even pushing JSON, when they have access to all the XML tools
that you mention (http://en.wikipedia.org/wiki/JSON).  This is used
extensively by the likes of Google and Yahoo even though they have all
the resources to run XML services.

Like you, I would like the encoding debate to go away as well.  Your
approach is to pretend that there's no problem.  My approach is to
design a system that has the high-level design benefits of XSD, the
debuggability of a text protocol, and the compactness of a binary
protocol.  With Lumas there would be nothing to debate, and we could get
on with more interesting things.

It seems I also have more faith in the development capabilities of
developers.  If Microsoft (and Borland) can knock out C# then they're
not going to be bothered by Lumas.  Heck they can get the C and Perl
versions from my web site now.

Don't get me wrong.  XML is great.  But it is not the be all and end
all. 
In programming I use C++ for some things, Perl for other things, PHP/SQL
for yet more things, C# for other other things.  It depends on the
problem I'm tackling.  Why shouldn't I have the same choices for message
definition?

Pete.

---

From: Henning Schulzrinne <[email-masked]>
Date: Fri, 17 Mar 2006 12:07:15 -0500

Right. Mobile phones have processors running at several hundred MHz, 
i.e., roughly equivalent to desktop processors in the late 90s. I don't 
think they'll break a sweat encoding 200 bytes in gzip.

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 17:34:00 -0000

----- Original Message From: "Sean Olson" <[email-masked]>

> protocols makes it an attractive candidate for re-use.

My position is that if message size is not a problem, then you might as well 
use XML, for all the reasons that Henning mentions.  However, I'm hearing 
that there are worries about message size etc., and if that is the case, 
let's not invent another application specific message encoding, but adopt a 
more general one that enhances the programmers tool chest and can be used in 
other applications.

I mentioned JSON as an example of a precedent where a community well versed 
in XML has considered adopting an alternative approach in some cases.

Pete.

---

From: "Sean Olson" <[email-masked]>
Date: Fri, 17 Mar 2006 10:22:55 -0800

The reason JSON was chosen was to re-use an existing toolset inside
common browsers (Javascript). You'll notice that a brand new binary
encoding was not chosen in this context (I'm not saying this is a
perfect analogy for XCON) 

-----Original Message-----
From: Pete Cordell [mailto:[email-masked]] 
Sent: Friday, March 17, 2006 9:34 AM
To: Sean Olson
Cc: XCON-IETF
Subject: Re: [XCON] On encodings

> Javascript-enabled browsers. LUMAS doesn't have the same appeal 
> because it would be yet another new encoding. XML is not perfect and 
> not something I would try to use to solve all problems. Yet, it's use 
> in the conferencing event package, presence, and other 
> SIP/SIMPLE/SIPPING protocols makes it an attractive candidate for
re-use.

My position is that if message size is not a problem, then you might as
well use XML, for all the reasons that Henning mentions.  However, I'm
hearing that there are worries about message size etc., and if that is
the case, let's not invent another application specific message
encoding, but adopt a more general one that enhances the programmers
tool chest and can be used in other applications.

I mentioned JSON as an example of a precedent where a community well
versed in XML has considered adopting an alternative approach in some
cases.

Pete.

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 18:23:51 -0000

I for one need more numbers to be able to make a judgement on this.  Does 
anybody have an idea of:

- how many messages are likely to be sent to/from a client in an active 
conference?  (is it messages per second, or seconds per message?)

- does the messaging load for the client increase with conference size, or 
does it stay flat?

- How many clients is a server expected to interface to at one time?

Thanks,

Pete.

----- Original Message ----- 
From: "Henning Schulzrinne" <[email-masked]>
To: "Adam Roach" <[email-masked]>
Cc: "Pete Cordell" <[email-masked]>; "XCON-IETF" <[email-masked]>
Sent: Friday, March 17, 2006 5:07 PM
Subject: Re: [XCON] On encodings - observations

> Right. Mobile phones have processors running at several hundred MHz, i.e., 
> roughly equivalent to desktop processors in the late 90s. I don't think 
> they'll break a sweat encoding 200 bytes in gzip.
>
> Adam Roach wrote:
>>> three-tier web service architectures, and we worry about gzip  overhead 
>>> on a server that handles echo cancellation and video  transcoding.
>>
>>
>> Mobile phones.
>>
>> /a
>

---

From: Adam Roach <[email-masked]>
Date: Fri, 17 Mar 2006 12:35:34 -0600

[not as chair]

Pete Cordell wrote:

> My position is that if message size is not a problem, then you might 
> as well use XML, for all the reasons that Henning mentions.  However, 
> I'm hearing that there are worries about message size etc., and if 
> that is the case, let's not invent another application specific 
> message encoding...

CSCP encoding is BFCP, which is both likely to be implemented in 
conferencing systems and in the RFC editor's queue.

/a

---

From: "Pete Cordell" <[email-masked]>
Date: Fri, 17 Mar 2006 18:53:07 -0000

> perfect analogy for XCON) 

And I'm not pushing binary either.  Quite the opposite.

Pete.

---

From: Adam Roach <[email-masked]>
Date: Fri, 17 Mar 2006 13:14:29 -0600

[not as chair]

Henning Schulzrinne wrote:

> i.e., roughly equivalent to desktop processors in the late 90s. I 
> don't think they'll break a sweat encoding 200 bytes in gzip.

Your assertion is not completely well founded: many of the phones sold 
today are running in the tens of MHz, not hundreds. Even for smart 
phones, it's not unusual to get ahold of terminals based on 33 MHz 
DragonBall VZ processors. (If you like, I'll ask a few questions during 
the XCON meeting next week and probably find a dozen or so people in the 
meeting actually in possession of one). I have run tests on these 
specific processors using Deflate and 1000-byte messages. The resultant 
compression time is on the order of 300 - 500 ms. For smaller messages 
-- say, 200 to 300 bytes or so -- you're looking at about 100 ms to 
compress such messages.

For messages in that size range, you'd be hard pressed to make up 100ms 
of latency by sending fewer bits.

When I did my analysis of this topic, my conclusions were: on processors 
at this speed, and using the network data speeds we discussed before, 
using Deflate compression is pretty much a wash (in terms of latency) 
for messages of any size. The amount of time spent compressing a message 
was always nearly equal the amount of time you saved in transmission. 
There are some much lighter-weight, *patented* compression algorithms 
(e.g. v.44) that are orders of magnitude faster, but I'm not certain we 
want to mandate a patented compression algorithm for all users of the 
conference control protocol.

/a