Subject: RE: non-solutions for blogs.msdn.com: (was RSS consumes too much bandwidth)

From: Graham Parks <[email-masked]>
Date: Fri, 10 Sep 2004 12:10:54 -0700

On Friday, September 10, 2004, at 11:16AM, Bob Wyman <[email-masked]> wrote:

> "massive POLLING for a resource does NOT SCALE."

This is bullshit. Things are said to "NOT SCALE" when the load grows logarithmically with the number of users. RSS grows linearly, and therefore, it scales perfectly. The real problem is that publishers seem to expect to become massively popular without their costs increasing.

(Also, the aggregators can get smart, the format doesn't have to: http://www.fondantfancies.com/shrook/distfaq.php )

Graham

---

From: Antone Roundy <[email-masked]>
Date: Fri, 10 Sep 2004 13:54:44 -0600

On Friday, September 10, 2004, at 01:10  PM, Graham Parks wrote:
> This is bullshit. Things are said to "NOT SCALE" when the load grows 
> logarithmically with the number of users. RSS grows linearly, and 
> therefore, it scales perfectly. The real problem is that publishers 
> seem to expect to become massively popular without their costs 
> increasing.

This reminds me of a one-frame comic I've periodically thought of 
drawing.  A guy in a business meeting is showing a profits chart and 
says: "The good news is that our profits are growing exponentially. The 
bad news is that the exponent is negative."  If we can design a system 
where the load grows more slowly than the number of users, that system 
will scale better than one where the load grows linearly.  A design 
that maximizes cacheability can accomplish that.

Maybe it's time to draw the comic.

Antone

---

From: "Bill Kearney" <[email-masked]>
Date: Fri, 10 Sep 2004 17:58:20 -0400

> This is bullshit.

+1 to that.  But Scoble's a link whore (by his own admission) and what
better way to pimp himself out than by flogging the obvious as if it were
some sort of controversy?  After all, isn't that what all good media outlets
chant as their mantra these days?

>Things are said to "NOT SCALE" when the load grows logarithmically with the
number of users.

And I'm sure we'll get a neverending thread about what is or isn't the right
mathematical term.

> The real problem is that publishers seem to expect to become
> massively popular without their costs increasing.

I'm always reminded of Claude Raines in Casablanca...

    "I'm shocked, shocked to imagine bandwidth being wasted here!"

What did they expect when the published those huge content-laden feeds?
That people wouldn't read it?  What did they expect when they didn't include
any scheduling data in the feed, that aggregators wouldn't fall back to
hourly downloads?  What did they expect when they didn't apply compression?
Or ETags, timestamps or actually useful UIDs?  "Shocked" indeed.

-Bill Kearney

---

From: "jt" <[email-masked]>
Date: Fri, 10 Sep 2004 18:13:07 -0400

Just to lighten it up a li'l.

| -----Original Message-----
| From: [email-masked]
| [mailto:[email-masked]]On Behalf Of Bill Kearney
| Sent: Friday, September 10, 2004 5:58 PM
| To: 'Atom Syntax'
| Subject: Re: non-solutions for blogs.msdn.com: (was RSS consumes too
| much bandwidth)

| I'm always reminded of Claude Raines in Casablanca...
|
|     "I'm shocked, shocked to imagine bandwidth being wasted here!"
|
| What did they expect when the published those huge content-laden feeds?
| That people wouldn't read it?  What did they expect when they
| didn't include
| any scheduling data in the feed, that aggregators wouldn't fall back to
| hourly downloads?  What did they expect when they didn't apply
| compression?
| Or ETags, timestamps or actually useful UIDs?  "Shocked" indeed.

Start(British humour)

I've not been following close enough.  But I was also "shocked" as I
wondered how many copies of the works of Shakespeare 1000 code-monkeys could
put out, given the means and no end..

No end to their 'free time' and no end to my shock and disbelief, either one
that is.

Hey, I'm sure some of the content is useful, but to complain about the
results that come along with EXACTLY WHAT YOU WANTED TO HAPPEN??  Robert's
math is suspect, even when the numbers add up.

Fame has a cost, and apparently the bandwidth is the ONLY THING that shows
up on M$ bottom line.

End(semi-tongue-in-cheek)

---

From: Danny Ayers <[email-masked]>
Date: Sat, 11 Sep 2004 11:50:11 +0200

> And I'm sure we'll get a neverending thread about what is or isn't the right

Not even tempted. To the nth degree.
 
>     "I'm shocked, shocked to imagine bandwidth being wasted here!"

C'mon Bill, this is the blogosphere - raise that to a Kurtz "The
horror! The horror!" ;-)

> Or ETags, timestamps or actually useful UIDs?  "Shocked" indeed.

Any idea how much people are actually using (effectively) stuff like
<ttl>? I had come to the conclusion these things were probably more
trouble that it's worth (& if needed use HTTP), but in cases like
MSDN's it might well be useful.

Cheers,
Danny.

-- 

http://dannyayers.com

---

From: James Robertson <[email-masked]>
Date: Sat, 11 Sep 2004 06:07:36 -0400

> > Or ETags, timestamps or actually useful UIDs?  "Shocked" indeed.

Well, this is why I do the aggregated feed here:

http://www.cincomsmalltalk.com/userblogs

differently.  Instead of having it update on every post, there's a 
background process that wakes up every three hours - which regenerates the 
master file.  The individual blogs have feeds that update as changes happen 
- the aggregated feed doesn't.  This way, most requests end up getting 
304s.  I suspect that MS could solve the problem in a similar fashion.

>MSDN's it might well be useful.

<Talk Small and Carry a Big Class Library>
James Robertson, Product Manager, Cincom Smalltalk
http://www.cincomsmalltalk.com/blog/blogView
[email-masked]