Subject: A reminder we should think about congestion control at some point

From: Tim Bray <[email-masked]>
Date: Fri, 16 Jul 2004 17:21:41 -0700

http://www.infoworld.com/article/04/07/16/29OPconnection_1.html? 
source=rss&url=http://www.infoworld.com/article/04/07/16/ 
29OPconnection_1.html

I'm not sure this is *really* a problem, in that we might decide that  
existing HTTP caching & traffic management techniques will suffice for  
Atom.  But we should be conscious of it. -Tim

---

From: Mark Pilgrim <[email-masked]>
Date: Fri, 16 Jul 2004 20:46:55 -0400

> Atom.  But we should be conscious of it. -Tim

They could start by using HTTP properly.  There server does not
support Etags, Last-Modified headers, or gzip compression.  No wonder
they have bandwidth problems.  (They're also serving their feeds as
text/html, but never mind that.)

haven:~ mark$ curl -sI http://www.infoworld.com/rss/news.xml
HTTP/1.1 200 OK
Date: Sat, 17 Jul 2004 00:45:10 GMT
Server: Apache
Accept-Ranges: bytes
Content-Type: text/html; charset=UTF-8

-- 
Cheers,
-Mark

---

From: Sam Ruby <[email-masked]>
Date: Fri, 16 Jul 2004 21:01:28 -0400

> source=rss&url=http://www.infoworld.com/article/04/07/16/ 

http://tinyurl.com/6mdvo

- Sam Ruby

---

From: Sam Ruby <[email-masked]>
Date: Fri, 16 Jul 2004 21:10:42 -0400

Mark Pilgrim wrote:

> they have bandwidth problems.

Agreed that using HTTP more completely is a requirement before 
attempting to solve any larger problems.  Our job is to make sure that 
someone implementing Atom is aware of everything that they need in order 
to do it right.  A few months ago, I started to take a stab at this:

http://intertwingly.net/stories/2004/04/08/atomguide.html

It is amazing how quickly things get complicated when you add together a 
few "simple" protocols, each with their own options.  The next thing I 
started to tackle was I18N.  From there, I intended to go to URIs...

- Sam Ruby

---

From: Jonas Galvez <[email-masked]>
Date: Fri, 16 Jul 2004 22:35:10 -0300

Tim Bray wrote:
> existing HTTP caching & traffic management techniques will suffice
> for Atom.  But we should be conscious of it. -Tim

I've been studying the HTTP caching mechanism and I'm really amazed by
the performance improvement. I guess gzip compression is something that
too few people are using. I host about a dozen of scrapped feeds for
personal use (but I share them with various friends), and recently I
noticed they were eating about 300mb of my bandwidth. I've since then
started using cgi_buffer (a Python library that encapsulates all HTTP
caching techniques plus gzip compression) to generate those feeds, and
the bandwidth use reduced drastically.

So I think the solution already exists. But not everyone is aware of it.
Perhaps it would be ideal to have a separate document, some sort of
guidelines, with detailed documentation (and preferrably friendly,
easy-to-read documentation) on how to implement these methods.

Since I've been already dedicating a considerable amount of my time to
that subject (it's important to my current work), I'd gladly volunteer
to write a draft about it with the best of my non-native English. But I
have the feeling that that document (a friendly, easy-to-read,
syndication-specific, HTTP caching tutorial) already exists somewhere...

\\ jonas galvez
// jonasgalvez.com

---

From: Dare Obasanjo <[email-masked]>
Date: Fri, 16 Jul 2004 18:58:55 -0700 (PDT)

--- Tim Bray <[email-masked]> wrote:
> Atom.  But we should be conscious of it. -Tim

Testing their feed with
http://www.rexswain.com/cgi-bin/httpview.cgi I see
that Infoworld neither supports GZip encoding nor HTTP
conditional GET. No wonder their bandwidth usage is so
high. Using both techniques should reduce their
bandwidth usage by at *least* a factor of 10. 

Almost every time I see someone complain about how RSS
wastes bandwidth, I check their RSS feed and find they
aren't using basic HTTP techniques for saving
bandwidth which are supported by ALL the major RSS
aggregators. Until we find that existing solutions are
inadequate it seems like premature optimization to
start talking about even more sophisticated bandwidth
management techniques specific to syndication feeds. 

=====
THINGS TO DO IF I BECOME AN EVIL OVERLORD #95
My dungeon will have its own qualified medical staff complete with bodyguards. That way if a prisoner becomes sick and his cellmate tells the guard it's an emergency, the guard will fetch a trauma team instead of opening up the cell for a look.

__________________________________
Do you Yahoo!?
Vote for the stars of Yahoo!'s next ad campaign!
http://advision.webevents.yahoo.com/yahoo/votelifeengine/

---

From: Walter Underwood <[email-masked]>
Date: Mon, 19 Jul 2004 13:40:48 -0700

> bandwidth usage by at *least* a factor of 10.

For RSS, but not for the rest of the site. Public sites just
can't send cachable HTML because they need to rotate the ads
and count ad impressions. Cached pages mean lost ad revenue.

For ad-free feeds, caching is excellent. Then put in an Expires
header that shows when it would be smart to check again.

wunder
--
Walter Underwood
Principal Architect
Verity Ultraseek

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 19 Jul 2004 17:37:48 -0400

Walter Underwood wrote:
> Public sites just can't send cachable HTML because they need
> to rotate the ads and count ad impressions. Cached pages mean
> lost ad revenue.
	What is the policy on non-normative text for the Atom RFCs?
	I would like to write a  Pace for some "Best Practice" text that
recommends that people should not modify atom:id, atom:modified, or
atom:issued if they are inserting or modifying ad content embedded in atom
entries. The reason for this is to prevent aggregators from being fooled
into thinking that an entry is "new" when the only change is a modification
to an embedded ad. (Note: It is probably in the advertisers' best interest
to follow the recommended practice since doing otherwise will tend to
irritate users by exposing them to many duplicate entries.)
	Is it appropriate to include such non-normative "Best Practice"
guidance in these RFC's? Should I write a Pace on this?

		bob wyman