Re: Bandwidth concerns

["Bill Kearney" <wkearney99@xxxxxxxxxxx>]

Bearing in mind, however, that such operations would require dynamic code
running on the server.  A *great many*, if not the majority, of things providing
RSS run from static files.  That and the load caused by static files is
significantly more predictable than dynamic scripts.

So the question is how is scalability impacted?  I'd venture a guess that
effective use of eTags, gzip compression and timestamp checking is going to end
up being a lot more scalable that dynamic delivery.  The basic premise being
that a static file can be served up a whole lot quicker than it would take for
it the calculate what actual content is needed and to then compress and deliver
it.

For headline feeds I just can't see it being worth the bother to get into
incremental delivery.

As for large content feeds I'd rather see things go the route of static
manifests linked to external full-content documents.  This way the headline feed
continues to be lightweight, scalable and cache-friendly.   If a site wants to
offer full content then stuff the data into a separate XML file and link to it
from the headline feed.  This would also make it a very simple proposition to
retrieve entire ranges of content.  Query for a manifest of items and then pull
them as needed.  Also bearing in mind that since these too are individual
resources they also benefit from the things like eTags, gzip and header
timetamps.  As in, as for a range of items from date X to date Y but if you're
already got some of them they don't get pulled again.

-Bill Kearney

----- Original Message -----
From: "Duncan Wilcox" <duncan@xxxxxxxxx>
To: <atom-syntax@xxxxxxx>
Sent: Monday, August 11, 2003 5:34 AM
Subject: Bandwidth concerns


>
> Because of the way I read feeds I love full content feeds. Many RSS
> feeds include a complete description or content:encoded or xhtml:body,
> some don't. People who avoid putting full content do so for two
> reasons, reader tracking and bandwidth concerns.
>
> I respect who wants readers to click through to view the full article,
> maybe something can be done for who wants to save on bandwidth, to
> encourage them to put full content in atom feeds.
>
> I wrote something up in http://www.intertwingly.net/wiki/pie/EchoFeed
> (for lack of a better place).
>
> Basically I suggest requiring clients to behave somewhat like nntp
> clients, fetching only newer entries, with fallback to "standard" last
> 15 items if the server only has a static file. (I guess it needs some
> work for modified entries, but you get the idea)
>
> This would be in addition to ETags and content compression support, of
> course.