RE: High volume feeds - one possible solution

["Bob Wyman" <bob@xxxxxxxx>]

Danny Ayers wrote:
>I've been going round and round with thoughts of HTTP chaining and
> all sorts. But a possible relatively low-effort approach finally
> occured to me:
> Split the feed into (say) 4 smaller feeds, which are rotated 
>(say) hourly -
	Danny, the system you are proposing would work... There are some
rough edges, but I don't doubt that they could be smoothed out with a
bit more discussion. However, I'm also sure that I really, really hope
we don't go this way!
	I must admit that my perspective is a bit different from that of
most folks. My system reads well over 3 million RSS and Atom files,
often multiple times, every day and that number is growing rapidly. I
would much prefer an RFC3229 based system since it means that much of
the bookeeping to get proper deltas computed is left to the server --
rather than forcing us (as a reader) to do the link crawling that your
proposal requires. I would also prefer an RFC3229 based solution since
it means that I don't have to open up multiple connections with remote
sites or depend on HTTP persistent connections (they are not always
available...)
	What you are proposing would work, but I think it would make
life much more complex for readers that consume large numbers of feeds.
Can you explain why your proposal is clearly superior to using an
RFC3229 based method? If we accept Sam Ruby's suggestion that
entry-oriented instance-manipulation be turned on by default when
serving Atom, can you still define a clear benefit to your approach?

		bob wyman