Re: High volume feeds - one possible solution

[Danny Ayers <danny.ayers@xxxxxxxxx>]

On Sat, 11 Sep 2004 14:43:02 -0400, Bob Wyman <bob@xxxxxxxx> wrote:
> Danny Ayers wrote:
> > The MSDN scenario has been bugging me. ... Split the feed into
> > (say) 4 smaller feeds, which are rotated (say) hourly - ...
> > for client-side tools that didn't support recombination, they
> > just get 4 separate feeds,
>         If you're going to increase the complexity of the server software in
> this manner, why just go the tiny further step of supporting RFC3229 delta
> feeds with an instance-manipulation method like "feed-delta" which, combined
> with Etag, could be used by the server to select the correct file to serve
> up? The method you propose pushes a good bit of complexity out to the client
> (i.e. the recombination problem and the problems of dealing with multiple
> feeds) when RFC3229 already provides a standard method to avoid this
> problem. There are far fewer servers then clients. If possible, complexity
> should be limited to server -- not pushed to the clients.

The main advantage of the approach I suggested is that it would be
possible to layer on top of current systems. I think perhaps you
exaggerate the client-side complexity. A client will already combine
data from document A at time X with data from document A at time Y, so
the step needed to combine data from document A with data from
document B (which will be essentially the same internally) can't be
that great, surely? The "problem of dealing with multiple feeds" has
already been solved by most aggregators ;-) Like I said, worst case is
a client sees multiple feeds - that would work *now*.

I've only skimmed RFC3229 so far, though appreciate that sending
deltas is the ideal solution to a problem like MSDN's. However the
impression I get is that implementation would involve more than a
"tiny further step". There appears to be quite a lot of new headers
and corresponding behaviours to take on board. Also I get a very mixed
message about proxy support - if the diff functionality is lost by a
large proportion of intermediaries, then there would be little net
gain.

I haven't yet figured out quite what would be needed to support this
in a typical system - say WordPress (PHP) on Apache serving, with a
desktop client like RssBandit (C#). Are standard libraries available
that could be plugged into such systems to provide delta support?
(Googling got me nowhere).

Cheers,
Danny.

-- 

http://dannyayers.com