Re: Is this really an LCUP show stopper?

[mcs@xxxxxxxxxxxx (Mark C Smith)]

John McMeeking wrote:
> My two bits worth:
>
> I favor the LDAP Sync approach over LCUP.  This is largely because changes
> to LDAP Sync appear to make LCUP a valid implementation of LDAP Sync, and
> it doesn't appear that LDAP sync would be significantly harder to
> implement, from either the server or client perspective.

I think a very careful reading of the two drafts does expose some 
important differences (and I admit I need to spend some more time 
carefully reading draft-zeilenga-ldup-sync-02.txt). But I generally 
agree that LDAP Sync includes much of what makes LCUP a good proposal.


> As I understand it, this discussion basically boils down to how a client
> finds out that an entry has left the result set and whether that imposes
> undo requirements on an implementation or results in excessive network
> traffic.  The choices being:
>
> 1.  Server sends "entry deleted" messages -- LCUP or LDAP Sync -- server
> must maintain history
>
> 2.  Server sends "entry present" messages for all entries in result set,
> client determines what is gone -- LDAP Sync -- no history required
>
> 3.  Reload required -- LCUP or LDAP Sync -- server doesn't have the
> necessary history, or changes are "too big"
>
> It seems to me that all these will all result in the client having the same
> content, so I don't see an issue with either approach not working.

Agreed, for some value of "working." As the recent discussions have 
shown, different approaches may work better in different situations 
(most involve a tradeoff between implementation complexity and 
performance or wire traffic).


> I think the notion of maintaining a history so that a server can send
> "entry deleted" messages is important.  Synchronizing a 100,000 entry
> directory with no (or limited) changes by sending 1000,000 entry present
> messages seems like poor behavior.  I believe that change to LDAP Sync was
> needed.
>
> It seems that LCUP could be implemented using LDAP Sync extended ops and
> controls.  A server implementation could always choose between options 1
> and 3.  And I think that would be a valid implementation of LDAP Sync.  The
> client, of course, would have to work properly with option 2, which might
> make testing the client problematic if the client and server vendors are
> the same.  Similarly, a server implementation could choose to always use
> options 2 and 3.

This is where we need to be careful. I am not convinced that option 2 is 
a good idea, at least not for the kinds of applications I have in mind. 
My guess -- and I admit this is only a guess -- is that the real 
thinking behind the addition of the refreshDeletes option in 
draft-zeilenga-ldup-sync-02.txt was to try to position LDAP Sync as a 
superset of LCUP, with the hope being that those who support LCUP will 
also support LDAP Sync. But I am not convinced we should shift 
implementation complexity to the client, and increase the on the wire 
traffic, by allowing server implementors to ignore the option 1 you 
describe above. It will simply be too easy for them to provide a really 
poor implementation of "a protocol that enables an LDAP client to 
synchronize with the content of a directory information tree (DIT) 
stored by an LDAP server and to be notified about the changes to that 
content." And then the protocol will not be widely adopted by client 
implementors, and then we will have solved no important problems.


> I don't think that either LDAP Sync or LCUP will be significantly harder to
> implement (either server or client side) if one accepts the notion that
> some history is desirable.  As stated earlier, I think that history is
> necessary to a well-behaved synchronization mechanism.

Which is why I for one have no problem (and believe it may in fact be a 
good idea) to encourage reasonable implementations to store some history.



> I have not been able to come to any firm position on the need (or lack
> thereof) for the "entry present" mechanism.  It is clear that if the data
> being synchronized is "large", and a change to entries and / or meta-data
> results in a situation where the server knows that the content of entries
> visible to the client has not changed, and a large number of entries has
> potentially left the result set (without any equally large number of
> entries entering the result set), that sending "entry present" results will
> take less bandwidth than resending entire entries.  It is far from clear to
> me how likely such scenarios are.  One case where the "entry present"
> mechanism does work well is when a client synchronizes infrequently with
> resect to the deleted/moved entry history maintained by the server.
>
> I suspect either method will be sensitive to certain classes of changes
> (most likely access control or group membership) that will result in some
> clients being reloaded even though the net content change on the client is
> small -- just too hard to figure out the delta.

Fair comments. It is certainly possible we are overthinking things in 
this entire area. Implementation experience has already been fed back 
into LCUP, and more of that will happen in the future.

--
Mark Smith
Netscape Directory Product Development
My words are my own, not my employer's.