RE: 3. "Change Reports - ProtocolOps or Primitives"

["Steven Legg" <steven.legg@xxxxxxxxxxxxx>]

Albert,

> -----Original Message-----
> From: owner-ietf-ldup@xxxxxxxxxxxx
> [mailto:owner-ietf-ldup@xxxxxxxxxxxx]On Behalf Of Albert Langer
> Sent: Tuesday, 18 July 2000 21:18
> To: steven.legg@xxxxxxxxxxxxx
> Cc: ietf-ldup@xxxxxxx
> Subject: 3. "Change Reports - ProtocolOps or Primitives"
> 
> 
> Steve,
> 
> I didn't really understand what you are getting at on this, 
> so it might be
> best to restate what point you were actually making afresh.
> 
> Nevertheless, here's my response to item 4, which I summarized as:
> 
> 3. "Change Reports - ProtocolOps or Primitives". Reference by Steve to
> non-LDAP and non-X500 replicating servers may involve clarification of
> requirements. Other issues can be clarified in discussion of 
> URP and MDCR
> approaches.
> 
> 
> http://www.imc.org/ietf-ldup/mail-archive/msg00614.html
> 
> ***
> Comments follow the original below.
> ***
> 
> [AL responding to Ryan]
> > As regards efficient propagation of the tree via the report
> > propagation
> > protocol I believe the encoding on the wire is as efficient
> > and as simple as
> > possible by just conveying the actual LDAP protocolOps for
> > the changes. The
> > only redundancy is for the relatively small proportion of 
> changes that
> > affect the Directory Information Tree (DIT), ie add, del 
> and modifyDN,
> > rather than just the Directory Information Base (DIT), ie 
> modify (for
> > changes to non-distinguished attributes and values).
> 
> [Steven]
> The replicating servers aren't necessarily LDAP or X.500 
> DSAs, so the update
> protocol operations aren't necessarily just LDAP, DAP or DSP 
> operations.
> Also, these protocols are still subject to change so 
> additional operations
> may be defined in the future, or existing ones extended. LDAP also has
> a means for vendors to define proprietary operations. We 
> can't expect LDUP
> implementors to cover all the possibilities.
> 
> The original request also doesn't carry the changes to operational
> attributes or other vendor specific DSA invoked changes such as might
> occur to maintain referential integrity of DNs.
> 
> Breaking all update requests into replication primitives gets 
> around these
> problems.

Perhaps I should have said: Breaking all update requests into replication
primitives for conveying changes in the LDUP protocol (instead of
using the original protocolOp) gets around these problems.

> [snip]
> 
> [Albert]
> I do not understand what problems you are referring to or how 
> breaking all
> update requests into primitives gets around them.

The problem is that a list of all the standard LDAP or X.500 update
operations does not describe all the possible update behaviours
exhibited by directory servers.
 
> My understanding was and is that URP breaking all update requests into
> primitives is intended to get around the problem that modify 
> operations to
> the same entry made concurrently at different replicas cannot 
> all succeed as
> a whole (ie "atomically"). URP breaks up the whole operation into its
> constituent primitive elements so that they can be merged or 
> "reconciled"
> independently regardless of any conflict between them. When 
> more than one
> operation includes a primitive element that changes the same 
> single valued
> attribute, only the primitive element that happens to have the highest
> timestamp will succeed. But if they change different 
> attributes, or add or
> delete different values to the same attribute, they can all succeed.
> 
> Please confirm whether that understanding is correct, or 
> explain why it is
> not.

For the protocol design there was a discussion on whether the user updates
would be conveyed in the replication PDUs in their original form, which
was then broken down in to primitives by each replica to be processed, or
instead as the broken down set of primitives determined by the originating
server. It was a given at that time that the updates would be broken down
into primitives at some point, so it naturally came down to a choice
between LDAP update request or replication primitives in the protocol. 

The replication primitives can describe exactly how a directory server
executed a user update request without regard for the access protocol,
protocol extensions, conformance level, support for operational attributes,
etc. The small set of replication primitives is sufficient to describe a
much larger set of current and future directory server behaviours, as
well as being the basis for reconciling updates.

> 
> Conversely, MDCR aims to ensure that each operation can only 
> succeed or fail
> as a whole, so it has no need to break it up into "primitives".
> 
> I was replying to Ryan raising an issue which I took to be about the
> efficiency of protocol encoding on the wire, but may not have 
> been. My point
> was that both MDCR and URP carry essentially the same 
> information, although
> they treat that information differently ("atomically" vs splitting the
> atom).
> 
> Now you *seem* to be saying that the use of primitives 
> enables URP to convey
> additional protocol information for proprietary non-LDAP and non-X500
> directories, which could not be conveyed by MDCR.

The use of primitives allows the *LDUP protocol* to convey the additional
information.

> I do not 
> see any provision
> for this in URP and I do not think any such provision is 
> desirable.

The latest version of the URP document explicitly talks about what to
do with non-LDAP/X.500 update operations.

> It could
> easily be added to both if it was desirable.

The LDUP protocol and URP don't need to change to accommodate other
types of update operations.
 
> 
> If there is any thought of doing so, I think a clear 
> statement should be
> added to the requirements document explicitly ruling this out, as such
> additions would guarantee incompatability between different 
> implementations.

It's the reality now, there's no chance of it being ruled out, and it's
not a problem for the current LDUP design.

> 
> On the other hand you might more plausibly have meant to say that the
> original protocolOp as received by the originating DSA may 
> include such
> non-standardized information, or not be based on a DUA 
> protocolOp at all,
> and therefore should or could not just be copied verbatim by MDCR.

That's what I was saying.

> 
> If so, I agree. My intention in the MDCR draft was that the 
> encoding should
> correspond to a canonicalized "protocolOp" that could have 
> been received
> from an LDAP DUA as specified in the LDAP RFCs - explicitly 
> excluding any
> "controls". I did not mean to imply that it would always be a 
> verbatim copy
> of such a protocolOp (although many implementations will be 
> simplified by
> being able to do that). The "protocolOp" ASN specified in the 
> RFCs includes
> all the information required by URP except the entryID, which 
> MDCR also
> carries separately. Thus MDCR does not omit any information 
> used by URP.

I think you might be underestimating how difficult it will be to translate
arbitrary extensions, controls and alternative access protocols into
canonical protocolOps. For one thing you will have to be prepared for
a single user update to map into multiple protocolOps affecting multiple
entries. I can think of a number of features in our directory and other
vendor's directories that would require this.

> 
> The (marginally) greater efficiency of MDCR "on the wire" 
> results simply
> from not repeating the entryID etc for every attribute value 
> that is changed
> by a single modify operation. MDCR just transmits this once 
> for each modify
> operation and also saves other overhead from multiple 
> protocol elements for
> a single operation (both "on the wire" and when processing 
> the protocol).

I regard efficiency on the wire as largely a non-issue for replication.
My experience with X.525/DISP is that supplier DSAs can pump
changes through to consumers at least an order of magnitude faster
than the consumers can process them. A super efficient LDUP protocol
won't make much overall difference.

> 
> I also believe it is important not to impede future extensions to the
> replication protocol to cover future standardization of 
> additional features
> such as "signed operations". URP impedes this by nuking 
> operations so that
> the very concept of an operation has been lost once it has 
> been subjected to
> atomic fission and its constituent elements sent wandering through the
> replication network. MDCR preserves the concept of a "whole" 
> operation (ie
> an "atomic" operation), for other reasons, but has the 
> additional benefit of
> not impeding future developments such as signed operations.o

Signed user operations are no more an issue for LDUP than they are
for DISP (i.e. irrelevant). X.500 has signed operations and doesn't
use the original DAP operations in its replication protocol and I've
never found a problem with that. In any case, a canonical MDCR change
report would invalidate the operation signature as surely as breaking
the operation down into primitives. 

> 
> Finally, in relation to the original point re efficiency on 
> the wire, I
> should mention that I said MDCR is only "marginally" more 
> efficient than URP
> because the really significant overhead is per replication 
> session, (for
> authentication, synchronization etc) not per item within a replication
> session. MDCR also carries the full DN of each modify 
> operation within the
> protocolOp, which is not strictly necessary for the current 
> draft (00), and
> could be omitted as URP does, if the number of bytes was a 
> dominant concern.
> However I believe it will be necessary for dealing with 
> potential conflicts
> between replication of prescriptive ACI information applying 
> to an entry and
> changes to the entry. I do not believe URP is capable of 
> addressing that at
> all.

It's not obvious to me what sort of conflicts you are talking about.
Changes to prescriptive ACIs are much rarer than changes to entries
and unlikely to be directly linked. If there is a concern then wouldn't
MDCR be in bigger trouble because it is liable to completely discard
one of the linked changes ? Merging has fewer failure modes.

Regards,
Steven

> 
> Also, MDCR always propagates every change report to every 
> replica, whether
> or not it results in a visible change, whereas URP does not 
> propagate a
> primitive further, once it has encountered another for the 
> same attribute
> value with a higher CSN. This is important for explaining the 
> difference in
> predicability etc of the two alternatives, but not a 
> significant difference
> in efficiency "on the wire". In practice most changes are not made
> concurrently to the same entry, and for the small proportion 
> that are, those
> with earlier CSNs will usually have propagated further than 
> those with later
> CSNs, which will therefore be catching up with them rather 
> than preventing
> their further propagation.
> 
>