[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
1. "Atomicity and related concepts" and 2. "ModifiersName and Other Operational Attributes"
Steve,
I believe that I have covered item 2, which I summarized as:
2. "ModifiersName and other Operational Attributes". Disagreement about
definition, semantics, and implications of URP and MDCR proposals for
handling them. Definition and semantics should be clarified in requirements
document before final call. URP and MDCR handling should be clarified in
further discussion of URP and MDCR. Unclear whether there is a disagreement
about a requirement to handle ModifiersName "correctly", because of lack of
agreed definition of what would be "correct".
This is covered both below in item 1, and in issue E of:
http://www.imc.org/ietf-ldup/mail-archive/msg00641.html
So this message completes the series of 9 items, except for responding to
your further comments on items 3 and 5, and any further comments you may
make on other items. Please let me know if I have failed to respond to
anything.
Here's the response to item 1, which I summarized as:
***
1. "Atomicity and related concepts". Disagreement about definition,
importance and relation to other concepts and requirements. Should be
clarified and resolved by definitions, explanations and requirements in
requirements document before final call.
http://www.imc.org/ietf-ldup/mail-archive/msg00614.html
***
Comments interspersed with original below.
***
> [RM]
> On the subject of the requirements / URP draft:
>
> 1. I still believe the requirements draft should be more
> explicit about atomicity of operations being maintained
> across replication. In my various re-readings of this
> draft, I have at times found justification for both
> sides (maintain and do not maintain) and I still think
> that maintaining atomicity is necessary.
>
> 2. The URP draft should be explicit in how it maintains
> atomicity of operations across replication. I'm pretty
> sure from my last perusal it doesn't now.
>
> [AL]
> Understood.
> URP certainly does not maintain atomicity and is explicit
> about that. This
> problem is not in the URP draft, but in the requirements
> draft. If there was
> a requirement to maintain atomicity there would be a
> completely different
> URP (not necessarily MDCR).
[Steven]
Atomicity is not the right concept to be arguing about in a multimaster
replication environment. Consider this example.
At server S1 at time t1 a user modify request on an entry adds attribute
A1 and replaces the existing values of attribute A2. At server S2 at
time t2 (t2 > t1) a user modify request replaces attribute A2. Some time
later the modify from server S1 arrives at S2. If S2 performs this
operation atomically it will replace the newer (time t2) value(s) of
attribute A2 with the older values (time t1). This is clearly the wrong
thing to do. If S2 adds the new attribute A1 but ignores the replacement
of A2 (as URP would do) then it produces the same outcome as the
serial execution of the two modify requests, though the action doesn't
fit the usual definition of "atomic".
[Albert]
The user U1 at S1 wanted to modify the entry (A1,A2) from (-,x) to (y,z).
The user U2 at S2 wanted to modify it from (-,x) to (-,w).
The attributes A1 and A2 are presumably related, since U1 updated them
both at once.
They cannot both do what they want, so URP does something that
neither of them wanted to do. It modifies (-,x) to (y,w).
If U2 had known that the entry was (y,z) instead of just having read that it
was (-,x) he or she might have decided to check with U1 before changing it,
since U1 would be listed as the modifiersName and has recently changed it.
So U1 sees U2 listed as modifiersName for (y,w) and asks U2 "why did you
stuff up my change to (y,z) by changing it to (y,w)?".
U2 replies "I didn't, the entry was (-,x) and I changed THAT to (y,w).
After a few such incidents, one or other of U1 and U2 realize that it is
unlikely that the other is making the same stupid mistake over and
over again and refusing to admit it. So they report the mysterious behaviour
of the application to the Help Desk.
After a few attempts to convince them that they must be hallucinating and/or
pacify them with muzak, the Help Desk accumulates a sufficient range of
credible
reports, or a single report from a more senior user who can't be ignored,
and
passes it on to a sysadmin.
The sysadmin runs a few tests. Every time the sysadmin logs in as two
different
users and makes similar changes they work fine (because they are happening
on
the same replica). So the sysadmin reports "no fault found".
After a few more such incidents the sysadmins that have been called in
notice
that the users have become more hallucinatory than usual and are becoming
"restless".
Eventually the problem escalates to the point where a more senior and
experienced sysadmin is told "FIND it and FIX it NOW".
The experienced sysadmin checks what changed around the time the users
starting to
hallucinate. After investigating the application, the directory software,
the
air-conditioning, the water supply, the operating system and so forth, none
of which have changed, the experienced sysadmin eventually notices that a
very minor configuration change occurred, with two way replication instead
of one way replication between directory servers.
The wise sysadmin has never heard of atomicity but knows exactly what to do.
He or she undoes that minor change, just because it correlates with the
hallucinations.
Everyone lives happily ever after, except perhaps the directory vendor, who
might get sued.
Of course with any luck, one of A1 or A2 will be ldapACI, so a CERT advisory
warning people not to turn on two way replication will go out relatively
quickly,
lower level sysadmins will be more likely to see that, and a great deal of
wasted
time will have been avoided - except of course for the LDUP WG developing
standards
and the vendors silly enough to implement them.
With MDCR the situation is also unsatisfactory, though not as bad. Some time
after
having made the change from (-, x) to (y, z), U1 will get an email
notification
explaining that this change was automatically revoked because another user
changed
the entry at around the same time on a different server.
If that happens too often, somebody will either have to reconfigure the
replication
scheduling or alter the application to separate A1 and A2 into different
child entries.
But the problem will be a clear and easily understood result of using
multi-master
in a situation that is unsuitable for it. Not just a mysterious "something's
broken".
[Steve]
If we are going to discuss atomicity in replication then we need a more
meaningful definition. URP makes all the replicas produce an outcome
that is equivalent to the serial atomic execution of all the updates.
That is as atomic as it needs to be.
[Albert]
The current requirements draft has a completely meaningless definition:
Atomic operation - The ability to treat and contain several updates
or attribute changes as a single operation for replication purposes
to guarantee that the several updates or attribute changes are
propagated to a replica as a single unit.
In order to discuss and even think about anything at all, we have to use
the "usual" definitions. The purpose of Orwellian doublespeak is precisely
to prevent discussion and thought.
If URP just had to meet that Orwellian definition, it would pass with flying
colors - every modification in a single operation is elaborately provided
with a sequential modification number, wrapped in a framed operation and
propagated to a replica as a single unit. The only point of doing so seems
to
be to distract attention from the fact that on arrival it is promptly
merged,
with concurrent changes at that replica and those that have arrived from
other replicas, and the merged changes, which are not the result of ANY
LDAP operation, are then elaborately sequenced and packaged to repeat the
charade elsewhere.
This produces a combinatorial explosion of intermediate visible states which
can be read by DUAs and be the basis for other changes. When there are no
such further changes the result eventually converges (leaving aside problems
with timestamps etc) to a MIXTURE of the various concurrent operations.
THIS is "clearly the wrong thing to do". It is less atomic than it needs to
be because it is the precise opposite of atomic. What is needed is for
applications to know that either the whole operation succeeded or the
whole operation failed.
Since they are built on that assumption they will break when it isn't true.
As well as making intermediate partial operations visible to DUAs, URP does
NOT converge to "an outcome that is equivalent to the serial atomic
execution
of all the updates". That can only be done by suppressing changes that are
not
part of a sequence in which each successor could have read the result of its
predecessor - which is the whole point of the tree maintained by MDCR.
For example if two users replace the same attribute value with a different
new
value serially, by deleting the old value and adding the new value, one of
the
atomic operations MUST fail, since it has attempted to delete a value that
isn't there. With URP, both succeed. In fact URP *always* succeeds, whereas
atomicity is all about sometimes failing.
URP will even "succeed" if two users concurrently delete two different
values
from a mandatory attribute that originally held two values. It will
"succeed"
by deleting both and marking it as a schema violation for the sysadmin to
clean
up. What then was the point of the schema?
For LDUP to be "as atomic as it needs to be" it needs to be as atomic as
expected by the existing LDAP applications.
The "usual" definition isn't just some convention, it is a mandatory
standard
for LDAP which MUST be complied with by LDUP standards.
[Steve]
The real question revolves around the preconditions of a user update
request. If we look at the equivalent serial processing of a collection
of updates performed at two or more multimaster replicas then some of those
updates will be "executed" against a different database state than actually
existed at the time they were really performed by one of the replicas.
[Albert]
Stop fantasizing about revolving around aspects of reality that are
inconvenient
for URP.
Here's the REAL requirement that LDUP MUST meet. It isn't a question, it is
a command:
- modification: A list of modifications to be performed on the entry.
The entire list of entry modifications MUST be performed
in the order they are listed, as a single atomic operation. While
individual modifications may violate the directory schema, the
resulting entry after the entire list of modifications is performed
MUST conform to the requirements of the directory schema. The
values that may be taken on by the 'operation' field in each
modification construct have the following semantics respectively:
add: add values listed to the given attribute, creating
the attribute if necessary;
delete: delete values listed from the given attribute,
removing the entire attribute if no values are listed, or
if all current values of the attribute are listed for
deletion;
replace: replace all existing values of the given attribute
with the new values listed, creating the attribute if it
did not already exist. A replace with no value will delete
the entire attribute if it exists, and is ignored if the
attribute does not exist.
(RFC 2251 4.6 Modify Operation, p33)
[Steve]
The URP philosophy is that most of the time for most applications it doesn't
really matter. The MDCR philosophy is that it is better to completely
disregard the user's update, after the fact, just in case the state of the
entry did matter.
[Albert]
The URP "philosophy" is that existing standards don't matter and that manual
sysadmin fixes for the results of breaking them are acceptable.
You *seem* to be also saying that the state of an entry before it is written
doesn't really matter to the user changing it.
If that was really true ANY of the time, for ANY applications, there
wouldn't
be much point changing it, would there?
MDCR certainly does suppress concurrent changes, "just in case" the state of
the entry did matter. Its "philosophy" is that the state matters or they
wouldn't be changing it.
It does so "after the fact", which WILL cause problems, because there is no
other possibility in a multi-master environment.
But it doesn't "completely disregard" the suppressed update. See item 7 on
Notifications.
Faced with the unpleasant consequences of multi-master, the vital thing is
to get the widest
possible input from people likely to be affected by them, as to what trade
offs should be made.
If MDCR doesn't pass that scrutiny, something better will be worked out.
By failing to present a requirements document that clearly and frankly
explains the issues
and solicits input, the LDUP WG can guarantee that whatever it decides on
will meet vendor
requirements and not have to worry about user requirements.
That is clearly the wrong thing to do.