RE: Comments on ldup-replica-req-00.txt

[Alan Lloyd <Alan.Lloyd@xxxxxxxxxxxxxxxxxxxx>]

> -----Original Message-----
> From:	John Merrells [SMTP:merrells@xxxxxxxxxxxx]
> Sent:	Thursday, April 23, 1998 2:13 AM
> To:	Alan Lloyd
> Cc:	'Uppili Srinivasan'; owner-ietf-ldup@xxxxxxx; ietf-ldup@xxxxxxx
> Subject:	Re: Comments on ldup-replica-req-00.txt
> 
> Alan Lloyd wrote:
> 
> >         The mechanism of stating UIDs, DNs (where names are
> modified)
> > and such things are easy to say and write down in an RFC - now what.
> > operationally it will hit walls and system robustness will be
> demaned
> > from high end directory systems that run banks, telephone
> > infrastructures, CAs and online services and military applications.
> 
> How else do you replicate the change?
> 
	As said in a managed way - that does not impose scaling issues
or information management issues or clock synchronisation issues.

	Replication in directory product can be performed two ways - via
the database tools (for those who are database focussed) and via DISP
for those who (dont have RDBs) are X.500 focussed. Again the rationale
for replication needs to carefully assessed. eg the requirement to get
at the master entry - is that required, how many replicas are practical.
Is distribution AND replication a better system if searches are
generally in a local context and occasionally a wider scope is used.

	Every one working on directories seems to be focussed on
replication and I see the need for it - but there should be a balanced
view of distribution with replication - that way many of the replication
issues do go away.

	If in a two DSA system a replication between one and the other
is easy - when its between 30 DSa with 100k+  entries - its gets tough
and exponentially bad (150 replication agreements). So one should split
DIBs according to access/ownership "regions" and distribute these. And
by all means have a few back end replicated systems as online back ups
and as local access system. But never design a system as all replication
(in fact if it is - it will never connect to anyone elses system
efficiently.)

	Why do we want replication... a) Back up and redundancy b)
Geographic distribution where high access rates are applied.
	ie. Replication does not satisfy directory interconnection with
"other directories "owned and managed" by "other named" groups.
	ie. Replication mechanisms do not have to deal with the whole
world being replicated to every where else and inconsistencies should be
allowed. Because consistency in all cases in all places with earthly
technology can never be guaranteed.

> >         As said the image that a directory is a mail address book
> that
> > must be replicated is still out there - and some simple tagging
> system
> > may be all thats needed for these -
> 
> We may not use tagging.
> 
	We wont either :-) - great minds think alike eh :-)

> >         If directories are to be a disciplined, robust infrastucture
> we
> > really have to watch tagging and transaction mechanisms - If one
> cannot
> > scale it and inteconnect it reliably - dont bother.
> 
> Yes, our goal is interoperability and a requirement is scalability.
> 
	us too - great minds again :-)

> >         and what happens if a DSA has a clock set to tomorow and its
> > updated at the same time - differently -  in the actual time + - a
> > second,  as a  DSA with its clock set to today ?
> 
> You don't have to use time for conflict resolution.
> 
	I think you do here. What happens if one user on one master
changes a DN Bill to Fred and in the other master a user changes Bill to
John or deletes Bill.
	Is the (real) time of the transaction not critical?

> > [snip... icl]
> 
	Why do you do this - I had a great time with Icl :-)

> >         X.500 permits inconsistencies for this reason - ie clock
> > synchronisation over the planet is hard - therefore a requirement to
> > grab the "logical master entry" is provided in the protocol options.
> 
> You don't have to use time for conflict resolution.There could be
> multiple masters, how do you know which holds theone true
> value. By the clock timestamp in the transation record of the last
> update ........resolution = 2 microseconds and synchronised -  and a
> system clock can NEVER have the same time when read sequentially .
	 been there done that - ICL .... snip -

> >         So the replication process really does not want to impose
> > exponential loads on this infrastructure just trying to keep every
> thing
> > consistent.
> 
> Scalable architecture is a requirement. yup.
> 
> > > Lovely, but customers are asking for replicated directory
> services.
> > >
> >         yes we know,  but that IMHO is because the "directory"
> experts
> > have been  preaching that due to products distributed operations not
> working the best
> or at all (LDAP) and the LDAP theme of doing replication now.  But
> > then reality hits,  as it always does - scaleability and
> interopration
> > and information consistency are now comming to the fore.
> >
> >         Its going to be an interesting world when all the "replicate
> > hypers" suddenly realise that all they are doing is casting a system
> > which is unscaleable by nature and when they want to eventually
> > interconnect for distributed operations - what a mess they will be
> in.
> 
> Yes, interoperability is the goal, scalability a requirement.
> 
> >         Distribution gets the naming and the knowledge and domain
> based
> > access control policies in place (and the distributed performance
> issues
> > resolved) that enable directory system scaling.
> 
> We still need Replication and LDUP is the Replication working group.
	Yes - but if the requirement of asking for a "master" entry is
adopted - then this can only be accessed in LDAP by a referral and in
X.500 - seamlessly via DSP. ie X.500 access - distribution/chaining
mechanisms support the replicated information environment.

> > Replication just
> > provides one with a "copy" of the inteconnection and performance
> > problems as described that have not been fixed.
> 
> Customers require replicated directory services.
	yup - we do DISP and DSP

	Can we agree on something.
	One can only get true consistency on replicated data where
directory clock synchronisation is maintained to the resolution required
eg 2 milliseconds, etc.
	If true clock synchronisation is not maintained - and
updates/deletes not reconciled - then the system must accept and deal
with replica inconsistencies. And what follows from this is:  a) Can
users get the "master" entry even if they are connected primarily to the
replica system and b) what is required to reconcile DIB discrepancies.
	If the list wants to go down the "true replica path" - it will
be a long haul.

	In addition we need a model for the "replication agreement" as
the first step.  Can we use the X.500 one.

	Regards alan - OpenDirectory (and ex ICL :-) )

> John
> 
> --
> John Merrells
> Netscape Communications
> Directory Server
> Software Engineer
> 
> http://people.netscape.com/merrells
>