[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: draft-weiser-replica-req-01.txt
Here's my detailed comments on draft-weiser-replica-req-01.txt. Thanks to
Russel & Ellen for all their work on it.
Jeff
some of my mark-up terminology...
req = requirement
"s/b" = "should be"
-----------------------------------------------------------------------------
Overall comments:
1. John Merrells' restating of the reqs in his msg entitled "Requirements
Categorisation" of Wed, 15 Apr 1998 17:30:01 -0700 has nice, concise reqs
statements. I suggest that the doc utilize those, or statements like them, for
titles for the reqs, and then provide additional text where necessary for
explanation.
2. High level, overall system reqs are assumed or implied rather than
stated. These are..
a. level of synchronicity of anticipated client apps?
b. required level of data consistency between replicas?
c. operational semantic requirements
The first two perhaps should be addressed in a separate section before the
"replication model" one, and the third should be addressed in the replication
model section.
3. We should also state and assign requirement levels to the overall
meta-requirements of..
scalability
availability
extensibility
adaptability
flexibility
security
replication transport protocols
This should be in a very early section of the doc, perhaps in the same one or
prior to the one with items from #2 above.
-----------------------------------------------------------------------------
INTERNET-DRAFT Russel Weiser
Informational Draft Novell, Inc.
Expires 13 October 1998 Ellen Stokes
IBM
13 April 1998
LDAP Replication Requirements
<draft-weiser-replica-req-01.txt>
Status of this Memo
This document is an Internet-Draft. Internet-Drafts are working
documents of the Internet Engineering Task Force (IETF), its areas,
and its working groups. Note that other groups may also distribute
working documents as Internet-Drafts.
Internet-Drafts are draft documents valid for a maximum of six months
and may be updated, replaced, or obsoleted by other documents at any
time. It is inappropriate to use Internet-Drafts as reference
material or to cite them other than as ``work in progress.''
To view the entire list of current Internet-Drafts, please check
the "1id-abstracts.txt" listing contained in the Internet-Drafts
Shadow Directories on ftp.is.co.za (Africa), ftp.nordu.net
(Northern Europe), ftp.nis.garr.it (Southern Europe), munnari.oz.au
(Pacific Rim), ftp.ietf.org (US East Coast), or ftp.isi.edu
(US West Coast).
Abstract
This document discusses the fundamental requirements for replication
of data accessible via the LDAPv3 [RFC2251] protocol. It is intended
to be a gathering place for general replication requirements needed to
provide interoperability between informational directories.
The key words MUST MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
"SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
document are to be interpreted as described in [RFC2119].
It's a nit, but I don't think this rfc2119 stuff belongs in the abstract --
the abstract gets copied out to different places and such, and such document
conventions stuff isn't meaningful unless you're reading the doc as a whole.
Weiser & Stokes 13 October 1998 [Page 1]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
Table of Contents
1. Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
2. Terminology . . . . . . . . . . . . . . . . . . . . . . . . . . . 4
3. Objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5
4. Applicability Statement . . . . . . . . . . . . . . . . . . . . . 5
5. Replication Model . . . . . . . . . . . . . . . . . . . . . . . . 5
6. Replication Protocol . . . . . . . . . . . . . . . . . . . . . . . 7
7. Replication Predetermined Agreements . . . . . . . . . . . . . . . 8
8. Administration and Management Considerations . . . . . . . . . . . 8
9. Other for furhter discussion or the garbage heap . . . . . . . . . 9
10. Acknowledgement . . . . . . . . . . . . . . . . . . . . . . . . . 9
11. References . . . . . . . . . . . . . . . . . . . . . . . . . . . 9
12. Author's Address . . . . . . . . . . . . . . . . . . . . . . . . 10
Weiser & Stokes 13 October 1998 [Page 2]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
1. Introduction
The ability to distribute directory information throughout the network
provides a two fold benefit to the network: (1) increasing the
reliabililty of the directory through fault tolerance, and (2) brings
the directory content closer to the clients using the data. LDAPs
acceptance as a access protocol for directory information is driving
the need to distribute LDAP directory content among servers within
enterprise and Internet. Currently LDAP does not define a replication
mechanism and only generally mentions LDAP shadow servers (see
[RFC2251] and [Changelog]) in passing. The requirements for
replication are critical to the successful deployment and acceptance
of LDAP in the market place.
2. Terminology
For the purposes of this document, the following definitions are used:
Replication - The process of copying portions of naming context
^^^^^^^^^^^^^^
What's a "naming context" (rhetorical question)? We should either define it in
this doc or provide a ref to where it is defined. Perhaps the former if we're
trying to minimize references to X.500 docs.
information and content, between multiple LDAP servers, such that
certain, predefined portions of the information are available from
many geographic locations. that which is done between either
^^^^^^^^^^^^^^^^^^^^^^^^^
Nit: perhaps should be "different servers", or "topologically different
locations".
homogeneous implementations across heterogeneous platforms
(operating systems) or heterogeneous implementations supporting
identical replication across heterogeneous platforms (operating
systems). No mapping mechanisms are required here.
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
The key seems to be the latter stmt.
Synchronization - The process whereby LDAP servers participating in
the replication, are brought into a state where copies of the
information they make accessible are consistent with the other.
that which happens between heterogeneous directory servers
onheterogeneous platforms (operating systems). The different
directory server implementations do not have to provide an LDAP
interface. One could certainly define a wire protocol. The
mapping of data means mapping of attributes. For example, in one
system, the attribute is full name, in the other it is common name
We agreed during the conf call to not utilize or define the term
"synchronization". (conf call: there was a conference call on Thur 16-Apr
among those proposing specific replication approaches, and the reqs doc was
discussed in fair detail).
Incremental Update - The process of updating a replica, or copy, of
a naming context, by updating only those fields or objects which
have changed.
Master Replica - A writeable copy of the replicated information.
Slave Replica - A read-only copy of the replicated information.
We agreed during the conf call to come up with another set of names for
various replica flavors.
Multi-Master Replication - A replication model where entries can
be written and updated on any of several replica copies, without
requiring communication with other masters before the write or
update is performed.
Another term for this is "read-anywhere/write-anywhere".
Weiser & Stokes 13 October 1998 [Page 3]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
Master Slave, or Single Master Replication - Replication model that
assumes only one server, the master, allows write access to the
replicated data. Note that Master-Slave replication can be
considered a proper subset of multi-master replication.
One-way Directory Synchronization - The process of synchronization
^^^^^^^^^^^^^^^
s/b replication (change throughout doc)
in a single direction where the authoritative source information is
provided to a replica.
Two-way Directory Synchronization - The process of synchronization
where change information may flow bidirectionally between two
replicas.
Need to add definitions for (and/or agree to equivalent terms for)..
anti-entropy, update propagation, etc. -- are names and/or descriptions for
the algorithmic, protocol-based process by which directory replicas are
reconciled.
Authoritative replica -- a replica containing complete replicated areas
which contain complete entries.
fractional replication -- The capability to replicate a subset of attributes
of any given entry.
partial (sparse?) replication -- The capability to replicate one or more
subsets of a given DIT. These subsets are synonymous with "replicated areas"
in X.525.
replica -- an instance of a replicated directory.
partial | sparse | fractional replica -- a replica that receives fractional
and/or partial(sparse?) data.
replicated directory -- a directory whose DIT is, or portions thereof are,
replicated across more than one server. Sometimes referred to as simply
"directory" in this doc.
topology - as in "the replicated directory's topology". Refers to the shape
of the directed graph describing the relationships between replicas.
Note: section 2.1 of draft-ietf-asid-ldap-mult-mast-rep-02.txt should be
merged into this section.
3. Objective
The major objective is to provide an interoperable LDAP V3
directory synchronization protocol which is simple, highly
efficient and flexible enough to support both Multi-master and
Master-slave replication operations to meet the needs of both the
Internet and enterprise environments.
Are you referring to the overall goal of the ldap replication design &
specification effort, or of this doc? The statement is ok if you mean the
former, but not really if you mean the latter. It seems to me that the goal
statement for this doc is something like..
"The goal of this document is to enumerate and organize the overall set of
requirements for LDAP replication."
4. Applicability Statement
TBD
5. Replication Model
5.1 LDAP Replication MUST be allowed to span different vendors
directory services in order to provide interoperability.
This is "by definition" of an IETF protocol spec. Doesn't seem to be a req to
me.
5.2 All replicas MUST eventually be updated with the changed
information, specified by the replication policy.
This seems to be in direct conflict with 5.7. I'd say that this req is the
correct one -- but I'm going to raise the topic of what level of consistency
we're designing for in a separate msg.
5.3 Replication schedules MUST be dynamic to allow for periodic
replication, with the replication period determined by
administrator of the replicated system.
perhaps "MUST facilitate administrator-specified replication schedules"?
5.4 Replication Model SHALL enable replication cycle to initiated
based in the number of pending changes.
perhaps "count of pending changes" should be added to a required list of
various metrics for replication scheduling policy.
5.5 The replication model SHOULD allow for initiation of
replication cycle for any replica that may have just come back
online or was unavailable previously.
perhaps "MUST support intermittent replica connection", or "MUST support
replica disconnection and subsequent reconnection". Note that
disconnection might be voluntary, as in the case of a replica hosted on a
mobile system (laptop, palmtop, wristband (did you see the Seiko
wristwatch-computer announcement?)), or it might be involuntary as in the case
of any networked system.
5.6 The replication model MUST support the both master-slave and
multi-master replication topologies.
^^^^^^^^^^
I don't think these are properly topologies -- rather they connote the
relationship between a pair of replicas.
Weiser & Stokes 13 October 1998 [Page 4]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
5.7 All replicated information between the master database and its
replica databases SHALL be identical including all no user
modify operational attributes such as time stamps. Note this
does not imply that the entire database is identical from
replica to replica, but that the subset of data, chosen to
replicate is identical from replica to replica.
This is in direct conflict with 5.2. It seems to be a requirement for strong
consistency within relatively short time frames.
5.8 Distributed multi-vendor environment, LDAP replication SHALL
NOT ensure all copies of the replicated information be
^^^^^^
require
complete copies of the replicated object.
Perhaps "Fractional replication MUST be supported."
5.9 Replication MUST allow Definition content of replicated
information.
I'm not sure what this means or is for.
5.10 LDAP replication SHALL encompass common schema objects and
attributes, access control and name spaces.
Perhaps "All non-directory-system-specific data contained in a directory MUST
be a candidate for replication". We probably need to define terms that'll
allow us to succinctly describe "non-directory-system-specific data"
(everything other than the root DSE?) and "directory-system-meta-data" (this
intersects with the root DSE (correct?), but is it necessarily congruent?)
5.11 Sub tree Replication SHALL be defined to allow for greater
flexibility replication topologies of the DIT as discussed in
X.525 section 7.2 [X.525].
Axe as discussed in conf call.
5.12 A propagation schedule SHALL be defined and SHOULD be tunable
within Predetermined replication agreement.
This should be combined with 5.3.
5.13 Replication of critical values SHALL be synchronized with
priority over non critical values, an example of a critical
value might be a password and or certificate value which
maybe considered critical.
What defines "critcal values"? The ldap protocol presently doesn't support
such a concept, so it needs to defined in this doc. Perhaps the desired req is
"replication algorithm MUST assign a higher replication priority to critical
data" and then we'd need a definition of critical data and a req for being
able to denote data as critical (I note that X.525(97) sec 9.2.4.1 makes a
similar claim in regards to ACI info, is that essentially where this req came
from?)
I'm not convinced that this req is a MUST or is needed in terms of having such
a distinction among data ~in general~. I have a hunch that
the needs this would be satisfying would probably be better satisfied by being
able to provide relatively strong consistency among replicas to applications
desiring it. However, I can see where it ~might~ work out to be desirable for
system meta-data as the X.525 ref above states. Something to think carefully
about.
5.14 Currently X.525 DISP [X.525] discusses this as a shadowing
agreement including such information as unit of replication,
update mode, and access point defining many of the policies
between the master and a replica. The use of predetermined
replication agreements between the directory replicas MUST be
addressed to provide proper knowledge of access requirements
and credentials between the synchronizing directories.
Not clear what this is addressing. Please restate more clearly. Is it here
because something akin is in X.525?
5.15 Due to the acceptance and usage of the Internet, and the
requirement that LDAP replication be available across
disparate vendors directory services, LDAP replication MUST
provide scalability for both enterprise and internet
environments.
This is a meta-requirement of scalability. should be addressed early in the
doc with other meta-reqs.
5.16 The replication model MUST define deterministic policy such
that replication cycle startup time conflicts between two or
more competing master replicas may be resolved
programmatically. An Example might be automatic submission and
rescheduling by one of the masters. In such a case, these
replication "conflicts" SHALL be resolved by the replication
policy.
Not clear to me what this is requiring. Perhaps "there MUST be the capability
to specify conflict resolution policies for replication agreements
themselves."?
Weiser & Stokes 13 October 1998 [Page 5]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
5.17 Replication SHALL be allowed to any LDAP server available on
the network; provided appropriate security authorization is
granted.
Lots of complexities to this, and am not sure what this is necessarily
implying. Is it something akin to "any two servers (impl'd to satisfy these
reqs), which are able to communicate via LDAP across a network of
indeterminate scale, SHALL be capable of carrying out an anti-entropy session,
subject to administrative controls in place at either or both server."?
6. Replication Protocol
6.1 The act of replication MUST have minimal impact on both the
system and network performance.
This is a meta-requirement or overall design goal.
6.2 The replica synchronization MUST be handled in such a manner
as to not saturate network with repetitive entry replication
from multiple synchronization providers points.
This is a meta-requirement or overall design goal.
6.3 Replication SHALL only be allowed after the authentication and
verification of authorization of both the replica and the
source directory.
Perhaps "Establishment of replication (anti-entropy) sessions is subject to
auth and authz requirements specified in subsection ?.?"
We need to specify that subsection and those reqs.
6.4 The transport for LDAP synchronization MUST allow for the
integrity and confidentiality of each replicated server.
This is one.
6.5 Replicated data MUST be transferred in a secure manner.
This is another.
6.6 Replication protocol MUST provide for recovery and
rescheduling of a replication cycle due to update conflict and
or loss of conncetion
These are two separate reqs as update conflict and connection loss are
orthogonal. Loss of connection is already covered by 5.5. Thus perhaps "the
repl protocol MUST provide for update conflict resolution".
I'd lobby to enhance it to: "the repl protocol MUST provide for
application-specificed conflict resolution procedures."
6.7 LDAP replication MUST allow for full update to facilitate
replica initialization and reset loading utilizing a
standardized LDIF [LDIF] format.
This is two separate reqs. One is "the means of conveying writes among
replicas is LDIF-based (and standardized)", the other is "one must be able to
create new replicas".
6.8 The normal means of synchronizing replicas SHALL be performed
through incremental synchronization and in accordance with the
scheduling policies.
Is this implying that there are abnormal means? Should talk about them if so.
Otherwise this item is stating the obvious. Unless we are going to
define/require different classes of replication, e.g. "incremental", we
shouldn't use such terms.
6.9 The replication Standard SHOULD NOT limit the size of a
replica.
Ok.
The unit of replication is defined to be the naming
context.
-> terminology as discussed in conf call.
Incremental replication SHOULD be allowed to attempt
to keep the amount of data replicated to a minimum.
What is incremental repl? Is the concern bandwidth requirements? We should
figure out what this is or it should probably go away.
6.10 Must allow updates to multiple replicas
Does this mean that a given replica MAY be paired with more than one other
replicas?
6.11 The replication protocol MUST allow either a master or replica
to initiate the replication process.
6.12 Additionally the initiator MUST be allowed to determine
whether it will become a consumer or supplier during the
synchronization startup process. This would allow a replica to
be periodically connected and synchronized from remote sites
at the local administrator's discretion.
If there are no practical reasons to differentiate between consumers and
suppliers in the replication process then this req goes away. It may well turn
out that having an identifiable "supplier" and a "consumer" in a given
pair-wise replication process is a special case of a more general one (which
is a mutual, pair-wise update between peers).
Weiser & Stokes 13 October 1998 [Page 6]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
6.13 Multiple LDAP changes, to a single server, SHOULD be allowed
to be treated as single atomic unit of work propagated during
replication.
Is this really a potential implementation approach for satisfying the more
general requirement to "not waste bandwidth"? Perhaps should be restated if so.
6.14 An LDAP Replication Standard SHOULD NOT limit the transaction
rate of a replication session.
Of course. Perhaps this is a non-req we shouldn't mention.
6.15 Entry change information SHALL be purged upon completion of a
synchronization cycle where all replica members have been
synchronized with the master(s).
This depends on how the anti-entropy (nee replication) algorithm is developed.
It may be that replicas decide on their own when to trim their change log.
7. Replication Predetermined Agreements
Russel noted this is being renamed "..schema.."
7.1 The Replication Protocol documents MUST define a standard
schema for representing replication agreements, and MUST
define the semantics associated with modifying the attributes
of replication agreements. The documents MUST also define a
standard method for determining the location of these
agreements.
7.2 The Replication Protocol documents MUST define a standard
schema for publishing state information about a given replica,
and MUST define a standard method for determining the location
of this information. This state information MUST include the
ID of the last update propagated to the replica. The format of
this ID is to be determined.
7.3 Location independent management point SHALL be defined to
provide authorized administrators with well known access to
the replication policies, regardless of network location.
7.4 Policies for the LDAP replication shall be defined in such a
manner as to allow programmatic representation; these policies
SHALL be kept as replica attributes or as entries of the
predetermined agreement and propagated during replication.
7.5 Replication agreements SHALL be accessible, via LDAP, to all
servers containing replicated information.
The above stuff looks ok..have to think more about it.
8. Administration and Management Considerations
8.1 Replication policies SHALL allow replication of changed
information to be postponed to a more convenient period ,
8.2 Allowance for non scheduled replication of a replica SHALL be
provided upon request such that the replica server has been
down or unconnected for a period of time.
Perhaps "There needs to be some way to initiate updating a replica that has
been disconnected. There might be a range of ways: automatic, admin
initiation, etc."
8.3 Each copy of a replica MUST maintain audit history of who it
has replicated with and who has replicated with it.
Architecture/implementation feature. Perhaps should be a "SHOULD".
Weiser & Stokes 13 October 1998 [Page 7]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
8.4 A replica MAY store the conflicted versions of the replicated
object to allow optional human review and intervention.
Architecture/implementation feature.
8.5 Access to replication predetermined agreements, topologies,
and policies attributes MUST be provided through LDAP access.
8.6 The capability to check the differences between two replicas
be of the same information SHOULD be provided for.
Architecture/implementation feature.
8.7 The capability to view the current state and replication
history of each replicated copy in the replication topology,
from a single server in the topology.
An architecture/impl feature that may easily materialize depending on how the
overall replication algorithm/protocol works. Perhaps another way to state it:
"capability to assess the current state and replication history of each
directory replica from a (any?) other given replica in the replicated
directory.
8.8 The ability to view replication conflicts, and override the
resolution derived by the replication policy SHALL be
provided.
Architecture/implementation feature.
9. Other for furhter discussion or the garbage heap
9.1 It is not realistic to assume that all vendors have
cooperating schemas, but it is required that different vendors
schemas allow replication from diverse schemas. and MAY define
a model whereby divergent schemas can replicate non-common or
extended attributes for common LDAP objects.
Are we trying to encourage or discourage schema (di|con)vergence?
9.2 Propagation behavior defines the general behavior of the
actual synchronization process between a consumer and a
provider of replication information.
MOve to definitions.
9.3 Entry Identifiers SHALL be defined Ref Christopher Apple.
9.4 Replica convergence and resurections of attributes and
objects; since( tombestones, Obituaries, deathwarrants,
graveyards) are used I?ll add one more ZOMMBIES. RFW
9.3 & 9.4 discussed during conf call.
10. Acknowledgement
This document is based on input from IETF members interested in LDAP
replication
11. References
[RFC2251] M. Wahl, T. Howes, S. Kille "Lightweight Directory Access
Protocol (v3), Standards Track , December 1997 . Availiable as RFC2251
[RFC2119] S.Bradner, " Key words for use in RFCs to indicate
Requirement Levels", RFC 2119.
[LDIF] Gordon Good, "The LDAP Data Interchange Format (LDIF)", Internet
draft, draft-ietf-asid-ldif-00.txt, November 1996.
Weiser & Stokes 13 October 1998 [Page 8]
INTERNET-DRAFT LDAP V3 Replication Requirements 13 April 1998
[Changelog] Gordon Good, "Definitions of an Object Class to Hold LDAP
Change records", Internet Draft, draft-ietf-asid-changelog-00.txt,
November 1996.
[X.525] "Information Technology - Open Systems Interconnection- The
Directory: Replication", ITU-T Recommendation X.525 and ISO/IEC
International Standard 9594-9, November 1993.
12. Author's Address
Russel F. Weiser
Novell Inc.
122 East 1700 South
Provo, Utah 84606
USA
E-mail: Rweiser@xxxxxxxxxx
Telephone: +1-801-861-7808
Fax +1-801-861-2292
Ellen J. Stokes
IBM
11400 Burnet Rd.
Austin, Texas 78758
USA
E-mail: stokes@xxxxxxxxxxxxxx
Telephone: +1-512-838-3725
Fax: +1-512-838-0156
Weiser & Stokes 13 October 1998 [Page 9]