Re: Fwd: Re: Synchronization [Was: Re: CAP:LastCallByMarchIETF: Need Volunteers!]

[Mark Paterson <markp@xxxxxxxxxxx>]

At 11:28 AM 2/4/2002 -0700, you wrote:

Mark Paterson wrote:
>
> At 02:39 PM 2/1/2002 -0700, you wrote:
>
> > I think it can be done in one single VQUERY.
> >
> >         EXPAND:TRUE
> >         QUERY:SELECT * FROM VEVENT
> >          WHERE LAST-MODIFIED >  ...last-sync-time...
> >
> > OR if the CUA knows how to expand the objects - just VQUERY
> > with EXPAND:FALSE and get the fewer objects - and expand
> > the instances in the CUA code.
> >
>
> Yes this would do it but the difference between your query and mine is
> that mine had a time range. Your query will give back everything that
> has changed on the CS within the users agenda and not just those
> within a specific time range. The Sync CUA may not care about new
> meetings that have been booked 3 years out. The requirement for CAP
> reads as follows...
>
> "CAP MUST allow synchronization, meaning at a minimum that the CUA is
> able to find and retrieve new, modified or deleted entries for a given
> time period. The CUA MUST be able to find out which entries have been
> added, modified or deleted since it last synchronized, in order to
> operate in disconnected mode. This requirement may be satisfied using
> the query requirements already defined."

It was my belief that text meant that it needed to know about
changes since a time. What good is it to 1/2 sync? That is if
you only get 'changes' from yesterday and ignore changes
from two days ago? Here is why you don't want that:

        Jan. 1  you sync and everything is up to date.
        Jan. 10  your CS gets METHOD:CANCEL for objects in Feb.
        Feb. 1   you sync with a time range of Feb. 1- Feb. 28.

     You now missed the METHOD:CANCEL for the VEVENTS in Feb.
     and the results you get back are exactly what you asked for
     and bogus.

Why have I missed it? If I ask for everything that has changed since Jan
1st which falls between Feb 1 and Feb 28th the objects within February
that have been deleted will get returned because their LAST-MODIFIED
attribute with be Jan 10th (greater than Jan 1) and their DTSTARTs fall
within the range. Are the meetings in February part of a recurring entry
in your example? If so, then yes they'd be missed which has been my point
all along that a query against LAST-MODIFIED with a time range catches
everything but instances of recurring entries that no longer exist. This
why in my last response I compromised and only used a time range when
querying for non-recurring entries and did not when querying for
recurring entries since I'll always have the chance of missing these if I
do.

Please note that above I mean the English word "deleted" and
not the CAP command so please don't correct me by telling me that I mean
the modify command updating the METHOD to CANCEL)  Note that in a
previous response you claimed the following...

"Entire objects are marked for delete by using METHOD:MODIFY 
and changing the object in the store to METHOD:DELETE"

Here you say CANCEL. Which is it? Its the "delete" command
which zaps the whole object which we don't want CUAs to do and the
"modify" command with METHOD:DELETE (or CANCEL). 


> > The five words of importance
> "for a given time period".
> > 
> > How does a Sync CUA find out about deleted instances of a
> recurring
> > entry that used to fall within a given time period but which have
> been
> > deleted since the last time a sync was performed without getting
> ALL
> > the changes on the CS and then doing its own time range
> filtering?
>
> You don't mean 'deleted' you mean METHOD:CANCEL, and those
> would show up with the above query I provided.

Yes I do mean deleted. I mean the English word deleted as in the instance
is no more and not the CAP command AND yes it would show up in your query
along with ever other change within the user's calendar but we were
trying to find a way to just get those changes within a given time
period. In my compromise I used your approach (with no time range) only
for recurring entries to cut down on the amount of unnecessary changes
being reported to the Sync CUA.


> You can't 'delete' an instance, you
> can only METHOD:CANCEL
> an instance. You can only delete an object, not instances
> that only exist when the CS expands an object.

I know that !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! From the Sync CUAs
perspective their is an instance that it no more and that is all I am
saying. The CUA which did the deletion should do so as you describe, yes,
we agreed to this several replies back. 


> > Perhaps something like this
> (once again I am simplifying things by
> > just using DTSTART and just worrying about VEVENTS)...
> > 
> > QUERY:SELECT * FROM VEVENT
> > WHERE LAST-MODIFIED >  ...last-sync-time
> > AND DTSTART > bottom of time range
> > AND DTSTART < top of time range
> > AND RRULE="" (I'm trying to say not recurring. What's the
> proper way
> > of doing this is?)
>
> "AND RRULE IS NULL" I think is what you are saying. But I
> don't
> think that is what you want - see below.

What I am saying is "I'm trying to say not recurring. What's the
proper way
of doing this is?". Exactly what's written in the parentheses. I'm
trying to express a query that gets all changes within a given time
period for all non recurring entries but I wasn't sure how to do so, so I
asked for your help.


> > followed by...
> > 
> > EXPAND:TRUE (or FALSE if the Sync CUA wants. It's not really
> CAP's
> > concern)
> > QUERY:SELECT * FROM VEVENT
> > WHERE LAST-MODIFIED >  ...last-sync-time
> > AND RRULE=* (I'm trying to say recurring. What's the proper way
> of
> > doing this is?)
>
> "RRULE IS NOT NULL" - Not what you really want.
>
> Your above query would return instances of VEVENTs with an RRULE
> (but not a RDATE, EXDATE, or EXRULE), 

I know that. That's why in parentheses I state quite clearly "I'm
trying to say recurring. What's the proper way of doing this
is?"  In other words, Please help me with this query. 

> within the given time
> range

I have no time range in this query. It follows your approach of no time
range since there seems to be no way of ensuring that the Sync CUA
catches instances that have been deleted (the English word deleted :-))
if you do. The only way to do it with a time range is if you could query
on something like an EXDATE but we've both agreed that forcing a CUA to
use an EXDATE to get rid of an instance is simply too
restrictive.

> that have changed since
> last-sync-time. Only instances in that
> time range would be returned - and only if they changed after
> last-sync-time.

As stated above my 2nd query does not have a time range.


> This below would get all instances
> (with or without RRULE's) that
> have changed since your last sync within a time range:
>
>   BEGIN:VQUERY
>
>   EXPAND:TRUE
>
>   QUERY:SELECT * FROM VEVENT
>    WHERE LAST-MODIFIED > ..last-sync-time AND METHOD !=
> 'CREATE'
>
>   QUERY:SELECT * FROM VEVENT
>    WHERE DTSART > bottom of time range
>    AND DTSTART < top of time range
>    AND METHOD = 'CREATE'
>
>   END:VQUERY
>
> Above, the CS would EXPAND 'original' VEVENTs into instances,
> and only return the ones that fall within the provided time range.
> AND it would return any modified objects that might effect
> the results. So you would now see the METHOD:CANCEL done
> in January that effected the February objects.
>
> ONLY a CUA combines objects, the CS does not, so the CS could
> not 'know' that the January 10th object effected the February
> objects in order to return them in the query.

Here's where we disagree and I think why we seem to think differently and
have trouble understanding each other's proposal. I'm going to reply to
this separately next (this reply is already too long).


> A problem with your examples, is
> that they ignored RDATE,
> EXDATE, and EXRULE. The CUA would have then had to calculate
> those exceptions and additions from the result set.

They did not ignore them. I simply didn't know how to express a query
which only returned recurring entries and was hoping that you make a
suggestion as to how it could be done.


> > These two queries will give
> the Sync CUA all new, changed and deleted
> > non-recurring entries within a users calendar within a given
> time
> > period and will give the Sync CUA ALL new, changed, and 
> deleted
> > recurring entries within a users calendar. This means that the
> Sync
> > CUA may get back more recurring entries then it actually needs
> but
> > this would seem to be a reasonable compromise. Statistically
> users
> > would tend to have less recurring entries in their calendar I
> think
> > and they tend to change less often then non-recurring entries
> so
> > hopefully the extra overhead of the Sync CUA getting recurring
> entries
> > not actually within the given time period would be kept to a
> minimum.
>
> Your query would have missed instances with a RDATE, and had
> excessive instances that did not exists because of ignored
> EXDATE and EXRULE. And because you specified a time range
> in the same query based on DTSTART, you would missed any CANCEL
> for objects or instances of those objects.

I think we've covered this. I didn't ignore EXDATE and EXRULE and my
second query had no time range.

> I suspect Sync solution implementors should be OK with this compromise
> as well. Sync'ing recurring entries is difficult and a lot of
> solutions just don't do it properly. Those that do just deal with them
> as one object with their recurrence set and don't worry about
> expansion or if they do sync them all every time since keeping track
> of what instances have been sync'd in a range is just too confusing.
> So I don't think the compromise I am proposing should really harm
> their implementations (lurkers on the list thinking about implementing
> a sync solution against a CAP compliant CS in the future may want to
> speak up now if you disagree).
>
> If we can agree that this is sufficient then the following 2 things
> needs to be done to put this issue to rest.
>
> 1) We still need some text within the CAP spec explaining that CAP
> CUAs should delete things use METHOD MODIFY (as suggested by Doug in a
> previous reply and seconded by me).

        (1) Mark them for delete with <modify>

        (2) Delete them with <delete>

        (3) Cancel entire object with METHOD:CANCEL
            and RECURRANCE-ID IS NULL

        (4) Cancel instances of object with METHOD:CANCEL
            and RECURRANCE-ID IS NOT NULL.

With (3) and (4) currently being defiend by iTIP.

I'm not sure what this text is for? Is this what you want to put in CAP?
We don't want them to zap things with the delete command. You've lost
me.


> > 2) Either the CAP sync
> requirements should be updated  to reflect what
> > we can actually support or perhaps a section within the CAP Spec
> could
> > be added to explain how well it lives up to the requirments. As
> the
> > CAP Requirements document is an official document going back
> and
> > changing it would probably be a pain so my vote is for a small
> > informational section within the CAP spec.
>
> I agree we need more text in CAP. As part of the editors conference
> call last week - I (I think it was me) agreed to write a
> proposal.
>
> FYI - anyone is also allowed to propose text.

I would be willing to work with you on this but we first need to get on
the same page. Wait for my next reply before commenting. I'm pretty sure
I know why we don't really seem to understand each other's
ideas.

--
Mark Paterson
Director, Client R&D
Steltor
mailto:markp@xxxxxxxxxxx
http://www.steltor.com