[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: UID SEARCH responses
On Sun, 12 Aug 2007, Pawel Salek wrote:
I wonder whether it is relevant or not but the copy of RFC3501 that I use 
does not mention UIDs on page 61... Can you please be more specific?
Page 61 of RFC 3501 is the latter part of section 6.4.8. That is the
section that describes the UID command!
"...if no other UIDs exists...". The client may not know whether they do or
do not exist.
Unlike message numbers, the UID sets
are exactly equivalent if no other UIDs exist within that range.
That doesn't matter. The server knows, and that exact sequence will
refer to the messages in question, and only the messages in question.
The information conveyed to client is different in these two cases. I see
that the basically only thing the client can do with the response generated
by your implementation is to send the sequence-set back to the server.
That is what RFC 4731 says as being the purpose of those ESEARCH results.
It cannot use it to eg. purge local cache for message with UID 40000
because it may exist.
Hence there are two other mechanisms to get the UID map which will
accomplish that purpose
I cannot help the feeling you are way too dramatic here. I have no idea how
IETF deals with imprecise specifications but my impression is it is not the
first time there is some dispute how to interpret RFCs.
The IETF generally does not make incompatible changes to a published RFC
(I-Ds are a different matter) that will have the effect of rendering
previously compliant implementations non-compliant. There have been
lawsuits, and threats of lawsuits, over such matters in the past.
The usefulness of the
response generated by current version of UW-IMAP is strongly limited and
considered by me, a client implementer, a bug.
That is your opinion. I disagree. I see no point to having the
complexity of range syntax in returned UID results if it is limited to use
for those cases where there are three or more monotonically consecutive
matching UIDs. I will vigorously fight any ex post facto incompatible
change to the published semantics of ESEARCH.
Please observe it does not say "usable ONLY in other IMAP commands. Users
must be cautious that the UID sequence sets may be collapsed by the server if
separated only by non-existing UIDs". I do not think your argument is valid.
The absence of text in an RFC can not be construed to indicate that the
RFC takes the opposite position.
The absence of text in an RFC indicates just one thing: that the RFC is
silent on that matter.
However (and this is important!!) if an RFC does not forbid something that
is syntactically and semantically valid in the RFC, then that something is
permitted EVEN IF the author of the RFC thinks that it is bad or wrong or
It is true that RFCs can not forbid every possible bad or wrong or stupid
behavior. But if the RFC does not prohibit the behavior in question, and
that behavior can be reasonably inferred from the specification, then the
only way to deal with a conflict of interpretation is either to convince
the other side of the rightness of ones interpretation, or to accept that
both interpretations are valid.
I have encountered that problem innumerable times. The majority of IMAP
client implementations things which I consider to bad or wrong or stupid.
But, because RFC 3501 (and its predecessors) did not specifically forbid
the behavior in question, nothing can be done.
In this case, I consider my interpretation to be reasonably inferred from
the specification AND reasonable in the overall context of IMAP.
But your interpretation cannot be used for the same client-side tasks as
ordinary UID SEARCH response can.
. my interpretation returns a valid sequence-set usable in other IMAP
You can use ordinary UID SEARCH if you need a UID map, and use ESEARCH to
do traffic-efficient searches once you have the map.
. your interpretation prevents optimization unless there are at least
three contiguous extant UIDs matching the result; without such the
ESEARCH response is larger than the SEARCH result.
And that's quite common case, I think.
That assumes that UIDs are assigned monotonically by the server, and that
messages are rarely expunged.
In fact, the implicit requirement in your interpretation that UIDs be
assigned monotonically by the server is the final nail in the coffin.
IMAP has NO such requirement.
This is on top of earlier nails, such as the assumption that sequence-set
means one thing in RFC 3501 and something completely different in RFC
4731. Nothing confuses matters more than to have protocol elements that
change their meaning depending upon context or defining document. People
criticize IMAP (justly, in my opinion) for being weighed down with a mass
Why not try fixing implementation in question and add a clarification in RFC
That only works if the implementor agrees that it is a bug in the
implementation. He does not.
He thinks that including ranges in ESEARCH is pointless otherwise.
He objects strongly to the idea of IMAP having two types of UID ranges
with a subtle semantic difference between the two, not to mention adding a
new hidden requirement to IMAP that UIDs be assigned monotonically. IMAP
already has entirely too many subtle semantic differences and hidden
requirements; this is not a case where "more is better".
I do not see any client
author being happy with UID ESEARCH responses generated by current UW-IMAP -
what could it be used for?
It returns shorter search results if you do not need a UID/msgno map.
You only need to get a UID/msgno map once in a session; after that
additions to the map are a simple operation and removals from the map are
easily calculated. What's more, two other ways exist in IMAP to get the
Searches, on the other hand, are done many times in a session. Why should
those results be weighed down with extra traffic in the assumption that
(1) the client wants to get the map
(2) the client does not want to use the two ways which exist in the base
(3) UIDs are assigned monotonically increasing.
I will be happy, too, with only adding a clarification to RFC 4731 that the
UID ESEARCH responses may include non-existing UIDs for the benefit of
compression. Such interpetation makes the extension less useful for me but is
not a deal-breaker.
Either that, or let it be.
I understand that you would like to have a UID map operation that
compresses monotonically increasing UIDs into ranges, but does not
compress UID ranges that are not monotonically increasing. However, some
servers do not assign monotonically increasing UIDs; and the operation is
pointless on those servers compared to the existing UID map operations.
Furthermore, we need to recognize that IMAP's lock into a 32-bit world
will not be forever. We are already encountering implementations that
incorrectly treat IMAP numbers as being signed (and thus limited to
2GB-1). The first domino that is likely to fall is the QUOTA extension,
followed by the base specifications's limit of message size to 4GB-1
32-bit UIDs are also in trouble. Although 4GB messages in a mailbox seems
unplausible, lots of people have expressed unhappiness with UIDs not being
globally unique and that creates pressure on a 32-bit space. Should the
floodgates of 64-bit be opened, globally unique UIDs will rear their head
and pretty much preclude monotonically increasing UIDs in a single
I don't think that any IMAP extension should be designed with implicit
assumptions that are not in the base specification, but are doomed.
-- Mark --
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.