[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]

Re: Extended RETR - Draft posted

> > "The offset is given in octets. Everything transferred with a standard
> > RETR command should be accounted for."
> The sentence following the above sentence in the draft states:
> "This means that top headers, followed by messageparts will be
> transferred, in that order."

Yes, but I didn't read it as "byte-stuffing will not affect offsets",
the "everything" seemed to imply the opposite.

> > I am assuming the "everything" applies to byte-stuffing.
> Read the above quote.

So it needs to be clearer.

> > The question: what offset should it request this time?  It could have
> > stored the amount of data it has received, along with the incomplete
> > message.  However, that amount is one octet larger than what it would
> > be if the transfer wasn't interrupted the first time.  Obviously, the
> > server, according to this definition of the extension, expects to see
> > offsets into the data that would be "transferred with a standard RETR
> > command" -- that is, without any previously recovered transfers.
> OK, this is something I thought about when I was drafting the memo. I
> decided not to include an explanation because I saw it as obvious.
> Having given it some more thought, I think a more thorough explanation
> and an example might be in order. I will probably add a section
> explaining how byte-stuffing and calculation of offsets should be done.

Yes, it seems we need that.  There's one more issue that you'd need
to decide on -- are we calculating CRLF's the way they're transferred
over POP3 (would be more essential for the protocol), or the way line
separators are typically stored on a UNIX-based server (to allow for
a simple seek).

> The model should be very simple, please correct me if I'm wrong:
> msg-offset-bytestuff-server-client-bytedestuff-msg

Unfortunately, there may be a few more conversions:

server_mailbox -> skip(*)_to_offset -> bytestuff -> LF_to_CRLF -> ...
... transfer -> bytedestuff -> convert_to_client_mailbox_format

(*) we haven't defined the "skip" for the server_mailbox "layer", yet.

> Example:
> Message number is '1'.
> Message size is 217,743 bytes.
> Offset 113,435 starts with a '.'.
> Lets assume our transfer got interrupted after 113,434 bytes. Now, the

To illustrate the case of having to byte-stuff, you'd want the
transfer to be interrupted after 113,435 bytes.

> received data has been bytedestuffed and saved, thus the client executes
> 'RETR 1 113435'. The server opens the message and jumps to this offset.
> It now bytestuffs the '.' as usual and transmits the data to the client
> who bytedestuff the data and adds it to the incomplete data it received
> earlier.
> > "       a) Less consumption of bandwidth
> >         b) Less resources used on the server-side
> >         c) Less time used online for the client"
> > 
> > I would consider (c) the most important reason.  Bandwidth isn't that
> > important here, as POP3 transfers are typically within one ISP, from
> > a mailbox at the ISP to the ISP's dialup line, which is busy anyway.
> You have a point about bandwidth, in some cases, not being a problem.
> However, taking in account the fast evolution on the internet, this
> might in the future apply to other connections than modems, i.e. a
> company's internal network, since attachments might be much larger than
> they are today.

Yes, that's a valid point as well, but, in my opinion, (c) is more

> > (b) is not fully achieved with your definition of the offsets, as the
> > server has to read the entire message from the mailbox to account for
> > byte-stuffing.
> Well, this shouldn't be a problem with the above explained model. The
> server just need to do an fseek() to the specified offset.

I wish it was that simple.

To allow for a simple seek, we would need to define that offsets are
into the message as stored in a BSD-style mailbox.  (I am assuming
that performance is less critical on non-UNIX servers, and that they
can choose the mailbox format that allows for better performance if
they care.)  The calculation of offsets on the client end would get
more complicated, then -- unless the client uses single LF's for its
mailboxes as well.  Maybe it's not that bad, after all.

Solar Designer