[Date Prev][Date Next][Thread Prev][Thread Next][Date Index][Thread Index]
Re: [apps-discuss] Comments on Malformed Message BCP draft
Tony Finch wrote:
Dave Cridland <dave@xxxxxxxxxxxx> wrote:
It may be that the discussion suggests rejecting, in which case I suggest the
document should clearly explain why, and what the implications of not doing so
are, beyond "it makes some problems harder to diagnose".
It does not make sense to have a uniform policy for dealing with corrupt
messages. Some kinds of corruption are caused by common legitimate
software, in which case you will want to treat it leniently; others are
caused by malware or rare kinds of incompetence, in which case it makes
more sense to reject. You can only determine which is which based on
operational experience, and the least-worst response changes from time to
My view is that since only Good Guys complain, Bad Guys do not, there
are some well established areas for strong compliancy checks that has
proven to have a high efficiency bad mail filtering rate with little
false positives for good guys. This is because most good guys are
compliant, especially for these particular areas of compliancy.
Others are more subjective, so it generally it becomes a decision to
how its enabled - on/off by default.
Traditionally, the transport is a passive payload transfer system but
has have options for decisions than it does in applying subjective
decisions related to the payload.
I think good examples are:
- anal strong syntax checks, i.e, "MAIL FROM:space"
- Checking for explicit EHLO [ip literal] matching connection IP
requirement. Exceptions used for growing NATS related issue
in SOHO market with more than 1 PC. PORT 587 can be used
to skip this check.
- Return path verification - methods in debate, but it is
increasingly recognized that the validity requirement
has become more important in the area of bounce avoidance.
822/2822/5322 PAYLOAD LEVEL
The first thing here is making sure we have a valid message per RFC
822/2822/5322. I listed all three because it depends on the receiver
on how it determine a valid message.
First, many systems don't need any RFC headers at all. Just the body
and will automatically fill in required RFC fields:
From: <--- SMTP Return Path
To: <--- SMTP Forwarding path
Date: <--- Reception Timestamp
822 systems require From:, To/Cc:, Date:, 2822 systems relaxed it to
From:, Date and it remains this way with 5322 systems. I recall
reading long ago an I-D by Keith Moore to even further relax it to
remove the From: requirement. That would of been terrible.
Anyway, when some of these are missing or none, the design question
becomes whether the missing headers should be filled in or rejected,
always or sometimes. For some, the answer was straight forward, for
others it was not.
I will note that with RFC2821bis/RFC2822bis, we explored enabling
stronger compliance (default on) and almost immediately, we got a few,
not a lot, reports and that was enable the default to a FILL IN to
avoid any future support overhead. I believe it was 2-3 reports
only, and I recall one report because the issue here was scenarios
where two identical list messages (by content and context) for two
users at a host and one fail and the other did not was found to be
because both took different mail processing distribution paths.
I honestly don't recall the details of the incorrect headers, but the
main thing was how the differences occurred. Following the trace of
both, it was clear the sender was using two path distributions, where
one path was using the original non-compliance software path and the
the second was a new beta software path (I recall the "Beta" term in
Received lines). So it appeared the sender was aware of the
distribution problem and was migrating to more RFC5322 compliant
software. Of course, while this is all interesting to the customer,
he could care less and wanted a solution now. I forget what was done
Anything beyond this has not really been a big issue for us. Some of
the examples listed in the report are interesting to note, but I can't
see a real justification to add time and energy to automate
corrections when the most practical reality by far is majority of the
good guys are compliant. If an issue does occur, and the issue is
related to broken legacy software, then you have no choice by to help
solve the problem. But if the offending software is active, then you
have to weigh all the factors on how to feasibly serve the customer.
Severity and frequency of problem. It is more than one, is the
offending active vendor aware and have solution you can research. Does
the customer have access to the offending senders? Can you provide a
work around, etc, etc, etc.
Just consider the report malformed message scenarios with the space
for colon, this will require you to change YOUR software, not trivia
for everyone, when parsing RFC headers to one that checks for two
I am not sure I really want to do that. It will fall into the same
engineering compliance debates with SMTP space issues:
Mail From: <address>
The difference here is that it was well known that the #1 MUA Outlook
was doing this and most systems could not ignore that fact. Today, I
believe these MUAs were corrected and you don't expect it, so you can
probably get away (to a higher degree) with enforcing the syntax.
But do we have such a high level of the malformed 5322 header with a
space in it, in particular from any big brand vendor? I'm not seeing