Unsolicited Bulk Email: Mechanisms for Control

prepared by
Paul Hoffman, Internet Mail Consortium
Dave Crocker, Brandenburg Consulting

Internet Mail Consortium Report: UBE-SOL
IMCR-008, revised May 4, 1998

Table of Contents

  1. Introduction
  2. Overview of Proposed Anti-UBE Mechanisms
  3. Origin-based Heuristic Filtering
  4. Message-based Heuristic Filtering
  5. Enforcement by Regulation
  6. Enforcement by Contract
  7. Labeling the Data
  8. Recipient Registration
  9. Accountability
  10. Conclusion

1. Introduction

As the Internet moves into increasingly commercial areas of use, an activity which entails considerable emotion and complexity is the use of direct marketing. The common term for this kind of activity on the Internet is "spam", encompassing commercial and non-commercial sending of bulk email.

The topic of unsolicited bulk email (UBE) has become highly emotional and political. Neither factor is conducive to careful discussion. The Internet Mail Consortium wishes to encourage careful deliberation in the public debate over this phenomenon and is developing a series of reports to facilitate that care.

A previous IMC report (Unsolicited Bulk Email: Definitions and Problems, available from the IMC Web site) discussed the different types of UBE, to distinguish their nature and effect. This current report reviews the many technical and legal alternatives for controlling the occurrence or handling of UBE that have been discussed to date. It is not the intent of these reports to recommend particular positions or actions; rather, it is our hope that these reports facilitate informed debate.

These reports are intended to be living documents. For a topic developing as rapidly as unsolicited bulk email, new ideas and mechanisms are emerging constantly. Readers are encouraged to provide feedback to the IMC so that future versions of this report can be more complete.

It is essential to understand that that none of the mechanisms described in this report is perfect. Even the solutions that might help eliminate the most obnoxious of today's UBE may not work well in the future because UBE senders will use tools to circumvent new methods used to prevent their mail from getting through. Today's UBE senders often already use many techniques that circumvent some of the solutions described here, and it is likely that they will simply create new techniques as better methods for stopping UBE appear. Even legal solutions to UBE can be circumvented, and many UBE senders already take advantage of the legal system's slow pace in addressing problems like UBE. This report does not list specific implementations of the proposed solutions, but instead gives an overview of the solutions themselves.

This is the second edition of this report. After the first edition was issued in October, 1997, IMC received many thoughtful comments and suggestions for the report from a variety of people. Many of those suggestions have been incorporated here. A summary of the additions is:

1.1. Definitions

The earlier IMC report on UBE gave in-depth descriptions of many of the important terms in the UBE discussion. This section gives a brief overview of the people and systems involved in the sending and receiving of UBE.

An originator creates a message and sends it to a list of recipients, who are Internet users; the originator might also send the message to one or more Mailing List Agents (MLAs) with the intention that those MLAs would send the message on to people who were subscribed to the associated mailing lists. The message first goes to the originating host, typically a computer directly attached to the Internet and running the SMTP protocol. The originating host is possibly operated by the originator or a contracted Internet Service Provider (ISP), but may also be operated by an unsuspecting third party, used to relay the message and hide source information about it, if the originator is dishonest.

To process all the mail, the originating host determines the publicly-known receiving host for each addressee. The receiving hosts may be computers that actually store the message for the addressees, or they may be relays for the addressees, such as firewalls or corporate mail receivers that distribute messages to receiving hosts internal to the company. Once a final receiving host receives the message, it stores the message on a message store (usually a hard disk) so that the recipient can access it later. Recipients use client software to receive their messages from the message store.

1.2. Other anti-UBE resources

UBE is being discussed in many communities on the Internet. There are many good resources for people investigating the various aspects of UBE. IMC's anti-UBE resource page has other IMC reports about UBE (such as our survey of the use of SMTP relays), legal research about UBE sponsored by IMC, information about what IMC's members are doing about UBE, and links to other good anti-UBE sites.

2. Overview of Proposed Anti-UBE Mechanisms

Because of the great aggravation caused by UBE, thousands of people have devised methods to reduce it, both for themselves and others. These proposed solutions cover a wide range of ideas, and generally fall into three broad categories:

Another way to look at this is to ask whether an originator is allowed to send UBE. If they are, the recipient or the recipient's host must filter; if not, then there must be some legal prohibition on sending UBE. The argument for requiring legal force is that it is necessary to counter the very strong economic incentive present for originators of UBE. That economic incentive can only be reduced by changing the economic basis for all Internet mail, as described below.

The distinction between legal solutions is, unfortunately, not clear-cut. Some of the specific solutions described in this report cross the categories; for instance, some of the proposed legal solutions rely on filtering as well. Still, these categories give a reasonable framework for comparing and analyzing the anti-UBE mechanisms.

2.1. Filtering

Most of the work in trying to stop UBE has been in the area of filtering. Filtering proposals can be divided into two broad categories:

2.1.1. Heuristic filters

The majority of existing efforts to filter incoming UBE presume that it is possible to detect such messages without the cooperation of the originator. Some commercial packages for doing filtering have thousands of rules. Unfortunately, originators of UBE are highly creative and are likely to develop sophisticated technical mechanisms to fool all but the most sophisticated filtering systems.

The heuristic approach fall into two categories:

Origin filtering happens before a message has been fully received by the recipient's host computer, and is based in attributes such as the domain name and IP address of the originator. Message filtering happens after a message is received; attributes for message filtering can include the same as those for origin filtering, but can also include things like known keywords, "tip-off" headers, and so on. Examples of tip-off headers include headers that indicate that the message was sent in bulk or in a way that is hiding its true origin. Some message filters also work by only allowing messages from previously-known senders to get through.

2.1.2. Cooperative filters

In addition to origin and message heuristic filtering for tip-offs that a message is UBE, it is possible to have collaboration between UBE originators and UBE recipients. Two entirely different schemes fall into this category:

With content labeling, filtering is done by the recipient or the recipient's host; with recipient registration, the filtering is done by the originator.

For content labeling, messages can contain additional information supplied by the originator, such as the type of the content or an assurance of the originator's identity. When the originator cooperates by assigning useful labels, these messages can then be definitively filtered with full assurance that the filtering for those particular messages will be effective at preventing the UBE from getting to the recipient. However the recipient can also specify that they in fact desire labeled UBE.

Another proposal for reducing UBE, commonly called "opt-out", requires that the user inform either individual senders or keepers of universal lists that they don't want to receive UBE. This becomes a type of filtering, where the UBE originator filters who they were going to send to against a local or outside list of potential recipients. This type of originator filtering only works for honest and/or conscientious UBE originators, of course. Dishonest UBE originators would simply pretend to honor lists and then send the UBE anyway; other UBE originators would not even pretend to honor the lists. Because different forms of filtering can cause different negative side-effects, each type of heuristic filtering is described in this report separately. The sections later in the report describe what the filtering choices are made on, how effective that type of filtering is for blocking honest and dishonest UBE, the type of coordination that is needed for effective filtering across the Internet, the potential for information loss, and the impact of the filtering on recipients (which is, after all, the reason that anti-UBE tactics are being pursued.) Of course, the positive impact on recipients is not seeing unwanted UBE.

2.2. Legal

Many of today's UBE originators have shown little regard for honesty or normal business practices. Thus, many people think that the only way to prevent them from sending UBE, even if other anti- UBE mechanisms are employed, is through legal means.

For postal mail, direct marketing has relatively well-established practices, along with relatively long- standing regulation. However, UBE over the Internet is very new and is essentially unregulated. Of course, the Internet crosses many local, regional, and national boundaries, so any application of law to the Internet is sure to be problematic. This severely limits the ability to develop a coherent set of laws that can be applied to all (possible) source of UBE.

Legal controls are achieved through two different paths:

To date, neither of these avenues is significantly exploited, so that specific alternatives are almost entirely theoretical.

2.2.1. Regulation

The nature of a law concerning UBE is to specify behaviors which are prohibited and provide for sanctions against violation of the proscription. Discussion of these kinds of controls focuses on two actors:

Control by the originator simply says that certain kinds of UBE shall not be posted to the mail system, or that certain kinds of UBE shall not be posted to certain kinds of recipients. Control by the relay says that the operator of an email relay service (including final delivery to the recipient) shall reject all or some UBE. A major difficulty with the development of either type of law is development of precise and useful definitions of the proscribed UBE. A particular danger is that a definition will cut too wide a swath. That is, an effort to control one type of content could turn out to include other, very different types of content.

2.2.2. Contracts

Legal solutions for UBE are not limited to passing new laws or enforcing current laws in a new territory. Some of the proposed UBE solutions involve contractual agreements between all senders of email and their Internet connections. By filtering for service providers who enforce anti-UBE agreements, a recipient can have more assurance that the mail that they receive will have less UBE, and that the UBE they do get will probably cost the sender a significant amount in penalties. Similarly, an association of mail relayers who have anti-UBE agreements with their senders may agree to mark email from that association so that a recipient would have a greater trust for that mail than for unmarked mail.

2.3. Economic

By its very nature, Internet mail costs little to send and little to receive unless the sender's or recipient's access charges are high and are sensitive to the duration of a session. Because of the low cost of sending, many people feel free to create email messages to known recipients; because of the low cost of receiving mail, most recipients freely accept all incoming messages, even though they will immediately throw away.

UBE senders rely on the low cost of sending large amounts of Internet mail. An often-suggested method for reducing UBE is to make it more expensive to send mail. A variety of methods for charging for outgoing mail have been suggested, including ones that allow recipients to return part of the sending fees. Some analysts have also pointed out that this kind of usage-based charges will make using the Internet more like other communications systems today.

This report does not look into these proposals in any depth because charging for sending Internet mail would cause such a fundamental change in the style of communicating, it would no longer be Internet mail. These proposed changes would require a massive financial infrastructure for very low-cost transactions, and every participant in the new system would have to allow tracking of all their mail in order to allow fair billing. Further, this type of system would eliminate the kinds of email that have attracted many people to the Internet in the first place, particularly open mailing lists and other kinds of mail-based discussions. These limitations are just a few reasons why the proposed fee-based sending has not garnered much interest from implementors.

Note that if sending UBE becomes illegal, it is likely that many senders of UBE would be willing to pay to send bulk email to recipients who were willing to accept payment. At that point, the mail is by definition no longer unsolicited, since acceptance of payment is a way of saying you want the mail. Payment methods and amounts would be based on market acceptance, and it is possible that today's noxious UBE would disappear and be replaced with commercial mail that is similar to typical bulk postal mail. Recipients who didn't want to receive any of this type of commercial mail could easily refuse all such mail, or could accept mail only from senders willing to pay a very high rate.

3. Origin-Based Heuristic Filtering

Origin-based mechanisms distinguish entire groups of UBE originators, usually everyone from a particular ISP or a particular domain. The rules used rely on originators to use the same or similar addresses each time they send UBE. Origin filtering prevents mail from UBE originators from being saved in the destination host's message store, and can reduce the load on destination hosts by reducing the amount of interaction they have with UBE originators.

The four methods described here are:

Note that it is possible to combine two or more of these techniques on a single server.

There are many groups on the Internet collecting information about UBE senders. They make this information available in a variety of formats that can be automatically be used by filters. Many of the groups are cooperatives, in that the members contribute names and addresses of UBE senders on messages they have received but are not listed on the group's list. These lists are often inspected to minimize the number of non-UBE senders that appear on the lists.

3.1. Refuse IP connections from known UBE originators

This technique, also know as "black holing", is based on telling routers at the local site to not route IP packets from a list of addresses that correspond to UBE originators.

Information filtered on IP address of connection.
Effectiveness against honest UBE High after first UBE is sent. It is expensive to change IP addresses often.
Effectiveness against dishonest UBE Low. It is easy to misappropriate the services of unaffiliated SMTP servers to relay UBE, thereby using IP addresses that might be legitimate originators.
Information that must be distributed List of IP address ranges for mail hosts with UBE originators.
Potential for information loss Users in the filtered IP address ranges will be denied access to all Internet services at the destination site, and vice versa. This includes the ability to send any email to users inside the filter or to access any Web server inside the filter. A mistaken inclusion of a site or set of sites in the filter list can prevent anyone from that site contacting the filtering site to ask them to correct the filter.
Impact on recipients Users cannot know that a delivery attempt from the banned site was made. Users cannot receive any mail from prohibited addresses, and in fact cannot send mail to those addresses either. If the user tries to send a message to a banned site, their sending host will typically hold outgoing mail for three to five days, then return it to the originator as undeliverable.

3.2. Refuse TCP connections from known UBE originators in the SMTP server

Modern SMTP servers can be configured to look up the IP address or domain name of a originator as it connects to the SMTP server; the domain name is determined by a reverse lookup of the IP address. If the originator is on a list of prohibited sites, the SMTP server can refuse to accept any SMTP commands. Thus, the filtering is performed immediately after the TCP connection is opened, before any SMTP commands are exchanged.

Information filtered on IP address of connection or domain name returned from reverse lookup of the IP address.
Effectiveness against honest UBE High after first UBE is sent. It is expensive to change IP addresses or domain names often.
Effectiveness against dishonest UBE Low. It is easy to misappropriate the services of unaffiliated SMTP servers to relay UBE, thereby using IP addresses and domain names that might be legitimate originators.
Information that must be distributed List of IP address ranges and/or domain names for mail hosts with UBE originators.
Potential for information loss No mail from the sending site can be received. A mistaken inclusion of a site or set of sites in the filter list can prevent anyone from that site contacting the filtering site to ask them to correct the filter.
Impact on recipients Users cannot know that a delivery attempt from the banned site was made. Users cannot receive any mail from prohibited addresses, although they can almost always send mail to sites they are prohibited from receiving from. This can lead to a situation where a user sends a message and never receives a reply, but doesn't know why.

3.3. Refuse SMTP messages from known UBE originators at the MAIL FROM

Modern SMTP servers can be configured to check the domain name given by a sending SMTP server during the MAIL FROM command in SMTP. If the originator is on a list of prohibited sites, the SMTP server can refuse to receive a message. This filtering is performed immediately after the MAIL FROM command, before any message is transmitted.

Information filtered on Domain name announced by the sending SMTP server.
Effectiveness against honest UBE High after first UBE is sent. It is expensive to change domain names often.
Effectiveness against dishonest UBE Low. It is trivially easy to lie about the domain name in the MAIL FROM command. It is also easy to misappropriate the services of unaffiliated SMTP servers to relay UBE, thereby using domain names that might be legitimate originators.
Information that must be distributed List domain names of mail hosts of UBE originators.
Potential for information loss No mail from the sending site can be received. A mistaken inclusion of a site or set of sites in the filter list can prevent anyone from that site contacting the filtering site to ask them to correct the filter.
Impact on recipients Users cannot know that a delivery attempt from the banned site was made. Users cannot receive any mail from prohibited addresses, although they can almost always send mail to sites they are prohibited from receiving from. This can lead to a situation where a user sends a message and never receives a reply, but doesn't know why.

3.4. Refuse SMTP messages if the domain name and IP addresses do not match

Modern SMTP servers can be configured to perform a domain name search to find the IP address associated with the domain name, then check if that address match the IP address of the TCP connection. If the two IP addresses don't match, the SMTP server can refuse to receive a message. This filtering is performed immediately after the MAIL FROM command, before any message is transmitted.

Information filtered on Whether or not the IP address of the domain name given in the MAIL FROM command matches the IP address of the current connection.
Effectiveness against honest UBE None. All honest UBE will pass the check.
Effectiveness against dishonest UBE Low. It is easy to misappropriate the services of unaffiliated SMTP servers to relay UBE, thereby using IP addresses and domain names that might be legitimate originators.
Information that must be distributed None.
Potential for information loss No mail from the site can be received. Misconfigured SMTP servers are common, and if the administrators of the sending server is not watching the logs, they will not notice that some of their mail is being refused. Large sites that use this type of filtering report daily instances of finding accidentally misconfigured SMTP servers.
Impact on recipients Users cannot know that a delivery attempt from the site was made. Users cannot receive any mail from those servers, although they can almost always send mail to sites they are prohibited from receiving from. This can lead to a situation where a user sends a message and never receives a reply, but doesn't know why.

4. Message-Based Heuristic Filtering

These mechanisms attempt to distinguish special content, tell-tale headers, style of addressing, and so on, for partitioning regular mail from UBE in the recipient's own environment. The rules for filtering can include domain names and IP addresses, similar to the coarse filtering described earlier. The rules can also be based on whether the recipient has previously received mail from the sender and wants to continue to do so.

Specific filtering occurs in two places:

The main difference between these two is that filtering in the message store usually removes the message from the message store or labels it in a way that the mail client can recognize.

Implementations of specific filtering must decide what to do with messages that get caught by the filter. If the filter simply throws out the message, it runs the risk of losing non-UBE that simply looked like UBE, or throwing out UBE that the recipient actually wanted. Even if the filter doesn't throw out the UBE, but merely sets it aside for later reading by the recipient, misfiltered mail can easily get lost among the UBE that it appears in.

4.1. Filter messages in the message store

A receiving host can scan each message and filter out suspected UBE. It can also periodically scan the message store, for example when it has updated its list of rules about what might be UBE, to see if any of the received but unretrieved messages should be marked as UBE.

Information filtered on A wide variety of filters are available, including domain names, format of particular headers, key words at the beginning of the content, message ID numbers, and so on.
Effectiveness against honest UBE Mixed. UBE that is from a new originator or is about a new topic may not be caught, but UBE that is sent repeatedly is easy to catch.
Effectiveness against dishonest UBE Low. It is simple to cloak UBE to look like legitimate mail, and to change enough of the content of each message sent that patterns cannot be found.
Information that must be distributed Patterns for the headers and content of known UBE messages.
Potential for information loss If a filter accidentally deletes a legitimate message, the recipient will not know that the message was ever sent. However, this can be mitigated by the filter listing header information for all UBE that was removed, or by having the filter not really remove messages, but simply to put them aside and made available through different means.
Impact on recipients Some legitimate mail might be lost or mislabeled. There might be a delay between when mail is received and when it is available to the recipient, but this is likely to be extremely short (under a second).

4.2. Filter at the mail client

A mail client (such as one using POP or IMAP) can scan each message as it comes in from the message store and filter out suspected UBE. This happens as the recipient is receiving messages.

Information filtered on A wide variety of filters are available, including domain names, format of particular headers, key words at the beginning of the content, message ID numbers, and so on.
Effectiveness against honest UBE Mixed. UBE that is from a new originator or is about a new topic may not be caught, but UBE that is sent repeatedly is easy to catch.
Effectiveness against dishonest UBE Low. It is simple to cloak UBE to look like legitimate mail, and to change enough of the content of each message sent that patterns cannot be found.
Information that must be distributed Patterns for the headers and content of known UBE messages. Getting this information to end users is more difficult than getting it to filters that run on message stores.
Potential for information loss If a filter accidentally deletes a legitimate message, the recipient will not know that the message was ever sent. However, this can be mitigated by the filter listing header information for all UBE that was removed, or by having the filter not really remove messages, but simply to put them aside and made available through different means.
Impact on recipients Some legitimate mail might be lost or mislabeled. There might be a delay between when mail is received by the recipient and when he or she can read the rest of their mail, but this is likely to be short, depending on the speed of their computer.

4.3. Filter based on previous acceptance of mail

A mail client can filter based on whether the user had previously accepted a message from this sender. This happens as the recipient is receiving messages. If the sender is new to the recipient, the recipient's client somehow "challenges" the sender to prove whether the client should accept the message. Some of the types of suggested challenges include just a response, a response that takes computational power (computing a problem given in the challenge), and monetary payment.

This challenge process inherently assumes that a UBE sender would not be able to (or not bother to) answer the challenge, and that all other senders would be able to and would want to respond to the challenge. If the challenge process is too easy, UBE senders will set up responders to reply to the challenges; if the challenge process is too difficult, human senders from whom the recipient probably wanted to hear will not reply to the challenge, and the recipient will not get their mail.

Information filtered on Sender's address, compared to a database of acceptable addresses that the recipient has built based on previous mail.
Effectiveness against honest UBE Mixed. UBE senders that set up responders to the challenges will still get their UBE through; senders who don't set up challenge responders will not be able to get through.
Effectiveness against dishonest UBE Mixed. UBE senders that set up responders to the challenges will still get their UBE through; senders who don't set up challenge responders will not be able to get through.
Information that must be distributed None. This method relies only on local history, although it could be augmented with list of UBE senders whose challenge responses should be ignored.
Potential for information loss Very high. Depending on the type of challenge, a legitimate sender might not want to, or may not be able to, reply to the challenge. Further, if the challenge is lost in the mail due to processing errors, the original message will not be read.
Impact on recipients Some legitimate mail might go unread if the sender does not reply to the challenge. All initial contact will also include challenges, which might make the original sender less interested in communicating with the recipient. This will particularly impact cross-cultural exchanges.

5. Enforcement by Regulation

To date, none of the heuristic filtering mechanisms have enjoyed enough use to afford recipients with much freedom from UBE. Because many of these filtering mechanisms have significant side-effects, it is unlikely that they will have long-term effectiveness, in spite of their wide appeal. Even if the general populace of recipients and receiving hosts decide to use heuristic filtering, UBE originators will simply work harder to get around the filters and are likely to succeed.

Legislators, most notably in the U.S., have taken notice of the UBE problem, and laws have already been passed in some locations. In order to be effective, the laws must define and proscribe specific behaviors. This requires that the legislators understand the technical nature of both the problem and the enforcement of the laws, and that the enforcement agencies understand and have the ability to prosecute transgressors. To date, many people are skeptical that, even if well-crafted laws could be passed, that they would be enforced with enough vigor to deter senders of UBE.

The biggest problem associated with legislation about UBE is the fact that the Internet by and large ignores political boundaries. For instance, some states in the U.S. are already considering (and, at least one case, passed) legislation regulating UBE. However, intrastate communications pretty much ignore local content-specific rules. The same is true for national laws: the content on the Internet is global and it would be impossible to limit originators based on their location.

Current laws and proposed legislation may help some of the technical anti-UBE methods be more effective. Specifically, most of today's UBE purposely misrepresents who the UBE sender is, both by incorrect mail headers and by originating at an SMTP server not affiliated with the sender. These actions could easily be seen as fraud, trespass, and unlawful conversion; some proposed legislation makes the use of these techniques more clearly illegal. If UBE senders were effectively prevented from using this kind of deception, they would have to identify themselves more honestly, and heuristic filters would be able to identify them more accurately.

5.1. Types of anti-UBE laws

Because UBE is fairly new and most legislatures are fairly slow, there is little experience with anti- UBE laws. To date, most of the legislation has been in the U.S., and at the time this is being written, no national laws have passed.

The proposals fall into two categories:

The latter is particularly worrisome to many people because it means that specific heuristic filtering must be in place in order for the law to have any effect; users without such filtering would likely receive even more UBE than they do now. Many also worry that once we have mandated content labeling for some Internet traffic, labeling other types of traffic is sure to follow. Also, such mandatory labeling doesn't do anything to reduce the cost on relays or recipient hosts.

Even in the case of outright bans on UBE, it is expected that there will be protracted lawsuits about the definitions used in the laws. Defining "bulk" and "unsolicited" will certainly entail compromise, and it's not clear how much UBE will be reduced in the long term if today's fly-by-night UBE is prohibited but more "legitimate" originators step in.

5.2. Enforcement of anti-UBE laws

There are two approaches to the enforcement of laws. One has agents who prevent the occurrence of the offense and the other is to apply a penalty to those who transgress. For example, in automobile speed control, the individual driver is responsible for prevention, with the police detecting transgressions.

5.2.1. Prevention

For Internet UBE, there appears to be little hope of having a broad base of conformance by individual originators. This has led to an effort at making relay service providers responsible for detecting and preventing the transmission of UBE.

The difficulty with relay control is both technical and social. The technical problem involves the difficulty of an email relay service provider in distinguishing one type of email from another. The social difficulty is with having an email service provider inspecting each user's messages. In many jurisdictions, this is considered to be an invasion of privacy and may be illegal.

5.2.2. Enforcement

When exploring new legal territory, as with development of new engineering, the process is often facilitated by re-using well-established practice. In looking for an established law which might translate well to the world of UBE, perhaps the most obvious example pertains to U.S. regulation of the sending of unsolicited advertising to telephone facsimile devices.

The premise behind the fax control is that unsolicited advertising consumes the recipient's resources, both scarce communication service and costly ink and paper. At first it might seem that email is subject to neither concern, yet there is strong indication to the contrary. Current email handling technology for users does not allow very high quality separation of UBE from other email, so that the recipient must spend significant time doing the separation by hand and, in the process, there is a tendency to lose non-UBE mail. Further, many users pay for their email access and UBE costs them money. An extreme example are researchers at sea who pay very high satellite charges for email access and for whom UBE represents an extremely damaging drain on the budget.

There are two general methods for bringing action in cases such as anti-UBE laws: have it done by the recipients of the UBE, or have it done by the government. Current legislation covers both of these options, although it seems likely that few governments will view anti-UBE laws as an important use of their limited prosecution resources. Some laws also allow those running receiving hosts to pursue UBE senders, while other laws make the recipient responsible for bringing the lawsuits.

A contentious issue being faced by legislators who want to control UBE is the question of jurisdiction. The U.S. is typical of many countries in that it has different enforcement agencies for commercial laws (the Federal Trade Commission) and communications laws (the Federal Communications Commission). Should the anti-UBE laws be enforced as commercial acts or as communications?

Of course, in many countries, both types of bureaucracies would love to be given the task, and are quite willing to fight for it. Control over some part of the Internet is clearly something with long-term potential, and being seen as the "good guys" at this early stage would help establish a regulatory foothold. However, each country's bureaucracies have different track records for their effectiveness, much less their understanding of the Internet, so a legislature often has a tough decision when they create anti-UBE laws that they actually want enforced.

6. Enforcement by Contract

Not all legal remedies have to come from legislation. For example, most ISPs in the U.S. require their customers to agree not to send UBE from their service. These agreements, while legally binding, are often difficult to enforce due to the requirement for reliably identifying originators of UBE and in proving the level of damage to the ISP. Even so, some companies are actively pursuing litigation against UBE originators that violate their license agreements.

ISPs who have strong service contracts and actively prosecute UBE originators could form groups, again by contract. Recipients might want to filter their incoming mail and prefer mail from ISPs in those groups. Of course, most users would not want to reject mail just because it didn't come from an anti-UBE ISP, but this type of preferential filtering could help give positive value to anti-UBE service providers.

7. Labeling the Data

Some UBE solutions have called for standardized labeling of UBE. The obvious requirement for labeling UBE at the source is cooperation by the originator. This might come from legal restrictions on UBE, or it might be voluntary, showing that a originator is being honest about their mail as a way to generate good will among recipients.

Labeling alone is not sufficient to reduce UBE; it must work in coordination with filtering. However, labeled mail is significantly easier to filter than unlabeled mail, since the filtering software can use definitive rules to check for labeling. More importantly, labeling permits accurate, definitive separation of UBE from non-UBE. There are two proposed methods for labeling UBE:

The first attaches labels directly to the messages, independent of its means of carriage. The latter identifies different data conveyance mechanisms or paths for different types of data. For example, non-UBE might be sent to the usual SMTP TCP "port" and UBE might be sent to a different port.

The kind of additional information that can be added by the originator falls into two categories:

Explicit labeling that the content is UBE makes filtering for UBE a simple task. The originator either uses a standard method to state in the message that it is UBE, or sends the UBE using a different mail protocol that is reserved for UBE.

Another method that aids definitive filtering is giving the recipient assurance that the originator is who they claim to be. Just as anyone with access to a mailbox can send postal mail, anyone with an Internet connection can send Internet mail. Accountability is desired because, in both systems, full accountability of originators would reduce the amount of bothersome mail being sent because recipients could more easily find the originator to complain. Accountability is discussed in Section 9 of this report.

7.1. Object labeling

Labels can indicate the "class" of the message, such as "commercial", "unsolicited", "for adults only", and so on. Proposals for this kind of labeling have almost all been within the RFC 822 headers for the entire message. Some proposals have been for new headers, while others have proposed reuse of common headers like "Subject".

Adding content labeling to distinguish individual components of the body of Internet mail messages could be difficult for messages which have a structure that does not allow easy modification for labels. On the other hand, messages which have their content structured using the MIME standard could employ a envelope header such as "Content-Label" to distinguish the nature of individual components of the message. Although it permits very fine-grained labeling, this approach would be expensive to employ for receipt-time filtering, since it requires thorough inspection of the entire message before deciding upon its disposition.

With either new headers or re-use of current headers, all filtering software must be aware of the rules used in labeling. This means that recipients who do not have filtering performed for them will receive all UBE.

7.2. Channel labeling

There are many ways to send a message. Some proposals would force (or simply allow) UBE originators to use a different TCP port for UBE than is used for normal Internet mail. Because this port would be reserved for UBE, a receiving host could decide on a case-by-case basis whether to accept the message for one or more of the recipients who use that host.

This type of labeling by delivery channel opens up interesting possibilities. If UBE is legally banned for normal Internet mail channels, using an alternative channel allows recipients who want to receive UBE to get it. Recipient hosts could choose whether to offer UBE receipt as a service to their clients. In fact, recipient hosts could make arrangements to only accept UBE that they have been paid to accept; they could then keep the money or even share it with their customers.

7.3. Choosing labels

Most proposed labeling requires the originator to determine the class of the message. Unless labeling is required by law, there is no one to adjudicate whether a particular originator mislabeled or failed to label a message properly.

The term "commercial" could have different meanings in different places. In some places, offering something free to someone has a different meaning than offering to sell them something, even if the free item or service is a lead-in to a commercial appeal. A significant portion of the UBE at the time that this report is written consists of statements to the effect of "come to this Web site". Since that act doesn't cost the recipient anything, some might not consider that "commercial".

There will certainly be debate about the exact meaning of other desired labels, particularly "unsolicited". What must a recipient do to in order to express a desire for UBE? Some say that simply visiting the Web site of the originator shows interest in that vendor's products. Others say that the recipient must actively ask to receive periodic messages. In the postal world, very little bulk-rate mail is actually solicited: most is sent to people who have not expressed any interest in receiving the mail, or only expressed interest in a related product.

8. Recipient Registration

Before it was clear how pervasive UBE would become, some people proposed there be a simple way for people to ask not to receive any more UBE from a particular originator; they believe that this would greatly diminish the problem. In the world of postal and telephone direct marketing, such explicit registration is called "opt-out" and is reasonably effective. The established direct marketing world primarily relies on a few centralized services which register people who opt-out, and these services then furnish the list of such people to marketers.

Proposals to have potential recipients register for Internet mail opt-out have varied between:

The former entail having each potential recipient contact each potential source of UBE, to request that they not send UBE to them. The latter has the potential recipient contacting a (presumably small) set of independent services which compile "opt-out" lists and furnish the lists to senders of UBE.

Both proposals have been shown not to work. Unscrupulous UBE originators have not only not honored individual requests from users, they have sold lists of the people who responded to other UBE originators as "people who at least read their mail". Other unscrupulous UBE originators who say that they participate in opt-out clearinghouses still send mail to recipients who have registered with the named clearinghouses to not receive UBE. Because these kinds of clearinghouses are voluntary, there is little incentive for a UBE sender to use them.

The requirement that a recipient should have to respond to anyone who sends them UBE does not scale well with the large number of potential originators. Each month, hundreds of different originators send UBE, and that number could easily become tens of thousands in a short period of time. Forcing Internet users to reply at least once to that many mailings is unworkable.

Similarly, forcing users to determine which opt-out lists that they should register with is also not feasible. For each originator of UBE that the recipient didn't want to receive mail from, the recipient would have to determine which opt-out lists the originator might use and register with them. As in postal direct marketing, UBE opt-out lists would require Internet users to register their email addresses with list-maintenance companies who make money by reselling that information. Current indications are that this seems unlikely to be popular with anyone except marketers.

9. Accountability

In the early days of the Internet, many people knew each other, or knew at least one person at most Internet sites. This caused people to treat each other with a fair amount of respect, since they could most likely be held responsible for their actions. Of course, the Internet is not like that today.

One way to bring a bit more accountability back to Internet mail is to force either users or mail originating hosts to certify who they are. This gives a recipient a definitive method for finding the originator of a message, which is particularly important for eliminating UBE. Because it is easy to forge identities in plain Internet mail, certification must be done using digital signatures, such as with S/MIME and PGP/MIME.

In order for most originators to want to start using accountability methods, a significant number of receiving hosts or recipients must filter for messages that aren't accountable. This filter would flag messages that were accountable so that the recipient might trust them more, or possibly read them before messages that weren't accountable. Without this kind of pressure, there is little incentive for originators to add the overhead of accountability.

Accountability for Internet mail falls into three broad categories:

9.1. End-to-end authentication

Secure email can be used to certify that the person sending a particular message is indeed who they say they are, assuming that both parties agree on particular arbitrator of authentication. Each message can be tagged with a certificate that is digitally signed by a trusted third party. These certificates contain unforgeable credentials, so that the recipient can trust the identity of the originator as much as they trust the certifier. These certificate authorities (CAs) might be a bank, a national postal service, or a private entity.

This type of authentication only works in a system as large as the Internet if there is a small number of CAs. If there are more than a few, then the authentication portion of the message can become quite long, and the processing time for verification of the authenticity also gets longer.

Further, there must be some persistence in the authenticated information. If an originator is constantly sending UBE under a different name, then the goal of locating them and resolving complaints will not be served.

9.2. First-hop accountability

A modification to the concept of end-to-end authentication is to only authenticate the SMTP server that injected the message into the SMTP stream (at the "first hop"). This kind of signature can be done as the message is being processed. In order for this mechanism to work, all first-hop originators who become certified must agree to not accept mail from anyone they do not know. This system entails two contracts: between first-hop server owners and their customers, and between servers.

CAs would only issue certificates to sending hosts which commit to honoring first-hop accountability. Because this belief might change over time, the certificates would probably have shorter lifetimes than certificates for end users. If a significant number of the certified hosts cheat, the system would quickly fall apart.

In all likelihood, this approach will use a hierarchy of trust which begins with ISPs who commit to enforcing a set of rules for their customers and provide service only upon obtaining contractual commitment by those customers. Hence a violation of the terms and conditions of that agreement will provide a basis for legal action against the offender.

First-hop accountability also requires that the originating host have a definitive way of knowing that the person who sent the mail is who they say they are. There are many methods of doing this, including checking the login information from the dial-in server, using domain names that are controlled by the owner of the sending host, or requiring login to the SMTP server. This last type approach is not typically available, although mechanisms for SMTP "login" are being deployed.

If a recipient has the ability to distinguish services that enforce accountability, recipients can filter out sources which do not provide the enforcement. Services which can identify their users during message posting can hold them fully accountable by requiring that users sign an agreement which specifies particular rules concerning UBE. If the user violates those rules, they have violated a contract, and the usual legal recourse is then available.

9.3. Relay accountability

In order for first-hop accountability to work, recipients must be able to determine which first-hop hosts enforce generally-accepted accountability rules. For instance, if a recipient receives UBE through a host that says it performs first-hop authentication, and that recipient asks the host's manager for verification of the originator's address but never hears back from the host's manager, then trusting the host doesn't get the recipient anything. Thus, the first-hop accountability system will have to be some sort of verification that the relays themselves are accountable and therefore trustworthy. Because it is easy to forge this information in unauthenticated mail, the use of digital signatures is required for this to work reliably.

10. Conclusion

Growth of UBE in the Internet is having an extremely negative effect on email use. Originators have no incentive to limit their transmissions and recipients have inadequate tools for distinguishing legitimate email from UBE. They also have no meaningful legal recourse.

In this report we have described the various technique currently being discussed for the control of UBE. The taxonomy of choices described in the report includes:

Filtering
      Type
            Heuristic
                  Origin
                        IP host or network address
                        Domain name or domain suffix
                        Mail-From
                  Data strings (headers and content)
            Cooperative
                  Recipient registration
                  Originator labeling
      Location
            Source
                  Opt-out and opt-in lists
            Relay/Recipient SMTP server
                  Origin information and/or data strings
            Message store
                  Origin information and/or data strings
            Recipient user agent
                  Origin information and/or data strings
Law
      Regulation
            Ban on sending
            Ban on relaying
            Mandated content labeling
      Contracts
            Originator prohibition
            Relay associations
            Voluntary content labeling
Accountability of originator
      End-to-end authentication
      First-hop accountability

None of the proposed solutions to the problems associated with UBE will work in all environments. Many of the technical solutions will alleviate some of the problems today, but do not adapt well to smart, motivated, uncooperative UBE creators. Heuristic approaches to control of UBE have the serious side-effect of silently losing some legitimate Internet mail. This effect, even in the name of preventing UBE from getting through, is unacceptable to most recipients. Of course, it would be best if the Internet mail community could devise technical means for dealing with the issue instead of resorting to the often-problematic legal system. To date, neither type of solution is the clear-cut answer to the problems being faced by Internet users.


This IMC Report is IMCR-008 and is named UBE-SOL.
The Internet Mail Consortium is an industry trade association for companies participating in the Internet mail market.