Blacklists, Blocklists, DNSBL's, and survival:
How to survive as a non-combatant emailer in the Spam Wars.
A collection of frequently asked and too-often poorly answered questions.
William K. Cole. My friends (and even enemies...) call me Bill.
This is version 1.24 of 2007/01/16, and it was being actively maintained as of that date. The current version is available at https://www.scconsult.com/bill/dnsblhelp.html
Yes. You can email me. Really. If you use a mailer that does not understand the way the address in that link is structured, switch to a standards-compliant mailer or decipher it manually with the clue that parentheses are specified as comment delimiters and comments can go between any parts of an email address. Note that older mailers from Microsoft reportedly do not understand this flavor of standard mail address syntax. I have some thoughts on that.
However, if you send anything to that address which is not clearly related to this document specifically (i.e. if you spam it) I will respond in an unfriendly manner and will treat your spam as a request for me to exert my strongest legitimate efforts to have your access to the net removed.
I have spent over a decade fighting spam as a sysadmin, consultant, and activist. I worked for the Mail Abuse Prevention System (MAPS) where the DNSBL technique was invented and its public use pioneered (starting with the RBL) in 2000-2001, as a consultant and as Director of Customer Operations. I have been involved in the design and operation of business and personal email systems since 1993 on both the technical and policy sides.
Perhaps most importantly, I have been an anti-spam activist. That may be a frightening statement, but it does not mean what you probably think. I am not a 'hacker' in the sense commonly used by the media of someone who tries to break in or otherwise disrupt other people's computer systems. I am not a hater of commerce. I do not despise the commercialization of the Internet. I am an IT professional with long experience using the net for legitimate business who wants to preserve its utility for legitimate business, and I am fortunate to be familiar with a diverse community of people who share my abhorrence of the practices commonly referred to as 'spam.' I'm familiar with the people involved with fighting spam and with the people affected both by spam and by the tactics used to fight spam.
Because the people running many of the blacklists have done a poor job of addressing the real questions of people who use addresses on those lists but who don't have any conscious connection to the reasons for the listing. In the phraseology of the corporate IT world: there is no adequate documentation for a significant customer community.
The jargon of DNS-based lists (DNSBL's) can be bewildering. Let's be honest: most of them are in fact designed and used as 'blacklists' of IP addresses that the DNSBL maintainer believes fitting for that particular list. Terms of interest include:
The short answer is: NO. Anyone can run a DNSBL if they control a DNS zone, and they need not register their list anywhere, so any list MAY be incomplete. There is however a pretty useful site called OpenRBL that can do lookups in many of the most well-known public DNSBL's, although in mid-2003 it adopted a JavaScript dependency that made it not work for some browsers. Later it revamped again, and as of 2007-01-16 it won't work in any browser I regularly use. I personally have switched to mostly using the multi-list query tool at Moensted. Another multi-list tester seems to exist at http://rbls.org/ but I have not used it much. A reasonably complete list of significant DNSBL's is maintained by Declude. There is also a rather extensive list in the Open Directory.
All sorts of people. ISP's large and small. Businesses. Individuals.
Just about any sort of person or organization that runs a mail server may be using a DNSBL with it. DNSBL support has become a mandatory feature for serious Internet mail server software. In addition, some individual users have figured out ways to use mail filters like procmail to apply DNSBL's to their mail after delivery (although this is unusual and rarely of much concern for anyone who is sending strictly legitimate email.
There is some variability, but most DNSBL use is by a mail server (a.k.a. 'Mail Transfer Agent' or MTA) to check the IP address of the computer which is offering it a piece of mail. If the sending machine's IP address is in a DNSBL which is checked by an MTA, that MTA will usually reject the mail with a permanent failure code (5** series response) in the SMTP transaction. Some MTA's use multiple DNSBL's, weighted scoring, and other techniques to decide whether to accept a piece of mail.
In the past, most mail rejections due to DNSBL listings were clear about their nature, because the DNSBL was itself cited in rejection messages and often the TXT record associated with the l;isting was used as well. This remains true sometimes, however the increasingly widespread use of compound filtering tools like SpamAssassin have made it increasingly common for mail rejection to be the result of multiple criteria which are tested outside of the MTA itself, and rejection messages in such cases tend to be very generic and uninformative. Some such tools even have support for making the MTA act as if it is accepting mail, but actually discarding or "quarantining" rather than delivering it. As a result, you may sometime see mail rejected with a generic message or simply vanish and fail to be delivered if your mail is being stopped as a result of the use of a DNSBL.
There are actually many reasons, but most sites that use DNSBL's do so to reduce the amount of spam that they handle. There are some sites where DNSBL usage is really more of a political tactic to exert pressure directly or indirectly on the people who are perceived to be harboring spammers or others who misuse the net. Some sites mix the two goals. Some DNSBL's are explicitly not about spamming at all, but about other characteristics or behaviors of the owners of the IP addresses. Unfortunately, some sites use such lists with the belief that they have a direct relationship to spam.
There can be no simple answer to that question. At the most theoretical level they are objectively bad because they are designed to reject mail. They reduce the functionality of email. However, that reduced function is exactly the intent of their users, as long as the list has what they expect it to list and they understand what the consequences of its use are. Most DNSBL's are targeted at spam. Some DNSBL's are useful for detecting when one is being offered spam, and some essentially never make a mistake that labels non-spam as spam. That's good. Some DNSBL's routinely list addresses that have never handled any spam but regularly handle legitimate email. That's bad, if it means that mail is rejected which no one really intends to be rejected. Yet some sites use such broad lists quite intentionally to break email selectively to make a point about spam. I personally think that is bad, but it is hard for me to scold people for breaking their own sites in a way they really want to break it. Many DNSBL's are such that they would be reasonable choices for some sites but not for others, or useful in a weighted scoring system but unsuited for simple absolute blocking use at a site that cares about delivering legitimate mail. Except for a few particular cases of DNSBL maintainers behaving in unethical ways (e.g. ORBS) it is not very useful to think of DNSBL's as uniformly good or to overemphasize their inherent conceptual bad: for nearly any mail server that accepts mail from the Internet there will be one or more DNSBL's that will reduce the amount of spam handled without significant 'false positive' mail rejection. For nearly any DNSBL, there is some mail server somewhere whose purpose and variety of mail received makes for a suitable match. DNSBL's have made spamming harder, and that is good. DNSBL's have increased the chances of legitimate mail being refused by mail servers that are perfectly able to deliver it, and that is bad.
Many DNSBL's exist. Many are quite extreme about listings, and as a result are not used by any mail server that exists to actually deliver mail. Many others are rather conservative and list only actual spam sources, as best the maintainers can determine, and so have broad use. Being listed on one of the extreme lists (e.g. Spambag, Selward Extreme BL, etc.) is probably not going to cause you any real trouble in the form of mail bouncing. Being listed in a broadly used conservative list (e.g. any of the MAPS lists, the SBL) is a serious problem even if you have not yet seen a bounce. If you are seeing bounces of mail coming from your machine, you have a problem that you need to address.
If your mail has bounced, the bounce message should cite a particular machine which sent an error code of some sort, probably a 3-digit number starting with 5. Mail error codes are always 3 digits, and 5xx codes indicate permanent failures. Typically a bounce due to a DNSBL listing will also include text naming a specific DNSBL. That DNSBL did not block your mail. The server that sent the 5xx code did. Someone runs that mail server and has chosen to use the DNSBL cited to decide which mail to accept and which mail to reject. The DNSBL operator did not force them to use the list. The DNSBL operator may not intend their list to be used to reject mail absolutely. The DNSBL operator doesn't have the ability to judge whether any particular mail server should be using any particular list.
In short: your mail was rejected as a result of a decision made by the owners and/or operators of the mail server that responded to your machine with a 5xx response code. They blocked your mail.
This may seem a redundant question, but it is not really. Deciding what mail an MTA should reject is a complex process. To really figure out why a bounce occurred, you have to decipher the bounce message. There are 2 machines involved in any affirmative bounce: the last machine to handle the message and a machine which rejected it. If the cause of a bounce is that your mail server is listed in a DNSBL, those 2 machines are almost sure to be your machine and the machine using the DNSBL respectively. if that does not seem to be the case, there are alternative scenarios that might explain the bounce and make you think that you are listed, when in fact the listing may not be of your machine. For example, some ISP's have found themselves in the embarrassing position of using a DNSBL on one of their machines and having another one end up on that DNSBL. That can mean that all of their mail routing through the listed machine ends up bouncing. Not a good thing, to be sure, but Not Your Problem.
If a bounce really looks like it was due to a DNSBL listing, make sure you confirm the listing. All DNSBL's I know of offer some means for anyone to check individual addresses. Even the commercial lists run by MAPS can be queried through their website. Don't take the word of some mail server you don't run about a listing, confirm it. That will also give you a chance to look at why the listing exists.
In the end, your mail was rejected because of a decision by the people running the mail server that rejected your mail. If that decision was to use a particular DNSBL (probably so, or you wouldn't be reading this) then you MIGHT want to try to deal with both the specific site rejecting your mail AND with the DNSBL.
There are as many reasons for an IP address to land on a DNSBL as there are DNSBL's. Each list has its own criteria, and some are very highly subjective. Some examples:
If you want to understand your listing, start from an assumption that it is not a mistake. DNSBL's really do exist that intentionally list addresses which generate no spam at all. In fact, nearly all DNSBL's which are aimed at spammer-controlled addresses and not at abused resources like open relays and open proxies will intentionally escalate listings to include sources of legitimate mail. That is no mistake. That is not the same as accusing the holder of the escalated blocks of sending spam. It is a calculated choice. The rationales for such escalations include but may not be limited to the following, some of which may be quite dubious:
Please note: I do not endorse all of those rationales. They are merely my rewording of what I've heard from people running DNSBL's and from people who claim to know the mind of the people who ran SPEWS, which was well-known for its 'collateral damage' escalations but whose operators do not discuss their DNSBL publicly beyond a rather poorly-written FAQ. Note that the DNSBL's like SPEWS which use criterion #6 do so openly, and their users (people running MTA's) presumably understand that they are using a list which includes addresses that are not known to send spam. A listing in SPEWS, SpamBag, or other list is not a mistake simply because the listed address has never sent spam. Some lists intentionally list some addresses that have not specifically sent spam, and the users of those lists presumably intend to reject mail from such addresses
There are 3 basic strategies, listed in order of ease of execution:
To execute #1 you need only find someone willing to handle your mail who is not on a listed address. Many companies exist which will provide such service for a fee. If you are a small operation sending small amounts of mail you may be able to find someone willing to do this for you for free. Often an ISP will provide this service for customers as a basic part of their service, but be aware that if your ISP has provided you with an IP address that is listed, their mail server may also be on a listed address.
To execute #2 you need to contact the operators of the mail servers that are bouncing your mail. This is probably going to require phone calls or physical letters, for reasons that should be obvious. In some cases, mail server operators will have 'whitehole' treatment of standard role accounts such as 'postmaster' but you should expect this to not be universally true. Note that any competent mail administrator using an external DNSBL on a mail server that is really intended to deliver desired mail to users will have some way to make exemptions from the DNSBL for specific sources. Note as well that there are a lot of incompetent mail admins in the world, and some of them use DNSBL's that are really poorly matched to their user community. You may have to engage in education of admins.
To execute #3 you have to look at the specific DNSBL that is causing your mail rejections. Some are very objectively managed: if a machine fits particular objective security criteria (i.e. open SMTP relaying) it is listed, otherwise it is not listed. Those lists are usually very easy to deal with: fix the security problem with your machine and have it retested, and the listing will be removed. Other lists (notably SPEWS and the MAPS RBL) are run subjectively and involve judgement calls by the operators. In the case of many of those you can contact the person making the judgement and get a definitive answer from them of what you need to do to have the listing removed. When it was actively maintained, SPEWS was a special case and needed special handling to get de-listed. Now it is a different sort of special case.
As far as anyone can tell as of 2007-01-16, SPEWS has not updated their lists or website at all since 2006-08-24. The lists remain accessible, and the SPEWS site remains functional. Some new blacklisting operations (UCEPROTECT, APEWS) seem to be trying to serve the same set of users as SPEWS, and APEWS seems to want to be a replica of SPEWS, so I will leave the old SPEWS analysis below...
This section is purely historical. It reflects my analysis of SPEWS when it was being maintained. Since late 2006, any hope of de-listing from SPEWS has been unrealistic.
Please note that I do not speak for SPEWS. The following is my analysis of SPEWS' behavior without the benefit of being able to query the people running SPEWS.
SPEWS is an acronym for Spam Prevention Early Warning System, and it is not technically a DNSBL. However, its data is commonly used as a DNSBL and it shares the feature of being a publicly available blacklist. SPEWS lists spam sources and address space owned by 'spam support operations' which is a class that appears to include any ISP which does not terminate spammers' connections swiftly enough. SPEWS seems to recognize that the only objective indication of who 'owns' an IP address is the top-level allocation by a Regional Internet Registry (RIR) such as ARIN, APNIC, or RIPE. In practice, this means that SPEWS will often list not only a spammer's single address, but also some address space around a spammer which is RIR-allocated to the same large provider. It does not seem from SPEWS' past behavior that the people maintaining the list see the inclusion of addresses that have never emitted spam and have never been used by anyone who has sent spam to be an error of any sort. One of the most common mistakes made by people trying to get their addresses de-listed by SPEWS is to assume that the listing is a mistake simply because the address is not used to spam. It is no mistake: SPEWS clearly intends to list such addresses. Many users of SPEWS have expressed their comfort with this practice, and their desire to reject mail from those addresses even though it is not spam. SPEWS is run by people who strive to remain anonymous. The domain registration points at Russia, but that is widely recognized as misleading.
SPEWS publishes 2 lists in forms readily adaptable to use in DNSBL's. The SPEWS lists are referred to 'Level 1' and 'Level 2' with the Level 1 list containing the core listings and Level 2 containing many more addresses as a sort of penumbra around the Level 1 listings. Others have at times used the SPEWS data to publish DNSBL's. Many sites use SPEWS directly from the SPEWS lists, and many use the 3rd-party DNSBL's based on the SPEWS data. Because of the larger set of non-spamming addresses in the Level 2 list, it is rarely used to block mail outright, but is used in weighted scoring systems (e.g. SpamAssassin.)
Getting your address off of the SPEWS list is very challenging. Unlike an open relay or proxy list, there are no objective criteria and no automated test system for removal. Unlike most other subjective criteria lists (e.g. SBL, MAPS RBL) there is no path for people with listed addresses to open any conversation with the person or persons maintaining the list to get a clear statement of why an address is listed or what would have to occur to get an address de-listed. Instead, SPEWS seems to monitor some public fora including:
SPEWS is believed to modify listings based on what is said in those fora. It seems possible that you can increase the chance of the SPEWS maintainers becoming aware of facts by posting them to one or more of those venues. From watching SPEWS since its inception, it seems to me that only 3 sorts of facts are really likely to be relevant to a de-listing decision at SPEWS:
In the case of some blacklistings, notably a listing in SPEWS, the only communication channel to the blacklist maintainer is via public spam discussion fora, primarily news.admin.net-abuse.email(a.k.a. nanae) and SPAM-L. The problem with this from the perspective of the person with listed address space is that both of those have more than their share of hecklers, fools, fanatics, trolls, script-kiddies, and fanboys. If you post to either about your listing, you can expect a lot of responses. Most are useless. Some are not. You can also expect responses in the form of probes of your system. If you are not familiar with the Usenet cultural flavor (and SPAM-L largely shares that...) you may find it hard to keep your cool in these places. Just remember: a discussion forum is not a unitary group. There are kooks and fools there but there are also some very sharp and sane people. Sometimes it is hard to tell the difference. Just remember that a lot of the people taking part in these discussions have been highly radicalized on the issue of spam and many are essentially impotent with regards to spam except that they have the power to write scathing things about anyone they perceive as connected to spammers in any way. For example, you will find people who have never run a mail system in their life and have no means of using SPEWS applauding a SPEWS listing that includes non-spammers. Just grow a thick skin.
A blacklisting may result in a chance to encounter examples of disturbingly abnormal psychology. In particular, a posting to a public forum to deal with a SPEWS listing is almost sure to result in some rabid email and probably some unauthorized security probing activity aimed at your systems. Assuming that the former doesn't actually include threats and the latter doesn't actually breach your security (both of which are suitable to report to the proper law enforcement agencies) you should not respond to either directly. You may find it helpful to block both from your systems, but even though they may be the result of announcing your situation in a public forum, there's no point in taking them as accusations to the same forum unless you have a smoking gun implicating a specific participant. Keep in mind that anyone can read a newsgroup and a lot of the worst kooks reading nanae never post. As a longtime nanae user, I know that many of my fellow users would benefit greatly from medication and counselling that they are not likely to ever get, and that's just the one who post. There is no gatekeeper for nanae and only a very permissive one for SPAM-L, so keep in mind that the worst kooks are probably disliked most by the other regulars who have no means other than a killfile of avoiding them.
One route that can help you deal with a blacklisting that is not a result of your own behavior is to take the issue to your ISP. In such cases it is generally their behavior that has caused the listing or that of their upstream provider. You should be sure to read your service agreement first and whatever terms of service/acceptable use policy (TOS/AUP) the provider has. You should also get comfortable with the nature of the Internet. The Internet is what the name suggests: a network of networks. It only works at all because of a default trust between network operators, such as your ISP. When you purchase connectivity, you are buying a slice of whatever trust the world has in your ISP. If your ISP has behaved in a manner that makes a significant number of other network operators (i.e. people running mail servers and running blacklists) lose trust in their ability to run a network that emits a minimal amount of spam or other abuse, they are selling you damaged goods. A blacklisted IP address is damaged goods, and unless your own behavior caused that damage, you can and should seek redress for the damage from the person who sold it to you. Maybe that redress is a new IP address, maybe it is compensation for having to route your mail otherwise, or maybe it is just their serious effort to get the blacklisting reversed. In nearly all cases where you are blacklisted through no fault of your own, it is an issue that only can be fully addressed by your provider.
Sometimes of course, a blacklisting is the result of spammers that you are actually involved with as customers, providers, or unwelcome abusers of your insecure systems. A spammer as your provider is a hard case: it is likely that their service contract is sufficiently weaselly that you can't just cut and run. Talk to your lawyer, and try to break it. A spammer as a customer should be easier: they need to be given the choice of losing service or ceasing spamming absolutely. If you have a well-written service agreement, you can do that, because a customer who makes other networks shun your IP space is damaging the most important value your network has. If you can toss a customer for persistently and intentionally overloading your systems, you should also be able to toss them out for spamming. You may not have specific spamming prohibitions in your service agreement, but if it is any good you will have some terms that allow you to turf a customer if they harm your network, and getting your space on a blacklist IS harming your network. If you want the blacklisting to end and your customer caused it, you need to have the guts to give them the choice between ending whatever behavior caused the listing or leaving your network.
Unwelcome hijackers are a different matter altogether. It used to be fairly common for mail servers to be "open relays" that allowed anyone to connect from anywhere and pass along mail for delivery to anywhere else. That norm has slowly died out over the past decade as a result of spammers. It has become clear that spammers will abuse any open mail relay that they can find and attack to the point where it is useless for its intended users. Leaving a mail server as an open relay is irresponsible in the modern world, and while you can try to chase down the spammers after the fact for punitive reasons, that is likely to be very difficult and unproductive, since there is always going to be another spammer ready to pound that open relay into submission. The other major mode (and now the most common) of spammers abusing security holes to send their spam through other peoples systems is the proliferation of unsecured proxy software. There are a wide variety of proxy systems, and some of them are installed unknowingly in packages primarily presented as doing other things, such as Apache. Having an open proxy and not knowing about it is pretty bad. Leaving it in place unsecured after you are made aware of it is nothing short of imbecilic. In the case of such proxies, there is usually no way to go after the actual spammers abusing it, since an unsecured installation usually is done for the sake of ease and administrative obliviousness, meaning that logs are being redirected to /dev/null.
The last way a spammer may be connected to you and causing your blacklisting is as a neighbor. Your best bet in this situation is to do nothing and have your common provider deal with the situation. Don't play neighborhood watch. At best, that would just annoy the spammer who would proceed to ignore you. In the worst case, it could lead to real nastiness from the spammer made more easy by your network proximity. Just don't do this.
It can be productive to address your blacklisting from the side of the blacklist users, instead of the blacklist maintainer. It is probably the best route to take if you are listed in SPEWS because of someone else's behavior, at least as an immediate measure while you try to get your provider to act. Keep in mind that for the most part, people use blacklists because they want to stop spam. That usually means that they care enough about mail to want to get legitimate mail. If you are sending legitimate mail, they should want it. If you can convince them that the mail that was bounced was not spam and that you do not send spam at all, you have a good chance of being whitelisted at that site.
Being blacklisted is stressful. It's easy to react badly to stress. Try not to do so like this...
This seems natural to many Americans, not so natural to Europeans. Or lawyers. Think about that for a second. Before you start screaming at anyone about how you're going to sue them or someone else, talk to a real lawyer. It is reasonably likely that a real lawyer would encourage you to not talk about any lawsuit, and particularly not one which has yet to be filed.
I am NOT a lawyer and you'd be a fool to treat this section as serious legal advice, but if you are really eager to sue over a blacklisting, I urge you to present these arguments against a suit to your lawyer and see if the two of you are still eager to file that suit against these targets:
Note what blacklist maintainers do: they publish lists. Every list is different, and most are documented as to what it means to be on a list, although a few oddballs intentionally do not document themselves. In the end, a blacklist is a form of speech, and in most places where blacklists are operated there are legal protections of speech, which are generally referred to in the US as "the right to free speech." Those protections are of course limited, but the limits in most relevant places (notably the US) are quite narrow. In general (and again I urge you to talk to a legal professional in your jurisdiction...) speaking unkindly (as in a blacklist) is immune from legal action if the negative speech is true. For Internet blacklists, that means that the list has to have discernable meanings for listings and your listing has to be false under those meanings. So, if SPEWS lists you as part of a large listing around a spammer, you have a hard case to make for falsehood: you'd have to show that your provider isn't harboring a spammer in that range. Similar arguments exist for nearly all of the blacklists. To have a successful suit against the blacklist for what they are saying to their users about your IP address, you need to be able to show that they are not telling the truth by their own definition of what a listing means. It may also be relevant to a defamation suit that a blacklist operator could make a credible case for IP addresses falling under the 'public figure' standards for defamation, which sets a somewhat higher bar than when defamation is against a private person.
Note that there have been lawsuits against blacklist maintainers. Multiple cases against MAPS by deep-pocket plaintiffs ended up being settled, some in ways that look good for the plaintiffs, some otherwise. In the end, none of the suits made it to trial, MAPS shrank by about 85% under legal bills that they described as '7 figures' and MAPS made its lists fee-based. ORBS was also sued in 2001, over listings that seemed quite clearly false and seemed to be motivated by the personal finances of the operator of ORBS, who fled from the suits rather than fighting them. The key to the ORBS case was that the listings were documented as being based on specific objective criteria, but the plaintiffs' listed machines did not meet those criteria. However, the plaintiffs were also attempting to collect delinquent debts from the operator of ORBS. The third well-publicized situation where a suit was filed over blacklists was the case created by the rather shadowy organization calling itself "EMarketers America" whose only public face was a rather eccentric lawyer named Mark Felstein, who claimed to represent bulk emailers whom he did not specify, suing people that he claimed to be involved with SPEWS and people who ran and assisted the SpamHaus Project, which provides very useful DNSBL's. This case never even made it into discovery and ended up being dismissed with prejudice at the request of Felstein before the defendants even had a chance to find out just what exactly he was suing over.
Overall a pretty seedy record to be adding your name to. Absent an ORBS-type situation of clear falsehood or an ability and willingness to drive an extremely expensive pre-trial process against a blacklist (which risks severe legal consequences in some places) and make your name dirt in the public eye, it seems unwise to try to sue a blacklist.
Another basis on which people have suggested suing blacklists is that the blacklist is blocking email. That has never gotten far for a simple reason: the blacklist is doing no such thing. All blocking is done on some server somewhere, and the blacklist doesn't control those servers. The blacklist is a tool used by the people controlling that server. They are the ones blocking mail.
They're blocking your email after all. Well, yes, they are. Making that actionable in court is tough in most places. The people running that mail server owe you no service. They deliver your mail to their users as a service to those users, NOT as a service to you. You are unlikely to have any standing to sue them for a failure in that service. It is also important to note that in many places (including the US) commercial ISP's have long been ruled NOT to be 'common carriers' and so are not subject to the rules that rail carriers, phone companies, and other common carriers are in offering service fairly to all. They are not required to be fair to you. In addition, there is explicit law in many places (including the US) that protects operators of mail servers or other online services from any liability for filtering that is done in good faith and inadvertently blocks the wrong material. In the US this is part of a law aimed at sexual content, but as written it seems applicable to any filtering aimed at anything 'objectionable' which can probably apply to filtering aimed at spam. In some US states (and maybe some other places) there are even specific laws protecting spam filtering mistakes. Of course, applicability of any of these laws to your situation is something to ask a lawyer about.
This begs for a comment on ethics, as discrete from law. The Internet is an odd place. People running mail servers have a tough task, because they are typically expected to protect their users from everything unwanted or dangerous coming in mail yet the key protocol, SMTP, was designed for an Internet without the modern risks and garbage flowing through email. Meanwhile they are expected to deliver every piece of wanted email swiftly and intact, despite the fact that they have zero control or knowledge of the sources of email. With spammers and virus authors working hard to evade filtering tactics and the key element (whether the mail is wanted) being beyond the ability of a computer to detect, there is no way to reach perfection in filtering that never shuns valid mail. This is a non-ideal situation. Life is full of such, and lawsuits are not a solution to most of them.
This is the most rational lawsuit target, but it is not a pragmatic choice. If you can't get them to fix your situation by providing an unlisted address or with handling of their spammer problem you could be in a position to sue them for failing to fulfill their service contract, but it is most likely that the contract is not really written that way. Most Internet service contracts have clauses that effectively remove any obligation by the ISP to meet any level of service that they can't directly control. On the other hand, there is a reasonable argument that they can control blacklistings to some extent by controlling spammers on their network, and that failing to do so is damaging the level of service that they can provide. Either way, you are probably best off ending your service with a poor provider, rather than suing them for performance of a contract. If they sue you over that, the argument that they were not providing needed service levels may be a winner. Again, talk to a lawyer.
If a blacklisting is causing real problems (i.e. bouncing email) and everybody ignores it, it definitely will not go away any time soon enough. Many blacklists practice 'escalation' that expands the listed space as inaction rules the situation. Providers perceived to be spammer havens gather more spammers and end up in more blacklists. This hits an event horizon of sorts when a provider becomes so dirty that large chunks of their space are in a large number of unpublished local blacklists that are generally badly maintained and can only be dealt with one at a time, for a very low yield relative to the effort. Don't let your situation get there.
This makes you a combatant in the Spam Wars. Most people with long histories online know where the endpoint of that war is if it is lost: a divided Internet. If you really want to enter a war where you will effectively be fighting for your own segregation into a ghetto shared with penis enlargers, horse molesters, pyramid scheme operators, and people masquerading as relatives of dead African despots, then I will not try to stop you. Just remember that the people on the side against spam are the people who are able to create that segregation when enough of us give up hope of ending spam for everyone. Some anti-spam activists have already headed that way and live almost spam-free at the cost of shunning a fair slice of legitimate mail.
This is tempting. It is a fight against free speech and the rights of mail server owners to make their own decisions for their own private property. It is a way to get yourself into a lot of little private blacklists forever.
I agree that many blacklists are run in careless ways and are used in ways that needlessly cause legitimate mail to bounce. People drive drunk and kill people too, but I'm not joining any crusade to ban cars or alcohol. If you see a particular problem where a particular blacklist is used on a particular server and causing legitimate mail to bounce, fight that particular situation. That's a winnable battle worth fighting.
I think I explained above that being listed in an anti-spam blacklist does not necessarily mean that anyone is calling you a spammer, but sometimes they really are. They may be right. You may be a spammer and not realize it. Really.
The term 'spam' has been used in an evolving way for most of the life of the Internet. It actually gained its greatest spread as a result of the infamous Usenet "Green Card Spam" in 1994, but for the purposes of this document I am using 'spam' to its email meaning, which is simply this: Unsolicited Bulk Email.
If you are sending bulk mail, you probably occasionally send a piece of spam. Really. That does not make you evil. You can't completely avoid it. You CAN reduce it. You SHOULD reduce it to a minimum. If you are accepting mailing list subscriptions without some sort of closed-loop confirmation of addresses, you are not doing enough to minimize your spam and you should not bother trying to claim that you are not spamming, particularly in the more radical public fora. There are understandable reasons why you might not be doing closed-loop confirmations, but you should not be bragging of them. There are understandable reasons, but there do not excuse that failure.
Let me explain a bit more. People forge addresses. People invent addresses and sell them as 'opt in lists' or give them to 'partners' who pay per address. People also mistype their own address. For example, there is or was someone in China with the local account 'binhuang' in a domain whose name is just like mine except for the addition of '.cn' on the end. This 'binhuang' apparently has erred a number of times in entering his or her address in web forms asking for mail. As a result, I bounce mail aimed at 'binhuang@scconsult.com' on a regular basis. There may be someone there with the local account 'bill' as well, which might explain why I get so much spam from China in Chinese. The point is that if the scores of people now mailing 'binhuang@scconsult.com' used good list management practices, and confirmed the validity of the address and the desire of the address owner to receive their mail before mailing to it, I'd have less spam aimed at my mail server. I'd also probably not be shunning most of China at the router level. To see an example of just how bad this sort of thing can get, see the story of Nadine at <http://www.honet.com/Nadine/>. In the end, failing to confirm addresses and permission, and particularly anything like an affiliate program that creates a market in addresses without policing the confirmation of those addresses both for validity and permission causes people to send spam, even if they do not intend to, and eliminates any chance of data integrity in a mailing list. Spam is not defined by what is in the message, but by whether the sender has a credible reason to believe that he has permission to send the mail. Lack of confirmation means there is no such credibility.
Complaint rates for essentially randomly addressed (i.e. addresses scraped from the web or Usenet or from one of the bogus 'millions' CD's spammers sell) average around one in ten thousand. That means that if you just send to 5,000 addresses at random with no pretense of 'opt in' at all, you stand an even chance of getting ONE complaint. At 10,000 random addresses, you will probably (but not surely) manage to hit someone who will complain. Now imagine what happens if you try to do the right thing and run unconfirmed subscriptions systems or maybe even pay affiliates for addresses that may or may not really be the result of willing subscriptions, and you send mail to them. If 90% of your addresses are valid (a reasonable number, if you don't have unpoliced affiliates) and you have a million addresses (not likely, unless you're a Big Name) then 100,000 addresses are in some way Not Right, probably because of mis-entry. 90% of those will simply bounce, because they are not valid addresses at all and most sites have the good sense to cause hard bounces for invalid addresses. That leaves you with 10,000 addresses that are no good, but look good as far as delivery is concerned. Half of those will end up being dropped without bounces, because a few sites have taken up the (totally wrong and lazy) practice of accepting everything offered them and silently dropping the undeliverables. So there are 5,000 pieces of mail landing in mailboxes where they should not be. Spam, pure and simple. There's an even chance that none of those people will complain to you or your upstream, particularly if you are not selling something that looks like a scam or marketing in a dialect that makes it look like a scam, or marketing a product that some people find offensive.
Note that in that situation, the sender to a million-address list most certainly is a spammer. The spam delivery rate is less than a percent of their total list, and it may not generate a single complaint, but there remain 5,000 pieces of spam delivered. If you run a list carelessly (i.e. without confirmation) then you are surely sending spam and unless you have an extremely large list you are likely to not see complaints before you get blacklisted by someone somewhere.
This is a tough one, but it's the best goal to set. If you are sending bulk mail (see above for the definition and note that volume isn't part of it) then someday someone will consider your mail spam and someday you will send a piece of spam (maybe those will be the same someday, but don't count on it.) The best way to assure that you find out about this before the blacklists do is to avoid looking like a 'real spammer' even to people who think (maybe correctly) that you've sent them spam. Complaints are not sent to people who look like spammers. Those complaints get sent to blacklists or upstream ISP's instead. Things you can do to look unlike a spammer and reduce the actual risk of sending spam include:
A common mistake of people sending legitimate bulk mail is trying to evade or firmly rebut complaints. Both are generally bad strategy. You want few complaints, but you want them all coming to you, not SpamCop or your ISP or NANAS. You should be able to answer every complaint with a simple and complete explanation of why you thought your mail was legitimate and a simple and true commitment to send no more mail to that address effective immediately. Aside from the evidence of subscription, a response to a complaint should have no more than a half-dozen lines of content. Don't get into extended arguments with complainants about whether they subscribed and don't ever leave complainants subscribed in the silly hope that they will recall having subscribed and decide that they really want to stay subscribed. The best you can hope for with someone who forgets having subscribed is to jog their memory and have them go away feeling silly but definitely GO AWAY.
The decade-long war between anti-spam activists, spammers, and everyone in between them has left everyone involved tired and bloodied. Unless you are absolutely committed to sending spam, there are things you can do to stay out of the line of fire, if not completely escape the War Zone.
When getting basic network service, figure out who really owns the network space you will be getting. All IP addresses are handed out by a handful of Regional Internet Registries (RIR's) including RIPE, LACNIC, ARIN, and APNIC. The RIR's allocate space to ISP's in /19 and larger CIDR blocks (approximately 8,000 contiguous IP addresses). Those ISP's allocate out of that space to customers who may then further sub-allocate to their customers. In the end however, anyone in the chain between you and the RIR can yank 'your' numbers and trade them for some spamming customer's old blacklisted numbers at any time. This is why many mail admins and many DNSBL maintainers have stopped paying attention to anything but the top-level RIR allocation when determining how wide to make listings. If you are downstream of a provider with a chronic pattern of tolerating spammers, you stand a good chance of getting listed at some point. Be aware that nearly all of the largest providers have had periods of very poor policy enforcement and using one of them is not a protection but in fact may end up being a risk in itself. Very often a smaller provider with their own direct RIR allocation and a squeaky-clean record will be quite able to provide you top-notch service affordably without all the baggage of the often fiscally shaky, ethically unstable 'Big Boys' who manage to get large swaths of their space listed regularly. In the most objective sense this is a simple matter: if you are in the middle of some /11 allocation (2 million addresses) then some spammer who is not even a direct customer of your direct provider and is a half-million addresses away from you could be the trigger for a listing that includes you, but if your provider only has a /19 allocated directly from a RIR, there are at most 8,000 of your fellow users of that provider whose bad behavior might result in you being listed. Whether you select a 'Big Name' provider or a smaller one, it is important to research their record of and reputation for rigorous policy enforcement.
If you absolutely must use an e-mail service provider, choose VERY carefully. There is no provider of bulk mail services that I am aware of (as of 2007-01-16) which has never had its sending addresses blacklisted in a significant way. Many of these companies operate quite consciously as spammers but lie about it to their legitimate customers, sending mail to lists they know or should know are dirty from the same machines that they mail to 'clean' customer-owned lists from. This means that even if you choose a company that claims to only send to 'opt in' lists and are very careful about managing your own lists ethically, you are likely in fact to end up with your mail coming from the same places as the zillions of ads for financial and medical snake oil that we all hate. It should go without saying that buying or renting a list is at least as risky if not more so: most email marketing lists available for hire are NOT truly opt-in and will bring you only trouble. There are a relatively tiny number of exceptions, but if a list vendor gets grumpy about your questioning him about collection methods, you should be very suspicious about his lists.
The chances are very good that if you are large enough to have your own full-time Internet connection and your own fill-time IT staff and your own mail server, you have adequate expertise and facilities to do your mailing on your own and not be concerned having to select a good bulk email provider from a field that is vastly overpopulated with scam artists and incompetents. Either way, you should do your homework and either stick to ethical list management practices (the most widely accepted standards are those collected by MAPS) or demand that your provider do so. The core principles behind good mailing list managemnent are:
This sounds radical and may not be suitable for you quite yet, but you should at least give it thought. There are alternatives. The best-looking one right now is RSS, a set of XML dialects designed to provide 'syndication' of URL-referenced materials. This is the way many people are keeping up with multiple 'weblogs' (aka 'blogs') with irregular publication schedules without having to run through a long list of websites daily or hourly to see who has made a new entry. Moveable Type, LiveJournal, Slashcode, and other 'blog' software supports RSS, and a growing number of major news organizations are now providing RSS feeds. One advantage of RSS over bulk email is that you don't need to manage a list of users and records of how they subscribed. Don't get me wrong: there are still issues with RSS, particularly in the areas of client penetration and the ability to track and customize for specific individual users, but there is reason to believe that these failings are short-term (i.e. there WILL be more and better clients for RSS next month and even more and better ones next year) and if you don't need to isolate individual customers and their behavior you are a very good candidate for RSS. RSS may represent the future of Internet publishing. It may not be where you really want it to be today to replace bulk email, but it is moving forward fast and refugees from the spam wars are playing an increasing role in its development.