Blacklists, Blocklists, DNSBL's, and survival:

Blacklists, Blocklists, DNSBL's, and survival:

How to survive as a non-combatant emailer in the Spam Wars.
A collection of frequently asked and too-often poorly answered questions.

Self-justification and other metadata for this document

Who maintains this document?

William K. Cole. My friends (and even enemies...) call me Bill.

Is there some sort of version and authoritative location for this document?

This is version 1.24 of 2007/01/16, and it was being actively maintained as of that date. The current version is available at http://www.scconsult.com/bill/dnsblhelp.html

Is there someplace to send suggestions for this document?

Yes. You can <http://www.honet.com/Nadine/>. In the end, failing to confirm addresses and permission, and particularly anything like an affiliate program that creates a market in addresses without policing the confirmation of those addresses both for validity and permission causes people to send spam, even if they do not intend to, and eliminates any chance of data integrity in a mailing list. Spam is not defined by what is in the message, but by whether the sender has a credible reason to believe that he has permission to send the mail. Lack of confirmation means there is no such credibility.

Why didn't anyone complain before?

Complaint rates for essentially randomly addressed (i.e. addresses scraped from the web or Usenet or from one of the bogus 'millions' CD's spammers sell) average around one in ten thousand. That means that if you just send to 5,000 addresses at random with no pretense of 'opt in' at all, you stand an even chance of getting ONE complaint. At 10,000 random addresses, you will probably (but not surely) manage to hit someone who will complain. Now imagine what happens if you try to do the right thing and run unconfirmed subscriptions systems or maybe even pay affiliates for addresses that may or may not really be the result of willing subscriptions, and you send mail to them. If 90% of your addresses are valid (a reasonable number, if you don't have unpoliced affiliates) and you have a million addresses (not likely, unless you're a Big Name) then 100,000 addresses are in some way Not Right, probably because of mis-entry. 90% of those will simply bounce, because they are not valid addresses at all and most sites have the good sense to cause hard bounces for invalid addresses. That leaves you with 10,000 addresses that are no good, but look good as far as delivery is concerned. Half of those will end up being dropped without bounces, because a few sites have taken up the (totally wrong and lazy) practice of accepting everything offered them and silently dropping the undeliverables. So there are 5,000 pieces of mail landing in mailboxes where they should not be. Spam, pure and simple. There's an even chance that none of those people will complain to you or your upstream, particularly if you are not selling something that looks like a scam or marketing in a dialect that makes it look like a scam, or marketing a product that some people find offensive.

Note that in that situation, the sender to a million-address list most certainly is a spammer. The spam delivery rate is less than a percent of their total list, and it may not generate a single complaint, but there remain 5,000 pieces of spam delivered. If you run a list carelessly (i.e. without confirmation) then you are surely sending spam and unless you have an extremely large list you are likely to not see complaints before you get blacklisted by someone somewhere.

How to get complaints BEFORE being blacklisted, or:
How to control spam without blacklists getting involved

This is a tough one, but it's the best goal to set. If you are sending bulk mail (see above for the definition and note that volume isn't part of it) then someday someone will consider your mail spam and someday you will send a piece of spam (maybe those will be the same someday, but don't count on it.) The best way to assure that you find out about this before the blacklists do is to avoid looking like a 'real spammer' even to people who think (maybe correctly) that you've sent them spam. Complaints are not sent to people who look like spammers. Those complaints get sent to blacklists or upstream ISP's instead. Things you can do to look unlike a spammer and reduce the actual risk of sending spam include:

A common mistake of people sending legitimate bulk mail is trying to evade or firmly rebut complaints. Both are generally bad strategy. You want few complaints, but you want them all coming to you, not SpamCop or your ISP or NANAS. You should be able to answer every complaint with a simple and complete explanation of why you thought your mail was legitimate and a simple and true commitment to send no more mail to that address effective immediately. Aside from the evidence of subscription, a response to a complaint should have no more than a half-dozen lines of content. Don't get into extended arguments with complainants about whether they subscribed and don't ever leave complainants subscribed in the silly hope that they will recall having subscribed and decide that they really want to stay subscribed. The best you can hope for with someone who forgets having subscribed is to jog their memory and have them go away feeling silly but definitely GO AWAY.

How to Escape the War Zone

The decade-long war between anti-spam activists, spammers, and everyone in between them has left everyone involved tired and bloodied. Unless you are absolutely committed to sending spam, there are things you can do to stay out of the line of fire, if not completely escape the War Zone.

Pick Your Vendors Carefully

When getting basic network service, figure out who really owns the network space you will be getting. All IP addresses are handed out by a handful of Regional Internet Registries (RIR's) including RIPE, LACNIC, ARIN, and APNIC. The RIR's allocate space to ISP's in /19 and larger CIDR blocks (approximately 8,000 contiguous IP addresses). Those ISP's allocate out of that space to customers who may then further sub-allocate to their customers. In the end however, anyone in the chain between you and the RIR can yank 'your' numbers and trade them for some spamming customer's old blacklisted numbers at any time. This is why many mail admins and many DNSBL maintainers have stopped paying attention to anything but the top-level RIR allocation when determining how wide to make listings. If you are downstream of a provider with a chronic pattern of tolerating spammers, you stand a good chance of getting listed at some point. Be aware that nearly all of the largest providers have had periods of very poor policy enforcement and using one of them is not a protection but in fact may end up being a risk in itself. Very often a smaller provider with their own direct RIR allocation and a squeaky-clean record will be quite able to provide you top-notch service affordably without all the baggage of the often fiscally shaky, ethically unstable 'Big Boys' who manage to get large swaths of their space listed regularly. In the most objective sense this is a simple matter: if you are in the middle of some /11 allocation (2 million addresses) then some spammer who is not even a direct customer of your direct provider and is a half-million addresses away from you could be the trigger for a listing that includes you, but if your provider only has a /19 allocated directly from a RIR, there are at most 8,000 of your fellow users of that provider whose bad behavior might result in you being listed. Whether you select a 'Big Name' provider or a smaller one, it is important to research their record of and reputation for rigorous policy enforcement.

If you absolutely must use an e-mail service provider, choose VERY carefully. There is no provider of bulk mail services that I am aware of (as of 2007-01-16) which has never had its sending addresses blacklisted in a significant way. Many of these companies operate quite consciously as spammers but lie about it to their legitimate customers, sending mail to lists they know or should know are dirty from the same machines that they mail to 'clean' customer-owned lists from. This means that even if you choose a company that claims to only send to 'opt in' lists and are very careful about managing your own lists ethically, you are likely in fact to end up with your mail coming from the same places as the zillions of ads for financial and medical snake oil that we all hate. It should go without saying that buying or renting a list is at least as risky if not more so: most email marketing lists available for hire are NOT truly opt-in and will bring you only trouble. There are a relatively tiny number of exceptions, but if a list vendor gets grumpy about your questioning him about collection methods, you should be very suspicious about his lists.

The chances are very good that if you are large enough to have your own full-time Internet connection and your own fill-time IT staff and your own mail server, you have adequate expertise and facilities to do your mailing on your own and not be concerned having to select a good bulk email provider from a field that is vastly overpopulated with scam artists and incompetents. Either way, you should do your homework and either stick to ethical list management practices (the most widely accepted standards are those collected by MAPS) or demand that your provider do so. The core principles behind good mailing list managemnent are:

  1. Mailings should be fully consensual
  2. No one should ever have to unsubscribe from mailings to which they did not knowingly subscribe.
  3. List owners should always know for sure whether an address owner actually wishes to be subscribed or not.

Abandon Bulk Email

This sounds radical and may not be suitable for you quite yet, but you should at least give it thought. There are alternatives. The best-looking one right now is RSS, a set of XML dialects designed to provide 'syndication' of URL-referenced materials. This is the way many people are keeping up with multiple 'weblogs' (aka 'blogs') with irregular publication schedules without having to run through a long list of websites daily or hourly to see who has made a new entry. Moveable Type, LiveJournal, Slashcode, and other 'blog' software supports RSS, and a growing number of major news organizations are now providing RSS feeds. One advantage of RSS over bulk email is that you don't need to manage a list of users and records of how they subscribed. Don't get me wrong: there are still issues with RSS, particularly in the areas of client penetration and the ability to track and customize for specific individual users, but there is reason to believe that these failings are short-term (i.e. there WILL be more and better clients for RSS next month and even more and better ones next year) and if you don't need to isolate individual customers and their behavior you are a very good candidate for RSS. RSS may represent the future of Internet publishing. It may not be where you really want it to be today to replace bulk email, but it is moving forward fast and refugees from the spam wars are playing an increasing role in its development.