Blacklists
Overview
Dealing with email spam is a difficult process. Sometimes the cure can be worse than the problem. Email blacklists provide an example.
Once in a while Stanford's email gateways get placed on an Internet spam blacklist. When this happens, email sent from Stanford mail servers gets bounced back to its sender, even when the message is perfectly legitimate (non-spam) email.
Why does this happen? Can Stanford do anything about it? This document explains the problem and discusses some possible solutions. In this document we'll use the blacklist maintained by SpamCop.net as an example, but many of the principles can be applied to any spam blacklist service.
Stanford and Email Blacklists
- What is an email blacklist?
- When spammers send unsolicited bulk mail, they rarely send it from their own machine: they hack into someone else's machine, usually from afar, and send all their spam from there. Anti-spam services, such as SpamCop, gather reports about spam coming from certain machines and publish the names of those machines on a "blacklist." People who run email systems configure their systems to filter, reject, drop, or tag any email that comes from a computer that has been blacklisted. By doing so, they hope to keep their own systems free of spam.
- Why would Stanford's email gateways (such as smtp.stanford.edu) get placed on a spam blacklist?
- Many people forward their @stanford.edu email to an external ISP, such as AOL. When these people receive forwarded spam at their AOL account they report it to SpamCop. SpamCop looks at the email header and notes where the message came from. Since the email was forwarded from Stanford, SpamCop assumes Stanford's mail gateway server is being used to send out spam. SpamCop therefore places the IP address of this server on its blacklist.
- Why does blacklisting cause my email to bounce back to me?
- If one of Stanford's email servers has been blacklisted, and you send a message to someone whose email system uses these blacklists to block mail (instead of filtering it), your email gets bounced back to you without being delivered. This can happen to mail forwarded from departmental email servers too, since most departmental servers route their outgoing email through Stanford's mail gateways. Most of the time, blacklist entries expire in a few hours, so people never notice. Other times Stanford's smtp servers can be blacklisted by SpamCop for several days in a row.
- Why is rejecting mail that comes from a blacklisted machine bad?
- Blacklisting causes problems when the administrator of an email system decides
to simply block all messages coming from a machine on a blacklist. Blacklists
are not intended to be used this way. Competent email service providers usually
filter spam, employing methods that consider multiple factors before deciding
whether an email message is spam or not. Most email administrators don't reject
mail just because an email server is on a blacklist, but will tag or quarantine
suspicious messages and provide the intended recipient with ways to view the
message safely. Even SpamCop advocates this method of using its blacklists.
Still, there are overzealous service providers who simply reject messages based on blacklists. This is bad because it prevents people from getting perfectly good email. It happens because many service providers are lazy. Using blacklists alone against spam is basically a cost-shifting exercise: instead of spending the necessary time and money to configure an email system correctly, the service provider pushes all the work for dealing with spam onto the administrators of remote email systems like Stanford's. - Can't you request that a Stanford IP address be removed from a blacklist?
- If only it were that easy. If you go to the SpamCop's page, for example,
you will see that the only option available to us (or any other system administrator),
is to promise on the page that our servers will never be guilty of producing
any further spam reports ever. This is, of course, nonsense. We cannot promise
this. No system can.
Reports received by SpamCop are usually reports from people with Stanford accounts who forward mail from their accounts to some third-party ISP; when they receive spam dutifully forwarded from their Stanford account, these people report it to SpamCop. Most blacklisting services are computer-based: their systems are not smart enough to understand that Stanford is not the source of the spam, that Stanford's systems are just dutifully forwarding email. The SpamCop blacklist is run nearly without human intervention, and is based on reports from untrained users: email systems can be blacklisted for all sorts of innocuous reasons.
Accordingly, SpamCop will continue to receive complaints about Stanford being a spammer, and will blacklist Stanford. Servers configured to use blacklists as a means of blocking mail will therefore occasionally block mail from Stanford. They will block mail from a lot of different sites too. - How can I deal with this blacklist problem?
- You can always attempt to contact the administrators of the system
to which you were trying to send email. Usually, however, it's better
to contact the actual person to whom you were trying to send mail (through
other means, such as telephone) and urge this person to contact their
system administrator. This works better because system administrators
listen to their own users better than they listen to strangers. In either
case, make sure the system administrators understand that your legitimate
email was rejected by their system. Ask them not to use SpamCop's blacklist,
or any blacklist, as the only factor involved when rejecting email. Alternately,
you can ask them to whitelist any email coming from a Stanford email
server. Towards this end, you may also want to direct that administrator
to:
which explains the SpamCop blacklist and includes, among other things, the following note:
The SCBL aims to stop most spam while not blocking wanted e-mail. This is a
difficult task. It is not possible for any blocking tool to avoid blocking wanted
mail entirely. Given the power of the SCBL, SpamCop encourages use of the
SCBL in concert with an actively maintained whitelist of wanted e-mail senders.
SpamCop encourages SCBL users to tag and divert e-mail, rather than block it
outright. Most SCBL users consider the amount of unwanted e-mail successfully
filtered to make the risks and additional efforts worthwhile.
The SCBL is aggressive and often errs on the side of blocking mail.When
implementing the SCBL, provide users with the information about how the SCBL
and your mail system filter their e-mail. Ideally, they should have a choice of
filtering options. Many mail servers operate with blacklists in a "tag only" mode,
which is preferable in many situations.
