Spam: put a lid on it

• Current reactions to spam are out of proportion to the problem.




• Anti-spam legislation and litigation are more damaging to internet communication than spam is.




• Most anti-spam technology is more damaging to internet communication than spam is.




• Anti-spam technology can only be effective if it allows for individual choice.

Unsolicited bulk email, or spam, seems to be the major issue facing the internet today. It provokes heated debate in government, business and technology circles, and among the general public.

Two recent pieces of research conclude that half of all emails sent today are spam. But the problem isn’t just restricted to ordinary email – it extends to newer technologies too, such as interactive TV and text and photo messaging over mobile phones. It has been reported that more than two-thirds of mobile phone users have received text message spam (1).

The spam problem has given rise to some thorny legal issues. There are courses for law students that are dedicated to the legal issues surrounding spam. The legal theorist Lawrence Lessig has even wagered his prestigious job on the spam question, pledging that if the anti-spam legislation he has proposed is introduced and proves ineffective, he will resign his professorship at Stanford University (2).

In the political world, the spam problem unites concerned politicians of all persuasions. In the USA, Democrats have allied themselves with conservative religious groups to combat spam. In the UK, the House of Commons All Party Parliamentary Internet Group is holding a high-profile ‘spam summit’ in July (3).

But perhaps those most concerned about spam are technologists, who encounter more spam than the rest of us do, and who have to deal with the consequences of spam for the systems they work with. Technologist views of the spam problem often verge on the apocalyptic. According to Steve Linford, head of the anti-spam project Spamhaus, internet communication may be only six months away from complete meltdown as a result of spam (4).

• Current reactions to spam are out of proportion to the problem

Given the hysterical tone of the reaction to spam, it is easy to forget that spam is a practical problem. Yet it tends to be characterised as something akin to physical abuse. Yes, it can be a pain having your inbox crammed with Nigerian financial scams and offers for viagra and penis enlargement – but it’s an exaggeration to claim that the faceless hucksters who send you this rubbish are invading your privacy or prying into the intimate details of your life.

It is also easily forgotten that for most of us, the ‘delete’ button on our keyboards is a fairly effective means of dealing with spam. Even internet guru Mark Hurst, who currently receives about 150 spam emails a day (and expects to receive more than that in future), argues bluntly that ‘spam is not a problem’ (5).

Having said that, it’s true that the increasing amount of spam can impose a terrible burden upon technical systems, and waste people’s valuable time. So it is entirely legitimate to consider what the solution to this problem might be. But in order to come up with a rational solution, we have to keep in mind that the problem of spam is a practical one first and foremost.

Given that spam is a practical problem, it was hardly useful when the internet security firm Symantec recently attempted to turn spam into a moral issue, by talking up its impact on children. A survey conducted by the firm found that ‘more than 80 percent of children surveyed who use email receive inappropriate spam on a daily basis’, and ‘half of the kids surveyed reported feeling uncomfortable and offended when seeing improper email content’ (6).

The problem of children encountering explicit content is not particular to email spam, but is part of a broader necessity for parents to monitor their children’s internet use, if they want to be certain of what their children are looking at. With older media, TV watersheds and newsagent top shelves provided some reassurance, that explicit material could be kept out of the hands of kids. With the internet, there are no such technicalities in place to reassure parents.

This doesn’t mean that the internet is a malignant force that corrupts and traumatises children, but rather serves as a reminder that parents are responsible for bringing up their children as they see fit. Besides, children will feel ‘uncomfortable and offended’ about many things they encounter while growing up. This does not mean they will be scarred for life, and short of keeping your progeny under lock and key, you can’t guarantee that they will never see anything you would rather they didn’t.

The tendency to characterise spam as a moral problem rather than a practical one is a recurring theme in attempts to use legislation and litigation to fight spam, and is one reason why such attempts are misguided. While it is tempting to run to the law when spam irritates you, in truth, using the law as a weapon against spam may well do more harm than good.

• Anti-spam legislation is damaging to internet communication

The kinds of laws that are being drafted to deal with spam tend to rely on universal definitions of the terms ‘unsolicited’, ‘bulk’ and ‘commercial’ to specify what it is that they are outlawing. But spam does not lend itself to the kind of universal definitions required by law; it is a problem that we all experience differently.

Depending on your profession, your personal disposition and the way you use your email, spam can be anything from a minor inconvenience to a crippling burden. And there’s no reason why the same email can’t be irritating spam to one person and interesting and legitimate to another person. One person’s spam is another person’s steak, and vice versa.

It is easy to be glib about this point, because, like pornography, spam is something that most of us think we recognise when we see it. Most of us are also convinced that the vast majority of spam is worthless – which is almost certainly true, but this doesn’t make spam any easier to define. Drawing the line as to what is and isn’t spam is inevitably a subjective judgement made in individual circumstances, and is therefore resistant to objectification in law.

Attempting to draw a distinction between the illegitimate suppression of free speech and the legitimate suppression of spam, Lawrence Lessig sneers: ‘Go through the spam in your inbox and ask yourself whether there’s any ambiguity about the spam you receive. Maybe it’s just me, but my inbox is not filled with unsolicited political speech.’ (7) This argument is not helpful – it assumes that the extent of the problem being tackled justifies the scope of the law used to tackle it. But legislation should be assessed according to all of its potential consequences, not just the positive and practical.

The internet has provided us with unprecedented opportunities to encounter new individuals, ideas and experiences. If we use sledgehammer legislation to foreclose what we might encounter via email – on the grounds that most of the unexpected things we encounter via email are worthless – then we might be denying ourselves a future opportunity to encounter something worthwhile or interesting. Calls for anti-spam legislation almost always neglect the importance of the internet bringing us into contact with the unexpected, and focus instead on the (exaggerated) inconvenience that spam causes us.

At least eight anti-spam bills have been proposed in the US Congress in 2003 alone, and it is likely that at least one of these will be accepted. These bills inevitably involve stringent forms of internet regulation – for example, calling for the Federal Trade Commission to have greater internet enforcement powers (8). At a time when the internet is already overregulated, this is the last thing we need.

The international raft of high-profile lawsuits recently brought by Microsoft, AOL, Earthlink, Yahoo! and others, seeking to make an example of spammers in court, will also do more harm than good (9). The consequence could well be that a growing number of people hesitate before using the internet to promote new ideas or products, for fear of litigation – while those with the financial clout or technical know-how to continue spamming regardless of litigation will continue doing so.

It is not just in the USA that the law is being used as a weapon against spam. Anti-spam legislation is being proposed and implemented around the world. Labour MP Paul Flynn has called for a ban on spam in the UK. The Organisation for Economic Cooperation and Development recently issued a set of government guidelines for tackling cross-border fraud, calling for strong anti-spam regulation at the national level (10).

Perhaps the most wrongheaded piece of imminent anti-spam legislation is the Directive on Privacy and Electronic Communications, which is due to be implemented throughout the European Union in October. This Directive outlaws ‘unsolicited communications for purposes of direct marketing…without the consent of the subscribers concerned’ (11). The Directive employs the definition of ‘consent’ laid out in the earlier Data Protection Directive, where a person’s ‘consent’ means ‘any freely given specific and informed indication of his wishes’ (12).

This means that, taken together, the Data Protection Directive and the Directive on Privacy and Electronic Communications, interpreted strictly, will make it illegal to send anyone an email that they weren’t expecting, if that email could be construed as being in any way commercial.

This is a ridiculous restriction to place upon internet communication, threatening to make us all tread cautiously before daring to communicate online. Far from saving the internet, as it is supposedly intended to do, such legislation can only serve to stifle it. Such blunt legislation will provide people and organisations with the means to bring spurious lawsuits against those they have a reason to dislike (13).

At a conference on freedom of the media and the internet held in Amsterdam in June 2003, I had the opportunity to confront Rogier Holla, a principal administrator with the European Commission (EC), about the potential negative consequences of the Directive on Privacy and Electronic Communications (14). His response? He told me that ‘the soup is not eaten as hot as it is served’ – meaning that you shouldn’t assume that strict law will be interpreted strictly.

This amounts to the EC making excuses for bad law simply by saying ‘trust us’. I wouldn’t give them an inch. But if the consequences of fighting spam with law are bad, what are the consequences of the alternative – fighting spam with technology?

• Anti-spam technology is damaging to internet communication

Most technological solutions to spam suffer from the ‘whack the mole’ problem – the more moles you whack, the more pop up out of the ground. So technological solutions have had to become increasingly drastic. Microsoft, for instance, is going to great lengths to incorporate methods of suppressing spam into its upcoming releases of the next-generation MSN, Microsoft Exchange, and the Outlook messaging and collaboration client in Office 2003 (15).

Technical methods of suppressing spam range from address-book-based systems that redirect mail from unknown senders, to image-blocking software, to collaborative reporting tools that allow users to report spam to others when they receive it. By and large, technological solutions to spam tend to make overly presumptuous decisions on behalf of users, resulting in ‘false positives’ – the problem of desired email being incorrectly categorised as spam, and so never reaching its intended recipient.

With two of the more popular technological methods of combating spam today – Realtime Blackhole Lists/Relay Blocking Lists (RBLs), and ‘challenge-response’ systems – the problem of false positives is endemic. But despite growing criticism of these methods, it is still wrongly assumed that the benefits of not receiving spam outweigh the ills of failing to receive some legitimate email.

Broadly speaking, RBLs are lists that administrators of email systems subscribe to in order to help block spam (16). They define spam not by scrutinising the content of email, but by scrutinising the servers that email passes through. Technologist Philip Jacob argues that RBLs are unscalable, that they will inevitably produce an increasing number of false positives, and that they discriminate against legitimate email sent from Asia and South America – where a disproportionate amount of spam originates from.

As with the legal methods used to fight spam, the problem that Jacob identifies with RBLs is that they rely upon untenable universal definitions – in this case, definitions which hold that email passing through a particular server must be spam. Jacob also criticises the unaccountability of the organisations that create RBLs. He argues that whereas the standard definition of an RBL is ‘a list of servers which send out spam or are known to be open relays’, in fact a more accurate definition would be ‘a system for arbitrarily rejecting email messages (spam or otherwise) based on an unknown entity’s unknown criteria’ (17).

Such arbitrariness has caused big problems for some. Organisations and institutions from BTopenworld to Oxford University have discovered that their users cannot send legitimate email, because these organisations and institutions have been placed on a promiscuous anti-spam blacklist. Such blacklisting is especially problematic when you consider the existence of ‘spoof’ spam, where a sender’s email address and the server from which their email originates are both forged. If a spammer makes a spoof email look as though it originated from you, then you could soon find excessive anti-spam technology preventing you from sending legitimate email to anybody (18).

‘Challenge-response’ systems have different but equally problematic consequences when used as a method of combating spam. With these systems, senders of email are asked to prove their good faith by responding to a question. This question must be answered correctly, in order for the original email to reach its intended recipient. This sounds fine in practice, but it militates against the sending of bulk email (19).

The sending of bulk email is not just confined to spammers, but is also a legitimate technology that predates the existence of the worldwide web. Technology commentator Declan McCullagh points out that ‘while they may not be as glamorous as the web, peer-to-peer applications, or instant messaging software, mailing lists are the internet’s oldest form of mass communication. They date back to the original “MsgGroup” list in 1975’.

McCullagh points out that ‘challenge-response systems, ironically, share some characteristics with spam: in small quantities, both are only mildly annoying to the recipient. But as quantities increase, they make it more difficult to use email at all’. As with anti-spam litigation, the problem McCullagh identifies with challenge-response systems is that they penalise the less well-off internet user: ‘Big corporations may be able to afford to hire someone to sit in front of a computer and spend all day proving they’re not a spambot, but non-profit groups, individuals and smaller companies probably can’t.’ (20)

Is there an anti-spam technology that doesn’t suffer from the problem of false positives, and that can therefore be legitimately championed?

• Anti-spam technology can only be effective if it allows for individual choice

The problem with most of today’s methods of combating spam is that they take the decisions about what email we do and don’t want to receive out of our hands. Whether it’s a hopelessly broad legal definition or a promiscuous blacklist, solutions to spam tend to involve decisions being made on internet users’ behalf.

Yet it would be misguided to argue that the only legitimate solution to spam must involve individuals sitting at their keyboard hitting ‘delete’ for a substantial part of the day. The two broad criteria that really need to be met, in order for anti-spam systems to be effective, are:

• email must be categorised as spam as a result of an informed decision made by the individual internet user (this could be a decision made at any time before the user receives the email that is or is not categorised as spam);




• there must be flexibility and scope for adaptation in the system, so that it can accommodate differences of individual use and preference.

The spam solution developed to date that most closely meets these two criteria is the technological method of Bayesian filtering, as pioneered by computer scientist Paul Graham and others. This method aggregates statistics to determine the likelihood of email being spam, rather than concluding bluntly that it either is spam or it isn’t. This method also pays attention to what users do and don’t delete as spam, and then incorporates those decisions into subsequent assessments.

It is symptomatic of the spam debate that while Bayesian filtering has been recognised for its technical merits, its political significance has hardly been commented on. Instead, it is assumed that any measure to reduce spam is a good thing, regardless of the consequences, and so Bayesian filtering is treated as one necessary solution among many. But while the technical merits of Bayesian filtering as compared with other methods must of course be proven, it’s the philosophy behind Bayesian filtering that we can learn something from.

When explaining his system, Paul Graham treats the problem of false positives very seriously. He recognises that ‘missing legitimate email is an order of magnitude worse than receiving spam, so a filter that yields false positives is like an acne cure that carries a risk of death to the patient’ (21). Rather than adopting a siege mentality, we should be focusing attention on Bayesian filtering or equivalent systems, so that we can raise our expectations of how technology can combat spam and grant individual choice to the internet user.

This is a difficult argument to win, because in the short term, focusing all of our efforts against spam at the level of the individual user will be costly to those who carry and store email before it reaches us. But in the long term, such individual-focused methods are the only means of reversing the ‘whack the mole’ problem without generating false positives. The beauty of Bayesian filtering is that if it was developed to its full potential, then in order to get past it, spammers would have to make their emails so interesting to you that they would effectively cease to be spam.

If we fail to concentrate our efforts at the level of the user, then the openness and universality of the internet will be compromised by regulation, bureaucracy and caution. Indeed, Paul Judge of the Anti-Spam Research Group argues that the extent of the spam problem justifies questioning the very expectation of openness on the internet: ‘Is it time to consider closed systems?’, he asks (22). No, it isn’t. To resort to closed systems on the internet, when we have recourse to other means of combating spam, would really be killing the patient with the cure.

• Conclusion

Technologists and internet gurus who are otherwise pro-freedom, and who championed the principles behind the early development of the internet, have thrown their political principles away in the face of spam. Technology commentator and historian Brad Templeton accurately describes the way such political compromises have been made in relation to spam:

‘People who would defend the end-to-end principle of internet design eagerly hunt for mechanisms of centralised control to stop it. Those who would never agree with punishing the innocent to find the guilty in any other field happily advocate it to stop spam. Some conclude even entire nations must be blacklisted from sending email. One-time defenders of an open net with anonymous participation call for authentication certificates on every email. Former champions of flat-fee unlimited net access who railed against proposals for per-packet internet pricing propose per-message usage fees on email.’ (23)

Such a response to spam is entirely wrongheaded. Just because spam poses a considerable practical problem, that doesn’t justify exaggerating or misrepresenting the problem. And it certainly doesn’t justify compromising the principles of freedom and openness that ought to be at the heart of the internet.

Sandy Starr has consulted and written on internet regulation for the Organisation for Security and Cooperation in Europe, and for the European Commission research project RightsWatch. He is a contributor to Spreading the Word on the Internet: Sixteen Answers to Four Questions, Organisation for Security and Cooperation in Europe, 2003 (download this book (.pdf 576 KB)); From Quill to Cursor: Freedom of the Media in the Digital Era, Organisation for Security and Cooperation in Europe, 2003 (download this book (.pdf 399 KB)); and The Internet: Brave New World?, Hodder Murray, 2002 (buy this book from Amazon (UK) or Amazon (USA)).

(1) See Half of all emails are spam, BBC News, 31 May 2003; Before Friday comes spamday, Wendy Brewer, PC World, 8 May 2003; Spam war settles into mobile phones, Will Sturgeon, CNET News.com, 11 June 2003

(2) See Law school serves spam as main course, Paul Festa, CNET News.com, 2 June 2003; Putting my job where my mouth is, Lawrence Lessig, 1 January 2003

(3) See Bill would let spamees sue spammers, Declan McCullagh, CNET News.com, 12 June 2003; Congress finds rare unity in spam, to a point, Jennifer Lee, New York Times, 23 June 2003; All Party Parliamentary Internet Group to hold public inquiry on ‘spam’, All Party Parliamentary Internet Group, 13 June 2003

(4) See Sick of spam? Then blame Alan Ralsky. He emails a billion of them a day, Paul Harris, Observer, 1 June 2003

(5) Spam is not a problem, Mark Hurst, Good Experience, 30 May 2003

(6) Symantec survey revels more than 80 percent of children using email receive inappropriate spam daily, Symantec, 9 June 2003

(7) A bounty on spammers, Lawrence Lessig, CIO Insight, 16 September 2002

(8) See Federal trade commission reauthorization, US Senate Committee on Commerce, Science and Transportation, 11 June 2003; FTC seeks more power to fight junk email, Jonathan Krim, Washington Post, 12 June 2003

(9) See Microsoft sues 15 groups in broad attack on spam, Saul Hansell, New York Times, 18 June 2003; Spam comes under attack in British courts, Stuart Millar, Guardian, 18 June 2003

(10) See the Spam Laws website; Spam emails face ban demands, BBC News, 10 June 2003; OECD guidelines for protecting consumers from fraudulent and deceptive commercial practices across borders (.pdf 185 KB), Organisation for Economic Cooperation and Development, 11 June 2003

(11) Directive 2002/58/EC of the European Parliament and of the Council of 12 July 2002 Concerning the Processing of Personal Data and the Protection of Privacy in the Electronic Communications Sector (Directive On Privacy And Electronic Communications) (.pdf 161 KB), Article 13.3

(12) Directive 95/46/EC of the European Parliament and of the Council of 24 October 1995 on the Protection of Individuals with Regard to the Processing of Personal Data and on the Free Movement of Such Data (.pdf 1.01 MB), Article 2(h)

(13) See Should we can spam?, Sandy Starr, Tech Central Station Europe, 9 April 2003

(14) Conference on Freedom of the Media and the Internet, Organisation for Security and Cooperation in Europe, Amsterdam, 13-14 June 2003

(15) See Microsoft attacks spam at source, CNET News.com, 20 June 2003

(16) Strictly speaking, ‘RBL’ is a trademarked term for a list issued by the Mail Abuse Prevention Service, and there are preferred generic terms for such lists. It is the case, however, that ‘RBL’ is commonly used generically. See the Realtime Blackhole List section of the Mail Abuse Prevention Service website; and the DNS-based spam databases section of the Declude website

(17) The spam problem: moving beyond RBLs, Philip Jacob, 3 January 2003

(18) See BT tackles spam blacklist, BBC News, 9 June 2003; Communication, code and control: the privatisation of media regulation and censorship (.pdf 15.8 KB), Christian Ahlert, paper delivered to the Organisation for Security and Cooperation in Europe ‘Conference on Freedom of the Media and the Internet’, 13 June 2003; Could you be sending spam?, Lincoln Spector, PC World, 30 May 2003

(19) See EarthLink to offer anti-spam email system, Jonathan Krim, Washington Post, 7 May 2003

(20) Spam blockers may wreak e-mail havoc, Declan McCullagh, News.com, 27 May 2003

(21) A plan for spam, Paul Graham, August 2002

(22) Do I know you? Email barricades, Reuters, 8 June 2003

(23) Reflections on the 25th anniversary of spam, Brad Templeton