△ MENU/TOP △

Holtz Communications + Technology

Shel Holtz
Communicating at the Intersection of Business and Technology
SearchClose Icon

Blacklisting blogs

Neville and I were copied on an email from Steve O’Keefe over at the IAOC in which he said he tried sending out the association’s newsletter but it kept bouncing back. The reason, he learned, was the URL for our podcast, “For Immediate Release.” The URL was in the body of the message, but the mail server at the IAOC’s Internet Service Provider wouldn’t let the email go through as long as our URL was included. We were, it turns out, on a couple of blacklists.

Specifically, we were listed on some RBLs, which stands for “Real-time Blackhole List.” One was through a mail provider called Outblaze, the other a private service called SURBL. According to the SURBL Web site, “SURBLs differ from most other RBLs in that they’re used to detect spam based on message body URIs (usually web sites). Unlike most other RBLs, SURBLs are not used to block spam senders. Instead they allow you to block messages that have spam hosts which are mentioned in message bodies.”

Email spam is at the heart of these blacklists. That confounded Neville and me, since we don’t send any email at all from “For Immediate Release.” The site, in fact, is a blog. We have an email address so listeners can send us their comments, but when we reply we use our personal email accounts. Not one email has ever been sent from the forimmediaterelease.biz domain. Our guess: somebody spoofed our domain in a spam, although we can’t get Outblaze to let us know if that’s what happened.

Once we got it resolved, which required a flurry of emails, Neville found out his domain, nevon.net, was also blacklisted. Another round of emails was required to have his domain removed from the blacklist.

In both instances, the owner of the SURBL site suggested he’d feel better about whitelisting us if we had spam policies on our sites. It was the combination of the vigilante approach to spam coupled with the requirement to publish an email policy that raised my eyebrows. At their core, of course, blogs are web sites. But they are part of what I have taken to calling the “collaborative” or “social web,” not the “reference web” with which most people are most familiar. How many blogs distribute email of any kind? Damn few, I suspect. Should bloggers be forced to post email policies just to comply with individuals creating blacklists that ISPs use to keep spam out of their customers’ in-boxes? How many bloggers have given any thought to posting email policies? How many bloggers have even figured out whether their blogs’ URLs are on a blacklist?

Spam is a problem, to be sure. It’s one of the reasons RSS is growing some popular. And I applaud anybody who can figure out a way to deal with the spam problem. Spam has had a serious impact on the level of trust most people apply to email. But I’m troubled by individuals with no formal authority who add domains to blacklists without notifying the domain owner. If, by chance, a domain owner discovers he’s been blacklisted, rectifying the situation can be time-consuming and difficult.

I’ve added an email policy to the “For Immediate Release” blog just to avoid any further problems with vigilante anti-spammers. But there has to be a better way to go about this than penalizing innocent site owners and bloggers who have never sent a single email of any kind through their domains.

If you want to check your domain, here’s one place where you can find out if you’ve been blacklisted: http://www.rulesemporium.com/cgi-bin/uribl.cgi

06/20/05 | 44 Comments | Blacklisting blogs

Comments
  • 1.Was it really so "time consuming and difficult" to send us an email so we could unlist your domain? Many other RBLs don't even listen to or acknowledge complaints. We removed your domain within 24 hours of your contacting us (on a holiday weekend). I wonder whether a commercial service would be as responsive or helpful.

    SURBL lists are being used to block literally billions of spams per day. SURBL is a voluntary effort organized over the Internet with dozens of people around the world helping to provide data or services. Many more Internet users provide reports that also drive our listings.

    Our goal is to have zero errors: to list only major spammers, but perfection is seldom easy to attain. Despite that, we expend a lot of energy, resources and programming to try to keep legitimate domains off our list. We are aware of the power of our tools and are attempting to wield them responsibiliy. Instead of painting us as irresponsibile, perhaps a fairer reading would take into account our willingness and ability to delist legitimate domains such as yours.

    As far as "no formal authority," welcome to the Internet. The government is nearly powerless to stop spam, and when it tries, it comes up with seemingly worse than useless solutions like CAN SPAM which is appropriately named since it allows many spammers to send spam *legally*.

    The two most effective antispam services on the Internet are SpamHaus and CBL. Both are non-govermental and both are responsible for keeping the Internet working. Without them, you would not be able to use email or possibly even the web or blogs, since the infected senders and virused PCs they list could be used for many nefarious purposes other than just attempting to send multiple billions of spams per day. Without those "vigilante" services, the Internet might very well crawl to a halt. Certainly email would be nearly unusable under the deluge of spam actually getting through. Imagine getting ten thousand spams for every ham (desired message) in your mailbox, or ten thousand blog spams for every legitimate posting, and you may get an idea of how Internet life would be without these "vigilantes."

    Some of these "vigilantes" also provide data to the FBI and other international, national and local police organizations about criminal spam gang activity. Governmental cybercrime resources are pretty severely limited, but occasionally they are able to make good use of this information, and we see spammers indicted for their crimes or put in jail:

    http://www.businessweek.com/magazine/content/05_22/b3935001_mz001.htm

    Cheers,

    Jeff Chan

    Jeff Chan | May 2005 | Internet

  • 2.Shel, you completely miss the point.

    You should be directing your anger at the criminals who spoofed your domain, not at the end users who are only trying to protect themselves from spam.

    Want to fix the problem? Help put spammers in prison, instead of attacking end users who are only trying to defend themselves against the avalanches of email trash by these criminals.

    You yourself had to implement spam blocking on your blog by using a graphic capcha verifier. You realize this unfairly blocks sight impaired individuals from posting to your blog? Every method of spam blocking has imperfections. Your blocking has flaws. So does SURBL. Deal with it.

    Your effort would be better used attacking spammers, not those trying to defend themselves against spammers.

    Dan | May 2005

  • 3.There are three problems that I see.

    #1. How did you end up on Outblaze's list? Even if someone spoofed your domain name, it should be trivial to verify it.

    #2. How did you get on SURBL's list? I may be wrong, but isn't that for URL's found in spam email? Did someone intentionally set you up by sending out a spam blast with your URL on it?

    #3. Why was IAOC blocking your email just because of that URL? Was it just because of that URL?

    Without the answers to those questions, we cannot identify the real issue. It is possible that someone, for whatever reason, decided to get your site blacklisted. There isn't much that can be done about maliciousness such as that, if that is what happened.

    Brandioch | May 2005 | Seattle

  • 4.Shel, a great deal more research could have gone into this post before applying the vigilante stamp and going after highly respected volunteer service. A great many people work on SURBL to maintain quality data, but given the huge number of domains to consider, mistakes will occur from time to time. However, as has been made clear a number of times, SURBL does not block messages, but rather helps other filtering systems such as SpamAssassin decide how to handle a message. By default, the vast majority of filtering systems including SpamAssassin will not flag a message as spam based on a single SURBL hit. Either the IAOC mail administrators adjusted the default scores upwards or your email had other elements that combined with the SURBL hit caused it to be flagged as spam. It would be great if you could do a bit more research and see if your post ought to be ammended.

    Andy | May 2005

  • 5.Shel, I think you severely underestimate the scope of this spam problem. Our mail servers receive tens of thousands of spams every day - and without these amazing free solutions like SURBL and SpamAssassin I would probably be forced out of business. Spammers have caused us countless hours of grief over the past few years - I will do anything to stop them - anything. That means occasionally innocent sites get blacklisted. If you ask me its a small cost to pay. Your vigilante is our hero.

    USA Church Admin | May 2005 | Phoenix, AZ

  • 6.The posting script on this page blows up if you leave something out, such as your email address. Then everything you typed is gone because the BACK button on the error page does not work!

    Greg | May 2005

  • 7.Shel,

    While you're out there whinning, posting stuff you haven't thoroughly researched, your OH SO COOL! blog doesn't even munge poster's email addresses.
    If there was a clueless-blogger.surbl.org I'd request you get listed NOW!

    SpamHater | May 2005 | @Home

  • 8.Shel, I'm a PR consultant, and very frankly I already suggest clients using email newsletters to switch to RSS feed or at least to add RSS feed aside email newsletter, giving RSS as an opportunity. The RSS feed is by far a better solution under so many points of view; the crucial point of course is that the target community has to be open to adopt the technique. But I believe this is the future. We already offer journalists this opportunity : at the moment is only an experiment, but I'm confident.

    Enrico Bianchessi | May 2005 | Milan, Italy

  • 9.Wow. Such a lot of angst! I find it amazing that a PR blog draws the most comments about posts that had nothing to do with PR -- such as podcasting software or, in this case, spam.

    Did I ever suggest these services shouldn't exist, or did readers (obviously not regular readers of this blog) fly off the handle because I dared pose any questions at all?

    So let me reiterate and amplify. I thought Jeff Chan's response time was amazingly good, particularly over a weekend. I think Jeff is performing a necessary and important public service. My assertion that rectifying a situation in which a blogger finds himself blacklisted can be difficult and time-consuming has nothing to do with Jeff's service. It took me months to resolve a similar problem, though, with AOL. In fact, I've had to deal with it with multiple services, none of which ever notified me that my domain was on such a list.

    I've re-read my post several times and simply can't find the "attack." It's a recounting of an experience and a discussion of the new wrinkle related to blogs that don't send email.

    I am more aware than most of the depth of the spam issue, since I work in this area and subscribe to regular updates on the topic. (Who among the commenters reads "Michael Osterman on Messaging," which deals almost exclusively with spam and its various effects and solutions? How many of you are aware of a new report by the Organization for Economic Cooperation and Development (OECD) that shows spam causes more grief for developing countries than others?)

    My ONLY complaint with these services themselves is the failure to notify anybody that they have been blacklisted. If not for someone from the IAOC notifying Neville, we might never have known mail containing our podcast domain was being blocked. The balance of my issue addressed sites that SEND NO EMAIL. Why is this such a difficult concept? Technorati tracks 10 million blogs and I haven't seen one of them with an email policy. This is not a criticism of Jeff (who, again, was gracious, professional, and helpful) or anyone else, but a discussion of the notion that the Web is branching into a couple distinct paths, one of which simply doesn't involve email, and the fact that much of the Internet world hasn't caught up, continuing to lump all sites into one big category. Do we really expect every blogger to add an email policy to their blogs?

    I am working hard, by the way, to migrate the 4,000 readers of my email newsletter to an RSS feed. Email for legitimate mass distribution is dead. Spammers killed it. Yes, I despise spammers. I get about 1,000 spam messages per day in my spambox; I pay hard-earned money to SpamSoap to keep those messages out of my in-box, but still have to deal with them routinely or my spambox would fill my server. But my post was about our experience being blacklisted, not not about spam. My post was also designed to alert other bloggers that they may want to check to see if they've been blacklisted. They could be just as unaware of the potential as Neville and I were.

    To clarify, IAOC was not blocking our email. THEY were being blocked -- they could not send THEIR email newsletter as long as OUR domain appeared in the body of the message. It was THEIR ISP blocking based on our domain appearing in the SURBL list. See? I really do get it.

    I apologize if the word "vigilante" caused offense. I meant it in the purest sense of the word: "One who takes or advocates the taking of law enforcement into one's own hands." The governor of California is applauding Texas' border vigilantes and asking them to come to our state, for heaven's sake. I did not mean it in any negative way.

    Finally, I appreciate everyone who, while clearly upset with what I wrote, was considerate and civil in their comments. Civil discourse is what blogging should be (but isn't always) about.

    Shel Holtz | May 2005

  • 10.As a former member of SURBL, and now running another list at URIBL, I have to comment. We add thousands, yes thousands of domains a week. We have no time to inform people they are listed. We don't get paid for this work.

    We all strive to get zero false positives in our lists. People in the industry know how much Jeff cares about reducing these.

    The response rates for URIBL and SURBL are faster then any other RBL out there. When we make a mistake, we fix it ASAP.

    I think if you used these FREE services on your own email account you would be singing the praises, instead of labeling us as vigilantes and "individuals with no formal authority". Your right though, we are. Because the people with authority have their head so far up the DMA's rear, they produce crap like the CANSPAM act.

    We found a solution that works. Its free. We work hard. If you knew how much work went into the list, you would be amazed. We do it to help, not hurt.

    ANyway I need to end this rant before I hurt myself :) May your inbox be spam free.

    (I didn't even mention how we tell people NOT TO BLOCK based on our list!)

    PPS: Your subbmission says my email was invalid. Its broken. Plus, '+', is a perfectly legit charachter in an email address.

    Chris Santerre | May 2005

Comment Form

« Back