Spam Proofing Your Website
Posted on 22 Oct 2002
by Dan Thies (optimization)
Rated 3.79 (Ratings: 17)
- More articles in Site Development
Spam-Proofing Your Website
Anyone who operates their own website knows that you need to provide a way for visitors to contact you by email. The big challenge is providing easy email access to your visitors, without letting junk mail (SPAM) flood your email inbox. The techniques described in this article have enabled me to dramatically reduce the amount of junk mail I receive on all of my websites.
Preparing and Preempting
You need a couple things before you can really take effective action against SPAM. Your email software must be capable of filtering incoming email. All of the major email applications (such as Eudora, Outlook, and Pegasus) support filtering. We will use multiple email addresses to allow us to filter out SPAM and identify the source - you can't combat SPAM effectively without them.
You need to use a website hosting provide that allows unlimited email aliases or addresses, and/or a catch-all email address. An "alias" is an email address that forwards to some other address (for example, firstname.lastname@example.org forwarding to your real email address). A "catch-all" email address will forward any emails sent to unknown addresses in your domain.
For my own websites, I just use the catch-all, so that every message goes to my real email address. If you have more than a one-person operation, however, multiple email accounts and aliases are pretty much a necessity. Any email address you use online could become a target of spam. If your hosting provider is especially good, you may even be able to create email aliases that automatically delete all incoming messages.
The first step in fighting back against the spammers is understanding where they get your email address. You must diligently protect your email address, if you ever hope to stop them. Once your email address gets into the wrong hands, it will be sold on CD-ROM (via junk mail, of course) to thousands of spammers. Once that happens, you've lost the fight.
Spam Source #1: Domain Name Registrations
When you register a domain name, you must provide a contact email address. If you give them your real email address, you've just given it to everyone, including the spammers. Instead, use a portable email address (like Hotmail) to set up your domain.
If you have multiple domains, you can also use an alias (email@example.com) on your primary domain for all registrations. With an alias, you can use your email software to filter out and save any emails that come to that address from your registrar's domain.
Spam Source #2: Web Forms & Email Newsletters
If you give your real email address on any web form, or use it to subscribe to an email newsletter, you are asking for trouble. Instead, create a unique email address for each website or newsletter. I just use the website's domain name for this.
For example, if you subscribe to the "evolt.org" mailing list as "firstname.lastname@example.org" and let your catch-all address route it to you, you will always know where the email came from. If that address ever starts receiving junk mail, you can filter it out using your email software.
If you submit to search engines or free-for-all links pages (FFA's), use a unique email address every time. FFAs, in particular, are famous for flooding the world with junk mail. Once you've given an email address to an FFA, you may as well forget about ever using it again.
Spam Source #3: Your Website
The biggest source of email addresses used by spammers is your website. Most websites list multiple contact addresses, etc. Any time an email address appears on your website in plain text, even if it's hidden in a form field, you're opening yourself up to having that email address captured.
The Big Battle: Securing Your Website From Spambots
Almost every website operator wants search engine spiders to visit. After all, search engines are the best source of free traffic on the web. In the event that you don't want them to visit, they are easily kept at bay with a properly formatted "robots.txt" file.
Unfortunately, there's another group of spiders out there crawling the web, with an entirely different purpose. These are the spiders that visit site after site, collecting email addresses. You may know them as spambots, email harvesters, or any number of unpublishable names.
When it comes to controlling these rogue spiders, a robots.txt file simply won't get the job done. In fact, most spam robots ignore robots.txt. That doesn't mean you have to give up, and just let them have their way. The following techniques will stop these spiders in their tracks.
In the examples below, simply substitute your username (the first half of your email address, everything before the @ symbol) and your hostname (everything after the @ symbol). To use the scripts, just insert them into your page's HTML wherever you need them to be displayed.
Example 1: Creating A Spam-Proof Mailto Link
Example 2: A Spam-Proof Mailto Link With Your Email Address Showing
Some visitors won't be able to use a mailto link. This snippet shows your email address in the link so they can copy and paste, or type it by hand:
Example 3: Display Your Email Address Without A Mailto Link
Here's a snippet that displays your email address a clickable link:
Technique #2: Use A Contact Form
Sometimes, the sheer volume of legitimate email from real visitors can become a burden. In this case, a simple solution is to remove your email address from your site entirely, and use a contact form. There are dozens of free ASP, Perl, and PHP scripts available online that will allow your users to fill in a form, and send you an email. Most hosting providers now offer this service for free to their customers.
A contact form can enable you to deal with a higher volume of mail, by allowing you to pre-sort different types of message. This is easily accomplished by creating a drop-down menu with different options (e.g. customer service, billing, tech support, etc.) that will populate the subject line of the email message, and/or change the email address to which the form is sent.
Example 4: Masking The Email Address In A Form Field
Instead of simply listing your email address in a form field, use the snippet below to replace the form field that contains your email address.
All contact forms, regardless of the language used, will work more or less the same way. Users will fill out a form, which is processed by a script on your server that emails the submissions to you. Because the script that runs on your server, your visitors never see the contents of that script. Hiding your email address in the script provides the greatest security, but this does require some programming knowledge. Form-to-email scripts that are offered by hosting companies almost always require the email address to be included as a hidden form field in your web page.
If you take this approach, it is much better to hide your email address within the script itself, rather than in the HTML code of your page. If you can't hide the address in the script, use an email alias, so that you can change the email address in the form from time to time, whenever the spam gets out of hand.
Advanced Techniques: URL Rewriting
Both the Apache and IIS web servers have plug-in URL-rewriting modules that can be used to provide additional protection to your website, by redirecting queries from known spambots to a blank page, or to another website. These techniques are beyond the scope of this article, and using them will slow your server down, if only a little. For a good discussion on using this technique, including its use to combat spambots, see http://www.webmasterworld.com/forum13/687.htm.
URL-rewriting is a powerful technique, however, and should not be overlooked. In addition to its potential value in deterring spam, it can also be used to prevent users from downloading your website with offline browsers, MS FrontPage, etc. If your content must be protected from unauthorized copying or other misuse, judicious use of URL-rewriting may be exactly what the doctor ordered. Be aware, though, that not everyone attempting to download your website is doing so with bad intentions.
For a good example of what you might want to show those who try to download your site, see this page: http://www.purplemath.com/terms.htm.
Thanks for reading...I hope that this tutorial has given you a clear understanding of how to protect your website, and your email address, from spammers and spambots. Your feedback is welcome. If you have any questions about this article, feel free to contact me through my website. The (spam-proof) email link can be found at the bottom of my home page.
I wish you success...