Block Bad Bots with Blackhole Pro + Save 25% with code: CENTAURUS Get plugin »
Tag Archive

Building the 3G Blacklist, Part 1: Improving Site Security by Recognizing and Exploiting Server Attack Patterns

In this series of five articles, I share insights and discoveries concerning website security and protecting against malicious attacks. In this first article of the series, I examine the process of identifying attack trends and using them to immunize against future attacks. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist. Improving Site Security by Recognizing and Exploiting Server Attack Patterns Crackers, spammers, scrapers, and other attackers are getting smarter and more […] Read more »

How to Block Proxy Servers via htaccess

Not too long ago, a reader going by the name of bjarbj78 asked about how to block proxy servers from accessing her website. Apparently, bjarbj78 had taken the time to compile a proxy blacklist of over 9,000 domains, only to discover afterwards that the formulated htaccess blacklisting strategy didn’t work as expected: deny from Blacklisting proxy servers by blocking individual domains seems like a futile exercise. Although there are a good number of reliable, consistent proxy domains that could be blocked directly, the vast majority of such sites are constantly changing. It would take a team of professionals […] Read more »

2G Blacklist: Closing the Door on Malicious Attacks

Since posting the Ultimate htaccess Blacklist and then the Ultimate htaccess Blacklist 2, I find myself dealing with a new breed of malicious attacks. It is no longer useful to simply block nefarious user agents because they are frequently faked. Likewise, blocking individual IP addresses is generally a waste of time because the attacks are coming from a decentralized network of zombie machines. Watching my error and access logs very closely, I have observed the following trends in current attacks: User agents are faked, typically using something generic like “Mozilla/5.0” Each attack may involve hundreds of compromised IP addresses Attacks […] Read more »

Three Ways to Allow Hotlinking in Specific Directories

After implementing any of the hotlink-prevention techniques described in our previous article, you may find it necessary to disable hotlink-protection for a specific directory. By default, htaccess rules apply to the directory in which it is located, as well as all subdirectories contained therein. There are (at least) three ways to enable selective hotlinking: Place hotlink images in an alternate directory This method works great if your hotlink-protection rules are located in a directory other than the site root. Simply create another directory outside of the directory containing the htaccess rules and place your hotlink-allowed images into that directory. Create […] Read more »

Stupid htaccess Trick: Enable File or Directory Access to Your Password-Protected Site

In this brief tutorial, we are going to enable users to access any file or directory of a site that is password-protected via htaccess. There are many reasons for wanting to employ this technique, including: Share public resources from an otherwise private site Enable visitors to access content during site maintenance Testing and formatting of layout and design during development As a webmaster, I have used this technique on several occasions. This trick works great for allowing access to any number of files, directories, and/or combination of both. We will begin with a generalized example, proceed with an explanatory discussion, […] Read more »

Creating the Ultimate htaccess Anti-Hotlinking Strategy

When I wrote my article, Stupid htaccess Tricks, a couple of years ago, hotlink-protection via htaccess was becoming very popular. Many webmasters and bloggers were getting tired of wasting bandwidth on hotlinked resources, and therefore turned to the power of htaccess to protect their content. At that time, there were only a couple of different hotlink-protection methods available on the internet, and the functional difference between them was virtually insignificant. All that was necessary for up-and-coming bloggers-slash-site-administrators to eliminate leaking bandwidth and stolen resources was a relatively straightforward copy-&-paste procedure. Implementing the de facto htaccess hotlink protection code required a […] Read more »

WordPress Lessons Learned, Part 1: Permalink Structure

While planning my current site renovation project, I considered changing the format of my permalinks. Reasons for modifying the permalink structure of a site include: Optimizing URLs for the search engines Simplifying URL structure for improved readability Removing the implication that your site content is somehow organized chronologically Removing other unwanted organizational implications (e.g., categorically, topically, etc.) Like many people who configured WordPress permalinks a couple of years ago, I chose to include the day, month, and year along with the blog URL and post title. For over two years now, Perishable Press has employed the following permalink format: Read more »

Ultimate htaccess Blacklist 2 (Compressed Version)

In our original htaccess blacklist article, we provide an extensive list of bad user agents. This so-called “Ultimate htaccess Blacklist” works great at blocking many different online villains: spammers, scammers, scrapers, scrappers, rippers, leechers — you name it. Yet, despite its usefulness, there is always room for improvement. For example, as reader Greg suggests, a compressed version of the blacklist would be very useful. In this post, we present a compressed version of our Ultimate htaccess Blacklist that features around 50 new agents. Whereas the original blacklist is approximately 8.6KB in size, the compressed version is only 3.4KB, even with […] Read more »

htaccess Combo Pack: WordPress Permalinks and non-www Redirect

WordPress users employing permalinks via htaccess to optimize their dynamic URLs transform complicated-looking links such as: ..into search-engine friendly links such as: Every rewritten URL relies on a common set of htaccess rules to transform the links. The htaccess rules for all WordPress permalinks look like this for root WP installations: # BEGIN WordPress <ifmodule mod_rewrite.c> RewriteEngine On RewriteBase / RewriteCond %{REQUEST_FILENAME} !-f RewriteCond %{REQUEST_FILENAME} !-d RewriteRule . /index.php [L] </ifmodule> # END WordPress ..and like this for subdirectory WP installations: Read more »

Temporary Site Redirect for Visitors during Site Updates

In our article Stupid htaccess Tricks, we present the htaccess code required for redirecting visitors temporarily during periods of site maintenance. Although the article provides everything needed to implement the temporary redirect, I think readers would benefit from a more thorough examination of the process — nothing too serious, just enough to get it right. After discussing temporary redirects via htaccess, I’ll also explain how to accomplish the same thing using only PHP. Read more »

Permanently Redirect a Specific IP Request for a Single Page via htaccess

Not the most interesting title, but “oh well”.. Recently, a reader named Alison left a comment requesting help with a particular htaccess trick. She wanted to know how to permanently redirect (301) all requests for a specific page when requested from a specific IP address. In other words, when a visitor coming from 123.456.789 requests the page requested-page.html, the visitor will be redirected to just-for-you.html. All visitors not coming from that specific IP address are not redirected, and thus will see the originally requested page. Further, the redirect must apply only to requested-page.html, such that every visitor — including the […] Read more »

Allow Google Reader Access to Hotlink-Protected Images

In our previous article, we explain the process of allowing Feedburner to access your hotlink-protected images. The article details the entire process, which covers the basics of hotlink protection and involves adding several lines of code to your htaccess file. In this article, we skip the detailed explanations and present only the main points. The discussion is very similar for both Feedburner and Google Reader, and may be extrapolated to serve virtually any purpose. If you are using htaccess to protect your images from hotlinking and have yet to check your feeds in Google Reader, you may be in for […] Read more »

Allow Feedburner Access to Hotlink-Protected Images

Recently, we installed and configured the excellent WordPress Feedburner plugin by the venerable Steve Smith 1. The plugin basically redirects our various WordPress-powered content feeds to Feedburner, which then delivers them to subscribers. This method enables us to take advantage of Feedburner’s excellent statistical tools. Further, all of the action happens silently, beneath the surface, and without the subscriber even realizing it. After a few weeks running the plugin with great success, we began hearing reports of broken and missing images messing up our feeds. After some investigating, we realized that our tried-and-true anti-hotlinking htaccess rules were doing their job […] Read more »

Major Problem with cPanel Hotlink Protection and htaccess

There is a major problem with the “Hotlink Protection” feature of cPanel. To summarize the issue, allow me to quote a recent email sent to a completely unresponsive tech support department: …The problem is that if I try to include any rewrite rules for permalinks, hotlinking, or blocking spambots, cPanel automatically enables its “Hotlink Protection” feature. And, even worse, it automatically adds every URL from every rewrite rule (even the ones for blocking spambots) to its “auto-discovered” list of URL’s for which image access is allowed. This means that every spammer that I am trying to block now has access […] Read more »

Ultimate htaccess Blacklist

For those of us running Apache, htaccess rewrite rules provide an excellent way to block spammers, scrapers, and other scumbags easily and effectively. While there are many htaccess tricks involving blocking domains, preventing access, and redirecting traffic, Apache’s mod_rewrite module enables us to target bad agents by testing the user-agent string against a predefined blacklist of unwanted visitors. Any matches are immediately and quietly denied access. There are many ways to obtain an effective htaccess blacklist. There are several excellent forums around the web that provide a plethora of priceless htaccess advice. Highly suggested. Additionally, after copying and pasting your […] Read more »

Compressed JavaScript Compression

In this article, we extrapolate our favorite CSS-compression technique for JavaScript. Below, we outline the steps required to auto-compress your JavaScript documents via gzip and PHP. Two different compression methods are presented. The first method does not require htaccess, but rather involves the manual editing of JavaScript files. The second method employs htaccess to do all the work for you, thus requiring much less effort to implement. In either case, the result is the same: automatically compressed content delivered only to supportive browsers, resulting in reduced bandwidth, faster loading times, and smiling visitors :) Read more »

Latest Tweets How to block bad bots:…