Perishable Press

After much research and discussion, I have developed a concise, lightweight security strategy for Apache-powered websites. Prior to the development of this strategy, I relied on several extensive blacklists to protect my sites against malicious user agents and IP addresses. Over time, these mega-lists became unmanageable and ineffective. As increasing numbers of attacks hit my server, I began developing new techniques for defending against external threats. This work soon culminated in the release of a “next-generation” blacklist that works by targeting common elements of decentralized server attacks. Consisting of a mere 37 lines, this “2G” Blacklist provided enough protection to enable me to completely eliminate over 350 blacklisting directives from my site’s root htaccess file. This improvement increased site performance and decreased attack rates, however many bad hits were still getting through. More work was needed..

The 3G Blacklist

Work on the 3G Blacklist required several weeks of research, testing, and analysis. During the development process, five major improvements were discovered, documented, and implemented. Using pattern recognition, access immunization, and multiple layers of protection, the 3G Blacklist serves as an extremely effective security strategy for preventing a vast majority of common exploits. The list consists of four distinct parts, providing multiple layers of protection while synergizing into a comprehensive defense mechanism. Further, as discussed in previous articles, the 3G Blacklist is designed to be as lightweight and flexible as possible, thereby facilitating periodic cultivation and maintenance. Sound good? Here it is:

# PERISHABLE PRESS 3G BLACKLIST

# PART I: CHARACTER STRINGS
<IfModule mod_alias.c>
 RedirectMatch 403 \:
 RedirectMatch 403 \;
 RedirectMatch 403 \<
 RedirectMatch 403 \>
 RedirectMatch 403 \/\,
 RedirectMatch 403 \/\/
 RedirectMatch 403 f\-\.
 RedirectMatch 403 \.\.\.
 RedirectMatch 403 \.inc
 RedirectMatch 403 alt\=
 RedirectMatch 403 ftp\:
 RedirectMatch 403 ttp\:
 RedirectMatch 403 \.\$url
 RedirectMatch 403 \/\$url
 RedirectMatch 403 \/\$link
 RedirectMatch 403 news\.php
 RedirectMatch 403 menu\.php
 RedirectMatch 403 main\.php
 RedirectMatch 403 home\.php
 RedirectMatch 403 view\.php
 RedirectMatch 403 about\.php
 RedirectMatch 403 blank\.php
 RedirectMatch 403 block\.php
 RedirectMatch 403 order\.php
 RedirectMatch 403 search\.php
 RedirectMatch 403 errors\.php
 RedirectMatch 403 config\.php
 RedirectMatch 403 button\.php
 RedirectMatch 403 middle\.php
 RedirectMatch 403 threads\.php
 RedirectMatch 403 contact\.php
 RedirectMatch 403 include\.php
 RedirectMatch 403 display\.php
 RedirectMatch 403 register\.php
 RedirectMatch 403 authorize\.php
 RedirectMatch 403 \/wp\-signup\.php
 RedirectMatch 403 \/modules\/
 RedirectMatch 403 \/classes\/
 RedirectMatch 403 \/includes\/
 RedirectMatch 403 \/components\/
 RedirectMatch 403 \/administrator\/
 RedirectMatch 403 \/path\_to\_script\/
 RedirectMatch 403 ImpEvData\.
 RedirectMatch 403 head\_auth\.
 RedirectMatch 403 db\_connect\.
 RedirectMatch 403 check\_proxy\.
 RedirectMatch 403 doeditconfig\.
 RedirectMatch 403 submit\_links\.
 RedirectMatch 403 change\_action\.
 RedirectMatch 403 send\_reminders\.
 RedirectMatch 403 comment\-template\.
 RedirectMatch 403 syntax\_highlight\.
 RedirectMatch 403 admin\_db\_utilities\.
 RedirectMatch 403 admin\.webring\.docs\.
 RedirectMatch 403 function\.main
 RedirectMatch 403 function\.mkdir
 RedirectMatch 403 function\.opendir
 RedirectMatch 403 function\.require
 RedirectMatch 403 function\.array\-rand
 RedirectMatch 403 ref\.outcontrol
</IfModule>

# PART II: QUERY STRINGS 
<ifmodule mod_rewrite.c>
 RewriteCond %{QUERY_STRING} ftp\:   [NC,OR]
 RewriteCond %{QUERY_STRING} http\:  [NC,OR]
 RewriteCond %{QUERY_STRING} https\: [NC,OR]
 RewriteCond %{QUERY_STRING} \[      [NC,OR]
 RewriteCond %{QUERY_STRING} \]      [NC]
 RewriteRule .* -                    [F,L]
</ifmodule>

# PART III: USER AGENTS
SetEnvIfNoCase User-Agent "Jakarta Commons" keep_out
SetEnvIfNoCase User-Agent "Y!OASIS/TEST"    keep_out
SetEnvIfNoCase User-Agent "libwww-perl"     keep_out
SetEnvIfNoCase User-Agent "MOT-MPx220"      keep_out
SetEnvIfNoCase User-Agent "MJ12bot"         keep_out
SetEnvIfNoCase User-Agent "Nutch"           keep_out
SetEnvIfNoCase User-Agent "cr4nk"           keep_out
<Limit GET POST PUT>
 order allow,deny
 allow from all
 deny from env=keep_out
</Limit>

# PART IV: IP ADDRESSES
<Limit GET POST PUT>
 order allow,deny
 allow from all
 deny from 75.126.85.215  "# blacklist candidate 2008-01-02 = admin-ajax.php attack "
 deny from 128.111.48.138 "# blacklist candidate 2008-02-10 = cryptic character strings "
 deny from 87.248.163.54  "# blacklist candidate 2008-03-09 = block administrative attacks "
 deny from 84.122.143.99  "# blacklist candidate 2008-04-27 = block clam store loser "
</Limit>

Installation and Usage

Before using the 3G Blacklist, check the following system requirements:

• Linux server running Apache
• Enabled Apache module: mod_alias
• Enabled Apache module: mod_rewrite
• Ability to edit your site’s root htaccess file (or)
• Ability to modify Apache’s server configuration file

With these requirements met, copy and paste the entire 3G Blacklist into either the root htaccess file or the server configuration file. After uploading, visit your site and check proper loading of as many different types of pages as possible. For example, if you are running a blogging platform (such as WordPress), test different page views (single, archive, category, home, etc.), log into and surf the admin pages (plugins, themes, options, posts, etc.), and also check peripheral elements such as individual images, available downloads, and alternate protocols (FTP, HTTPS, etc.).

While the 3G Blacklist is designed to target only the bad guys, the regular expressions used in the list may interfere with legitimate URL access. If this happens, the browsing device will display a 403 Forbidden error. Don’t panic! Simply check the blocked URL, locate the matching blacklist string, and disable the directive by placing a pound sign ( # ) at the beginning of the associated line. Once the correct line is commented out, the blocked URL should load normally. Also, if you do happen to experience any conflicts involving the 3G Blacklist, please leave a comment or contact me directly. Thank you :)

Wrap Up..

As my readers know, I am serious about site security. Nothing gets my adrenaline pumping more than the thought of a giant meat grinder squirting out endless chunks of mangled cracker meat. Spam and other exploitative activity on the web has grown exponentially. Targeting and blocking individual agents and IP is no longer a viable strategy. By recognizing and immunizing against the broadest array of common attack elements, the 3G Blacklist maximizes resources while providing solid defense against malicious attacks.

Updates

• 2008/05/14 — removed “RedirectMatch 403 \/scripts\/” from the first part of the blacklist due to conflict with Mint Statistics.

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. Wrapping up the series with this article, I provide the final key to our comprehensive blacklist strategy: selectively blocking individual IPs. Previous articles also focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. In the next article, these five articles will culminate in the release of the next generation 3G Blacklist.

Improving Site Security by Selectively Blocking Individual IPs

The final component of the 3G Blacklist establishes a vehicle through which individual IPs may be blocked. As previously discussed, the goal of the 3G Blacklist involves moving away from extensive lists of blacklisted user agents, IP addresses, and other identifying attributes. Nonetheless, a comprehensive security strategy benefits from the ability to quickly and easily neutralize threats by directly blocking individual IP addresses.

In my experience, an effective IP blacklist should be kept as current as possible. Unfortunately, once lists grow to over 100 entries, they become quite unmanageable. Periodic verification of blocked IPs becomes increasingly difficult as lists grow in size. Uncultivated blacklists that continue to grow in size will eventually do more harm than good. Beyond the burden placed on server resources, constantly changing IP ownership inevitably results in unintentional blocking of legitimate traffic.

Just as with individual blocking of user agents, blocking individual (or ranges of) IP addresses provides a secondary layer of protection against direct attacks, corrupt networks, and other case-specific situations. This level of granular control enables defensive fine-tuning for targeting and blocking specific machines. Known repeat offenders should remain permanently blacklisted, whereas short-term offenders should be blocked temporarily. Examples of known repeat offenders typically include:

Continue

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this fourth article, I build upon previous ideas and techniques by improving the directives contained in the original, 2G Blacklist. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist.

Improving the RedirectMatch Directives of the Original 2G Blacklist

In the first version (2G) of the next-generation blacklist, a collection of malicious attack strings were compiled from access logs, blocked via Apache’s RedirectMatch directive, tested for effectiveness and transparency (i.e., non-interference), and finally converted into the 2G Blacklist. This 2G Blacklist continues to serve as my primary line of defense here at Perishable Press, replacing several other lists, including a million individual IP addresses and even the Ultimate htaccess Blacklist itself. So far, the results have extraordinary.

Of course, as discussed in Building the 3G Blacklist, Part 3, static, “fix-it-and-forget-it”-type strategies ultimately fail when dealing with the perpetually evolving nature of the enemy (i.e., malicious attackers, crackers, and other scumbags). Thus, as stated in the original 2G article, the blacklist is a work in progress, designed for periodic updates as new attack patterns and trends are identified. In the remainder of this article, we consider various aspects of the previous blacklist and consider various improvements. Note that it is not necessary to assemble and transcribe any of the new blacklist directives presented herein. At the conclusion of this series, each of the different strategies will be combined into a single, comprehensive security strategy: The 3G Blacklist!

Continue

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this third article, I discuss targeted, user-agent blacklisting and present an alternate approach to preventing site access for the most prevalent and malicious user agents. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist.

Improving Site Security by Selectively Blocking Rogue User Agents

Several months ago, while developing improved methods for protecting websites against malicious attacks, I decided to remove the Ultimate htaccess Blacklist from the Perishable Press domain. In its place, I have been using the 2G Blacklist, individual IP blocking, and an extremely concise selection of blacklisted user-agents. So far, this new, streamlined strategy has proven just as (if not more) effective as the previous “ultimate-blacklist” method.

There are several reasons why using extensive lists of blocked user agents inevitably fails to provide worthwhile protection. First, creating and maintaining a current list of every nasty agent consumes far too much time. Once lists become significantly populated, efficient cultivation becomes increasingly difficult. Once an agent is added to the list, it typically remains there indefinitely, unless the required time is spent to periodically verify and edit each and every entry.

Continue

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this second article, I present an incredibly powerful method for eliminating malicious query string exploits. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist.

Improving Site Security by Preventing Malicious Query String Exploits

A vast majority of website attacks involves appending malicious query strings onto legitimate, indexed URLs. Any webmaster serious about site security is well-familiar with the following generalized access-log entries:

Continue

In this series of five articles, I share insights and discoveries concerning website security and protecting against malicious attacks. In this first article of the series, I examine the process of identifying attack trends and using them to immunize against future attacks. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist.

Improving Site Security by Recognizing and Exploiting Server Attack Patterns

Crackers, spammers, scrapers, and other attackers are getting smarter and more creative with their methods. It is becoming increasingly difficult to identify elements of an attack that may be effectively manipulated to prevent further attempts. For example, attackers almost exclusively employ decentralized botnets to execute their attacks. Such decentralized networks consist of large numbers of compromised computer systems that are remotely controlled to post data, run scripts, and request resources. Each compromised computer used in an attack is associated with a unique IP, thus making it impossible to prevent future attempts by blocking one or more addresses:

Continue

During my previous rendezvous involving comprehensive canonicalization for WordPress, I offer my personally customized technique for ensuring consistently precise and accurate URL delivery. That particular method targets WordPress exclusively (although the logic could be manipulated for general use), and requires a bit of editing to adapt the code to each particular configuration. In this follow-up tutorial, I present a basic www-canonicalization technique that accomplishes the following:

• requires or removes the www prefix for all URLs
• absolutely no editing when requiring the www prefix
• minimal amount of editing when removing the www prefix
• minimal amount of code used to execute either technique

I have found this “universal” www-canonicalization technique extremely useful in its simplicity and elegance. Especially when requiring the www prefix, nothing could be easier: simply copy, paste, done — absolutely no hard-coding necessary!

Continue

Recently, while restoring the popular Jupiter! WordPress theme, which several readers use to “skin” the Perishable Press website, I found myself searching for a simple, effective JavaScript technique for toggling element visibility. Specifically, I needed to accomplish the following design goals:

• Users should be able to toggle the visibility of any division containing post-meta information
• The post-meta information should remain visible by default and when JavaScript is unavailable
• The JavaScript should be as unobtrusive as possible, requiring minimal scripting in the markup

Here are a couple of screenshots demonstrating this repetitious toggling functionality as employed in the Jupiter! theme (click on either thumbnail for larger screenshot):

Continue

Welcome to the Perishable Press “Blacklist Candidate” series. In this post, we continue our new tradition of exposing, humiliating and banishing spammers, crackers and other worthless scumbags..

Since the implementation of my 2G Blacklist, I have enjoyed a significant decrease in the overall number and variety of site attacks. In fact, I had to time-travel back to March 1st just to find a candidate worthy of this month’s blacklist spotlight. I felt like Rod Roddy looking over the Price-is-Right audience to announce the next name only to discover a quiet, empty room. And then like Bob gets pissed that nobody showed up and begins to bark and snarl at Rod to go across the street to the clam store and find some damn contestants. Or, ..um, something like that. Needless to say, this month’s data isn’t as fresh as I would have liked it, but I think you’ll find the information fascinating nonetheless. So let’s get on with it then:

Blacklist Candidate number 2008-04-27, come on down! You’re the next clam-store loser to get blacklisted from the site!

Continue

The music of Pink Floyd is an audiophile’s wet dream. It is at once diverse, complex, intricate, rhythmic, mysterious, and downright heavy. The vastly inspiring soundscapes created by Pink Floyd are perfect for late nights spent working on the computer. Given a nice pair of high-quality headphones, the following hand-picked collection of Pink Floyd songs is sure to elevate your listening experience to the next level.

Continue

Following my recent post on CSS code formatting, I was delightfully surprised to have received such insightful, enthusiastic feedback. Apparently, I am not the only person passionate about the subtle nuances involved with the formatting of CSS code. So, to continue the conversation, let’s explore several techniques for writing the opening and closing brackets of CSS declaration blocks.

Continue