Building the 3G Blacklist, Part 4: Improving RedirectMatch in the Original 2G Blacklist

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this fourth article, I build upon previous ideas and techniques by improving the directives contained in the original 2G Blacklist. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in the release of the next generation 3G Blacklist.

Improving RedirectMatch in the Original 2G Blacklist

In the first version (2G) of the next-generation blacklist, a collection of malicious attack strings were compiled from access logs, blocked via Apache’s RedirectMatch directive, tested for effectiveness and transparency (i.e., non-interference), and finally converted into the 2G Blacklist. This 2G Blacklist continues to serve as my primary line of defense here at Perishable Press, replacing several other lists, including a million individual IP addresses and even the Ultimate htaccess Blacklist itself. So far, the results have extraordinary.

Of course, as discussed in Building the 3G Blacklist, Part 3, static, “fix-it-and-forget-it”-type strategies ultimately fail when dealing with the perpetually evolving nature of the enemy (i.e., malicious attackers, crackers, and other scumbags). Thus, as stated in the original 2G article, the blacklist is a work in progress, designed for periodic updates as new attack patterns and trends are identified. In the remainder of this article, we consider various aspects of the previous blacklist and consider various improvements.

Note that it is not necessary to assemble and transcribe any of the new blacklist directives presented herein. At the conclusion of this series, each of the different strategies will be combined into a single, comprehensive security strategy: The 3G Blacklist!

Miscellaneous Character Strings

Besides query strings, miscellaneous, random, and unusual character strings are perhaps the most frequently observed characteristics of common server attacks. For example, each of these sequences is a common sight to those of us concerned with site traffic and security:

• /,
• //
• f-.
• ...

Yet these strings are missing from the previous version of the blacklist. In the new version, we block these character strings, as well as several others:

• :
• ;
• <
• >
• .inc
• alt=
• ftp:
• ttp:
• .$url
• /$url
• /$link

This is powerful stuff. Any URL request containing one (or more) of these character strings will be denied via an immediate 403 Forbidden error. Attackers frequently target server-side includes (.inc), which are protected here in the fifth line. Inclusion of various hypertext protocols are also common, and are blocked here in the third and fourth lines. Another common attack vector involves the ubiquitous $url variable (often recorded as \\\'.$url.\\\'), which is also protected in this set in the last two lines. Compared with the equivalent section of the previous blacklist,

• .inc
• alt=
• http://

..the new version is much improved, covering a far more expansive field of vulnerability using only a few additional lines.

PHP File Types

In the previous blacklist, common PHP file targets were added to the list in ”shotgun”-like fashion. While sorting out the madness, readers pointed out several conflicts related to the blacklist:

• file.php — interferes with WordPress administration
• port.php — interferes with WordPress administration
• index2.php — interferes with Joomla backend [ reported by Don ]

In addition to correcting these discrepancies, PHP-related expressions that are highly specific to a particular platform or context (e.g., admin.webring.docs.php) have been generalized to match any file type. Conversely, PHP-related expressions that are more general in application (e.g., about.php) have been left unaltered. Further, several new PHP file names have been added, some have been replaced, and some have been removed.

Here is the PHP-related section as it appears in the previous, 2G Blacklist:

redirectmatch 403 menu\.php
redirectmatch 403 main\.php
redirectmatch 403 file\.php
redirectmatch 403 home\.php
redirectmatch 403 view\.php
redirectmatch 403 about\.php
redirectmatch 403 order\.php
redirectmatch 403 index2\.php
redirectmatch 403 errors\.php
redirectmatch 403 config\.php
redirectmatch 403 button\.php
redirectmatch 403 middle\.php
redirectmatch 403 threads\.php
redirectmatch 403 contact\.php
redirectmatch 403 display\.cgi
redirectmatch 403 display\.php
redirectmatch 403 include\.php
redirectmatch 403 register\.php
redirectmatch 403 db_connect\.php
redirectmatch 403 doeditconfig\.php
redirectmatch 403 send\_reminders\.php 
redirectmatch 403 admin_db_utilities\.php
redirectmatch 403 admin\.webring\.docs\.php

And likewise, here is the refined PHP-related section as it appears in the new, 3G Blacklist:

RedirectMatch 403 news\.php
RedirectMatch 403 menu\.php
RedirectMatch 403 main\.php 
RedirectMatch 403 home\.php
RedirectMatch 403 view\.php
RedirectMatch 403 about\.php
RedirectMatch 403 blank\.php
RedirectMatch 403 block\.php
RedirectMatch 403 order\.php
RedirectMatch 403 search\.php
RedirectMatch 403 errors\.php
RedirectMatch 403 config\.php
RedirectMatch 403 button\.php
RedirectMatch 403 middle\.php
RedirectMatch 403 threads\.php
RedirectMatch 403 contact\.php
RedirectMatch 403 include\.php
RedirectMatch 403 display\.php
RedirectMatch 403 register\.php
RedirectMatch 403 authorize\.php
RedirectMatch 403 \/wp\-signup\.php

Directory Path Segments

Also new in the 3G Blacklist are several common directory path segments. These directory names appear frequently in malicious attacks, even though they have no business being requested directly via http protocol, especially by someone trying to hack your site. ;)

Here are the newly added directory path segments:

RedirectMatch 403 \/scripts\/
RedirectMatch 403 \/modules\/
RedirectMatch 403 \/classes\/
RedirectMatch 403 \/includes\/
RedirectMatch 403 \/components\/
RedirectMatch 403 \/administrator\/
RedirectMatch 403 \/path\_to\_script\/

Certainly straightforward. The \/ on either side of these names ensure that only directories are matched. Including these terms in post titles is completely safe. As an aside, I may end up removing \/path\_to\_script\/ in the 3G Blacklist. If you think it should remain, drop a comment and explain. Likewise, if you see potential conflicts with any of the other expressions, please let me know.

Miscellaneous Functions

Another commonly observed component of malicious attacks includes a variety of miscellaneous functions appended onto the end of URLs. The previous, 2G Blacklist does not include such functions, but I finally got sick of seeing the little bastards squirming their way into my access logs. Thus, the new blacklist blocks any of the following:

RedirectMatch 403 function\.main
RedirectMatch 403 function\.mkdir
RedirectMatch 403 function\.opendir
RedirectMatch 403 function\.require
RedirectMatch 403 function\.array\-rand
RedirectMatch 403 ref\.outcontrol

These directives are my favorite part of the new list. If you know of other, similar functions commonly used for exploits, please share them so that they may added to the complete blacklist.

Omissions and Other Changes

Other potential improvements to the previous, 2G Blacklist provide protection for users running a vBulletin Board by blocking the following strings:

• f-\.html
• ImpEvData\.php
• check_proxy\.html

These rules are recommended by Peter and have been assimilated into 3G.

Lastly, I will mention the removal of eleven esoteric, randomly generated character strings that are no longer necessary. These were most likely one-time vulnerability probes no longer in service. Here are the dropped entries:

redirectmatch 403 keds\.lpti
redirectmatch 403 r\.verees
redirectmatch 403 pictureofmyself
redirectmatch 403 remoteFile
redirectmatch 403 mybabyboy
redirectmatch 403 mariostar
redirectmatch 403 zaperyan
redirectmatch 403 babyboy
redirectmatch 403 aboutme
redirectmatch 403 xAou6
redirectmatch 403 qymux

And finally, the redirectmatch directives as written in the 2G Blacklist have been rewritten to adhere to accepted standards:

• OLD: redirectmacth
• NEW: RedirectMacth

A subtle improvement, perhaps, but an improvement nonetheless!

Wrap Up..

With this “sneak peak” into the rules comprising the upcoming 3G Blacklist, we enjoy the opportunity to suggest changes, add directives, and improve the overall strategy before the final release. The changes discussed here represent a majority of the improvements made involving RedirectMatch directives, however there remain several other changes that will be included in the upcoming release of the 3G Blacklist. Also, if you see any potential issues or conflicts (or worse) involving any of the code presented here, please speak up and leave a comment. Thank you!

Next..

Stay tuned for the conclusion of the Building the 3G Blacklist series, Part 5: Improving Site Security by Selectively Blocking Individual IPs. If you have yet to do so, I highly encourage you to subscribe to Perishable Press. As always, thank you for your generous attention.

About the Author

Jeff Starr = Web Developer. Book Author. Secretly Important.