The Perishable Press 4G Blacklist
At last! After many months of collecting data, crafting directives, and testing results, I am thrilled to announce the release of the 4G Blacklist! The 4G Blacklist is a next-generation protective firewall that secures your site against a wide range of automated attacks and other malicious activity.
Like its 3G predecessor, the 4G Blacklist is designed for use on Apache servers and is easily implemented via HTAccess or the httpd.conf
configuration file. In order to function properly, the 4G Blacklist requires two specific Apache modules, mod_rewrite
and mod_alias
. As with the third generation of the blacklist, the 4G Blacklist consists of multiple parts:
- HTAccess Essentials
- Request-Method Filtering
- IP Address Blacklist
- Query-String Blacklist
- URL Blacklist
Each of these methods is designed to protect different aspects of your site. They may be used independently, mixed and matched, or combined to create the complete 4G Blacklist. This modularity provides flexibility for different implementations while facilitating the testing and updating process. The core of the 4G Blacklist consists of the last two methods, the Query-String and URL Blacklists. These two sections provide an enormous amount of protection against many potentially devastating attacks. Everything else is just icing on the cake. Speaking of which, there are also two more completely optional sections of the 4G Blacklist, namely:
These two sections have been removed from the 4G Blacklist and relegated to “optional” status because they are no longer necessary. Put simply, the 4G Blacklist provides better protection with fewer lines of code. Even so, each of these blacklists have been updated with hundreds of new directives and will be made available here at Perishable Press in the near future. But for now, let’s return to the business at hand..
Presenting the Perishable Press 4G Blacklist
As is custom here at Perishable Press, I present the complete code first, and then walk through the usage instructions and code explanations. So, without furhter ado, here is the much-anticipated 4G Blacklist [for personal use only – may not be posted elsewhere without proper link attribution]:
### PERISHABLE PRESS 4G BLACKLIST ###
# ESSENTIALS
RewriteEngine on
ServerSignature Off
Options All -Indexes
Options +FollowSymLinks
# FILTER REQUEST METHODS
<IfModule mod_rewrite.c>
RewriteCond %{REQUEST_METHOD} ^(TRACE|DELETE|TRACK) [NC]
RewriteRule ^(.*)$ - [F,L]
</IfModule>
# BLACKLIST CANDIDATES
<Limit GET POST PUT>
Order Allow,Deny
Allow from all
Deny from 75.126.85.215 "# blacklist candidate 2008-01-02 = admin-ajax.php attack "
Deny from 128.111.48.138 "# blacklist candidate 2008-02-10 = cryptic character strings "
Deny from 87.248.163.54 "# blacklist candidate 2008-03-09 = block administrative attacks "
Deny from 84.122.143.99 "# blacklist candidate 2008-04-27 = block clam store loser "
Deny from 210.210.119.145 "# blacklist candidate 2008-05-31 = block _vpi.xml attacks "
Deny from 66.74.199.125 "# blacklist candidate 2008-10-19 = block mindless spider running "
Deny from 203.55.231.100 "# 1048 attacks in 60 minutes"
Deny from 24.19.202.10 "# 1629 attacks in 90 minutes"
</Limit>
# QUERY STRING EXPLOITS
<IfModule mod_rewrite.c>
RewriteCond %{QUERY_STRING} \.\.\/ [NC,OR]
RewriteCond %{QUERY_STRING} boot\.ini [NC,OR]
RewriteCond %{QUERY_STRING} tag\= [NC,OR]
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} http\: [NC,OR]
RewriteCond %{QUERY_STRING} https\: [NC,OR]
RewriteCond %{QUERY_STRING} mosConfig [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(\[|\]|\(|\)|<|>|'|"|;|\?|\*).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(%22|%27|%3C|%3E|%5C|%7B|%7C).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(%0|%A|%B|%C|%D|%E|%F|127\.0).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(globals|encode|config|localhost|loopback).* [NC,OR]
RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]
RewriteRule ^(.*)$ - [F,L]
</IfModule>
# CHARACTER STRINGS
<IfModule mod_alias.c>
# BASIC CHARACTERS
RedirectMatch 403 \,
RedirectMatch 403 \:
RedirectMatch 403 \;
RedirectMatch 403 \=
RedirectMatch 403 \@
RedirectMatch 403 \[
RedirectMatch 403 \]
RedirectMatch 403 \^
RedirectMatch 403 \`
RedirectMatch 403 \{
RedirectMatch 403 \}
RedirectMatch 403 \~
RedirectMatch 403 \"
RedirectMatch 403 \$
RedirectMatch 403 \<
RedirectMatch 403 \>
RedirectMatch 403 \|
RedirectMatch 403 \.\.
RedirectMatch 403 \/\/
RedirectMatch 403 \%0
RedirectMatch 403 \%A
RedirectMatch 403 \%B
RedirectMatch 403 \%C
RedirectMatch 403 \%D
RedirectMatch 403 \%E
RedirectMatch 403 \%F
RedirectMatch 403 \%22
RedirectMatch 403 \%27
RedirectMatch 403 \%28
RedirectMatch 403 \%29
RedirectMatch 403 \%3C
RedirectMatch 403 \%3E
RedirectMatch 403 \%3F
RedirectMatch 403 \%5B
RedirectMatch 403 \%5C
RedirectMatch 403 \%5D
RedirectMatch 403 \%7B
RedirectMatch 403 \%7C
RedirectMatch 403 \%7D
# COMMON PATTERNS
Redirectmatch 403 \_vpi
RedirectMatch 403 \.inc
Redirectmatch 403 xAou6
Redirectmatch 403 db\_name
Redirectmatch 403 select\(
Redirectmatch 403 convert\(
Redirectmatch 403 \/query\/
RedirectMatch 403 ImpEvData
Redirectmatch 403 \.XMLHTTP
Redirectmatch 403 proxydeny
RedirectMatch 403 function\.
Redirectmatch 403 remoteFile
Redirectmatch 403 servername
Redirectmatch 403 \&rptmode\=
Redirectmatch 403 sys\_cpanel
RedirectMatch 403 db\_connect
RedirectMatch 403 doeditconfig
RedirectMatch 403 check\_proxy
Redirectmatch 403 system\_user
Redirectmatch 403 \/\(null\)\/
Redirectmatch 403 clientrequest
Redirectmatch 403 option\_value
RedirectMatch 403 ref\.outcontrol
# SPECIFIC EXPLOITS
RedirectMatch 403 errors\.
RedirectMatch 403 config\.
RedirectMatch 403 include\.
RedirectMatch 403 display\.
RedirectMatch 403 register\.
Redirectmatch 403 password\.
RedirectMatch 403 maincore\.
RedirectMatch 403 authorize\.
Redirectmatch 403 macromates\.
RedirectMatch 403 head\_auth\.
RedirectMatch 403 submit\_links\.
RedirectMatch 403 change\_action\.
Redirectmatch 403 com\_facileforms\/
RedirectMatch 403 admin\_db\_utilities\.
RedirectMatch 403 admin\.webring\.docs\.
Redirectmatch 403 Table\/Latest\/index\.
</IfModule>
That’s the juice right there. This 4G Blacklist is some powerful stuff, blocking and filtering a wide range of potential attacks and eliminating tons of malicious nonsense. Much care has been taken to beta test this firewall on multiple configurations running various types of software, however, due to my limited financial resources, it is impossible to test the 4G as comprehensively as I would have preferred. Even so, for the average site running typical software, everything should continue to work perfectly. With that in mind, please read through the remainder of the article before implementing the 4G Blacklist.
Installation and Usage
Before implementing the 4G Blacklist, ensure that you are equipped with the following system requirements:
- Linux server running Apache
- Enabled Apache module: mod_alias
- Enabled Apache module: mod_rewrite
- Ability to edit your site”s root htaccess file (or)
- Ability to modify Apache’s server configuration file
With these requirements met, copy and paste the entire 4G Blacklist into either the root HTAccess file or the server configuration file ( httpd.conf
). After uploading, visit your site and check proper loading of as many different types of pages as possible. For example, if you are running a blogging platform (such as WordPress), test different page views (single, archive, category, home, etc.), log into and surf the admin pages (plugins, themes, options, posts, etc.), and also check peripheral elements such as individual images, available downloads, and alternate protocols (FTP, HTTPS, etc.).
While the 4G Blacklist is designed to target only the bad guys, the regular expressions used in the list may interfere with legitimate URL or file access. If the directives in the blacklist are blocking a specific URL, the browsing device will display a 403 Forbidden error; similarily, if the blacklist happens to block a file or resource required for some script to function properly, the script (JavaScript, PHP, etc.) may simply stop working. If you experience either of these scenarios after installing the blacklist, don’t panic! Simply check the blocked URL or file, locate the matching blacklist string, and disable the directive by placing a pound sign ( #
) at the beginning of the associated line. Once the correct line is commented out, the blocked URL should load normally. Also, if you do happen to experience any conflicts involving the 4G Blacklist, please leave a comment or contact me directly.
Set for Stun
As my readers know, I am serious about site security. Nothing gets my juices flowing like the thought of chopping up mindless cracker whores into small, square chunks and feeding their still-twitching flesh to a pack of starving mongrels. That’s good times, but unfortunately there are probably laws against stuff like that. So in the meantime, we take steps to secure our sites using the most effective tools at our disposal. There is no one single magic bullet that will keep the unscrupulous bastards from exploiting and damaging your site, but there are many cumulative steps that may be taken to form a solid security strategy. Within this cumulative context, the 4G Blacklist recognizes and immunizes against a broad array of common attack elements, thereby maximizing resources while providing solid defense against malicious attacks.
Many Thanks
A huge “Thank You” to the dedicated people who helped beta test the 4G Blacklist. Your excellent feedback played an instrumental role in the development of this version. Thank you!
Further Reading
For more insight into the mysterious realms of blacklisting, the creation of the Perishable Press Blacklist, and DIY site security in general, check out some of my other articles:
- Eight Ways to Blacklist with Apache’s mod_rewrite
- Blacklist Candidate Series Summary
- How to Block Proxy Servers via htaccess
- 2G Blacklist: Closing the Door on Malicious Attacks
- Series Summary: Building the 3G Blacklist
- Building the Perishable Press 4G Blacklist
Next Up
Next up in the March 2009 Blacklist Series: The Ultimate User-Agent Blacklist. Don’t miss it!
Updates
Since the release of the 4G Blacklist, several users have discovered issues with the following 4G directives.
Joomla
In the query-string section, Joomla users should delete the following patterns:
request
config
[
]
In the character-string section, Joomla users should comment-out or delete the following lines:
RedirectMatch 403 \,
RedirectMatch 403 \;
RedirectMatch 403 config\.
RedirectMatch 403 register\.
WordPress
In the query-string section of the 4G Blacklist, the following changes have been made:
"%3D" character-string has been changed to "%5C"
Likewise, in the character-string section, the following change has been made:
"wp\_" character-string has been removed
And in the request-method filtering section, the following change has been made:
"HEAD" method has been removed
Also, the following changes may be necessary according to which plugins you have installed:
Ozh' Admin Drop Down Menu - remove "drop" from the query-string rules
WordPress' Akismet - remove "config" from the query-string rules
OpenID
OpenID users should read the information in this comment.
SMF
SMF users should read the information in this comment.
vBulletin
vBulletin users should read the information in these comments.
233 responses to “The Perishable Press 4G Blacklist”
Jeff, I implemented your 4G Blacklist code about two days ago, and I’m amazed at the “calmness” in my visitor logs now! I no longer feel besieged by one unruly bot after another. What a relief! I just had to say “Thank you!” Your blacklist is wonderful, and it is so good of you to share it.
All the best,
Deb
Jeff,
I’ve been waiting forever for the 4G! I installed it on my first WordPress blog for testing purposes. Here is what I noticed:
1. If you have the WP plugin “Ozh’ Admin Drop Down Menu” (latest version is 3.1.3.3.7 now), you’ll have to delete “drop” from the last line of the Query Strings Exploits:
RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare|drop).* [NC]
Becomes:
RewriteCond %{QUERY_STRING} ^.*(request|select|insert|union|declare).* [NC]
2. If you use the TinyMCE Advanced plugin (latest version is 3.2 now), the buttons won’t be visible because the css won’t load, because it’s located in a directory
wp_theme
. Comment out the line that says:RedirectMatch 404 wp\_
And a question: WordPress gives a 404 instead of Apache giving a 403, unless I create a custom 403 ErrorDocument. Any idea why?
Thanks Jeff, and I’ll be testing more in the next few days.
I can confirm that the default TinyMCE toolbar buttons also do not show up correctly. Only by removing the 4G code from the .htaccess file will the TinyMCE toolbar be displayed correctly.
I use the 404 Notifier plugin to alert me to any 404s occurring, and I’ve been getting a number 404s generated only when the 4G code is loaded, Unfortunately, I’m not savvy enough to pinpoint the exact cause of these errors, but almost all of the 404s occur at the point of WordPress’ cron check. I’m getting hundreds of 404s — daily — that are related to the cron check.
Another issue that I wonder if it’s related to the 4G blacklist is with one bot repeatedly coming in looking for the 403.shtml file, and a 404 is generated. I don’t *think* this was occurring before I began using the 4G code. Perhaps this is what Tony is referring to.
Even with these few issues, though, I really do appreciate all the work that went into the 4G blacklist. Hopefully, we’ll get the few kinks worked out.
Thanks very much.
Deb, you’re right. I could have sworn that the standard tinyMCE loaded fine when I tried, and I had already cleared my browser’s cache. Now I try again and the standard tinyMCE also doesn’t load okay.
The problem is this address that tinymce calls:
/wp-includes/js/tinymce/themes/advanced/skins/wp_theme/ui.css?ver=20081129
BTW, I used Firebug to pinpoint that. For those interested, Firebug is a Firefox Extension.
@Greg: Thank you — the “
request
” character string has been added to the list of query-string modifications for Joomla.@Andrew: That is interesting indeed. I will look into it and update the 4G with the appropriate location and
IfModule
containers.@Tony: Thanks for the help! I have removed the “wp_” directive from the list and updated the article with information about Ozh’ plugin.
@Deb: I have removed the directive that was preventing TinyMCE from loading. Thanks to Tony for his help with this.
Jeff,
It’s the least I can do!
wp-cron is making
HEAD
requests, so it’s being blocked. What do you think we should do, besides removingHEAD
from the 4G’s Request Methods Filter?From the raw access log:
"HEAD /wp-cron.php?check=561caf12167cb54c25589d71581df596 HTTP/1.0" 403 - "-" "WordPress/2.7.1"
I apologize for repeating the question I asked above, but maybe nobody noticed it:
Wordpress gives a 404 instead of Apache giving a 403, unless I create a custom 403 ErrorDocument. Any idea why?
Thanks!
Yes, I strongly second Tony’s inquiry regarding wp-cron. I’m getting deluged with 404 notifications related to it. These are email alerts I receive from the 404 Notifier plugin. They’re occurring about every one or two minutes.
Also, Tony, what issue were you running into with the Ozh’ Admin Drop Down Menu plugin? I only ask because I’m running the Ozh’ Better Feed plugin. I’m guessing there’s probably not any basically similar code between the two, but I wanted to ask. Thanks.
Deb,
Sorry, I should have been more clear about Ozh’ Admin Drop Down Menu. This plugin calls this address:
/wp-content/plugins/ozh-admin-drop-down-menu/inc/adminmenu.css.php?p=%2Fwp-content%2Fplugins%2Fozh-admin-drop-down-menu%2Finc&i=1&w=1&m=0&c=0&h=0&f=1&g=%23676768&n=0
Since there’s the “drop” word in the query string, it gets a 403 and the whole admin page gets messed up.
And some good news: googlebot accessed my site today and got a 200. That’s the one thing I was worried about the most!
Tony, thanks for the info on the Ozh’ plugin.
I’m glad Google is showing your site as 200.
As clarification, I’ve not encountered any 404 pages when actually visiting my website. The 404 notifications I’ve referred to are all happening behind the scenes. I can also see them showing up in the cPanel “Latest Visitors” module.
I’ve also received some 404s related to accessing files in the wp-content/uploads directory. But every time I’ve gone to the pages containing those files on the website, the images are all there, and there’s no problem on those pages. So I suspect the 404s on those image files are somehow related to the 404 triggered by the wp-cron issue that occurred on those pages.
These are my observations so far. I don’t know all the technically correct terms to use, so I’m trying to relay the info as clearly as I can!
Deb, you’re welcome! Your site is really nice, and the design is great, not cluttered at all. I opened several pages and every single item (css, images, js, etc.) on every page loads flawlessly. I really recommend using Firebug when trying something as serious as the 4G blacklist.
Thanks so much, Tony. I’m honored that you stopped by my website.
As far as Firebug, I do use it, but I usually use it related to experimenting with graphics or layout changes prior to actually modifying the CSS.
Can you describe the steps (maybe just the buttons or the menu sequence) you typically take — without having to spend a lot of time to answer this — to use Firebug to troubleshoot issues such as the ones we’ve encountered here? (I had mainly been looking at the Latest Visitor module in cPanel and noting the error codes that were generated on specific URLs.)
Please don’t take a lot of time to answer this. (In that case, I’ll just have to do a little research on it.) I don’t want to create work for other people!
Thanks!
By the way, Tony. Please tell me it was you who happened to do some “testing” while visiting my site. That’ll make me feel a little better. Thanks!