Perishable Press 3G Blacklist
![[ 3G Stormtroopers ]](http://perishablepress.com/press/wp-content/images/2008/blacklist-3g/blacklist-release.gif)
After much research and discussion, I have developed a concise, lightweight security strategy for Apache-powered websites. Prior to the development of this strategy, I relied on several extensive blacklists to protect my sites against malicious user agents and IP addresses. Over time, these mega-lists became unmanageable and ineffective. As increasing numbers of attacks hit my server, I began developing new techniques for defending against external threats. This work soon culminated in the release of a “next-generation” blacklist that works by targeting common elements of decentralized server attacks. Consisting of a mere 37 lines, this “2G” Blacklist provided enough protection to enable me to completely eliminate over 350 blacklisting directives from my site’s root htaccess file. This improvement increased site performance and decreased attack rates, however many bad hits were still getting through. More work was needed..
The 3G Blacklist
Work on the 3G Blacklist required several weeks of research, testing, and analysis. During the development process, five major improvements were discovered, documented, and implemented. Using pattern recognition, access immunization, and multiple layers of protection, the 3G Blacklist serves as an extremely effective security strategy for preventing a vast majority of common exploits. The list consists of four distinct parts, providing multiple layers of protection while synergizing into a comprehensive defense mechanism. Further, as discussed in previous articles, the 3G Blacklist is designed to be as lightweight and flexible as possible, thereby facilitating periodic cultivation and maintenance. Sound good? Here it is:
# PERISHABLE PRESS 3G BLACKLIST
# PART I: CHARACTER STRINGS
<IfModule mod_alias.c>
RedirectMatch 403 \:
RedirectMatch 403 \;
RedirectMatch 403 \<
RedirectMatch 403 \>
RedirectMatch 403 \/\,
RedirectMatch 403 \/\/
RedirectMatch 403 f\-\.
RedirectMatch 403 \.\.\.
RedirectMatch 403 \.inc
RedirectMatch 403 alt\=
RedirectMatch 403 ftp\:
RedirectMatch 403 ttp\:
RedirectMatch 403 \.\$url
RedirectMatch 403 \/\$url
RedirectMatch 403 \/\$link
RedirectMatch 403 news\.php
RedirectMatch 403 menu\.php
RedirectMatch 403 main\.php
RedirectMatch 403 home\.php
RedirectMatch 403 view\.php
RedirectMatch 403 about\.php
RedirectMatch 403 blank\.php
RedirectMatch 403 block\.php
RedirectMatch 403 order\.php
RedirectMatch 403 search\.php
RedirectMatch 403 errors\.php
RedirectMatch 403 button\.php
RedirectMatch 403 middle\.php
RedirectMatch 403 threads\.php
RedirectMatch 403 contact\.php
RedirectMatch 403 include\.php
RedirectMatch 403 display\.php
RedirectMatch 403 register\.php
RedirectMatch 403 authorize\.php
RedirectMatch 403 \/wp\-signup\.php
RedirectMatch 403 \/classes\/
RedirectMatch 403 \/includes\/
RedirectMatch 403 \/path\_to\_script\/
RedirectMatch 403 ImpEvData\.
RedirectMatch 403 head\_auth\.
RedirectMatch 403 db\_connect\.
RedirectMatch 403 check\_proxy\.
RedirectMatch 403 doeditconfig\.
RedirectMatch 403 submit\_links\.
RedirectMatch 403 change\_action\.
RedirectMatch 403 send\_reminders\.
RedirectMatch 403 comment\-template\.
RedirectMatch 403 syntax\_highlight\.
RedirectMatch 403 admin\_db\_utilities\.
RedirectMatch 403 admin\.webring\.docs\.
RedirectMatch 403 function\.main
RedirectMatch 403 function\.mkdir
RedirectMatch 403 function\.opendir
RedirectMatch 403 function\.require
RedirectMatch 403 function\.array\-rand
RedirectMatch 403 ref\.outcontrol
</IfModule>
# PART II: QUERY STRINGS
<ifmodule mod_rewrite.c>
RewriteCond %{QUERY_STRING} ftp\: [NC,OR]
RewriteCond %{QUERY_STRING} http\: [NC,OR]
RewriteCond %{QUERY_STRING} https\: [NC,OR]
RewriteCond %{QUERY_STRING} \[ [NC,OR]
RewriteCond %{QUERY_STRING} \] [NC]
RewriteRule .* - [F,L]
</ifmodule>
# PART III: USER AGENTS
SetEnvIfNoCase User-Agent "Jakarta Commons" keep_out
SetEnvIfNoCase User-Agent "Y!OASIS/TEST" keep_out
SetEnvIfNoCase User-Agent "libwww-perl" keep_out
SetEnvIfNoCase User-Agent "MOT-MPx220" keep_out
SetEnvIfNoCase User-Agent "MJ12bot" keep_out
SetEnvIfNoCase User-Agent "Nutch" keep_out
SetEnvIfNoCase User-Agent "cr4nk" keep_out
<Limit GET POST PUT>
order allow,deny
allow from all
deny from env=keep_out
</Limit>
# PART IV: IP ADDRESSES
<Limit GET POST PUT>
order allow,deny
allow from all
deny from 75.126.85.215 "# blacklist candidate 2008-01-02 = admin-ajax.php attack "
deny from 128.111.48.138 "# blacklist candidate 2008-02-10 = cryptic character strings "
deny from 87.248.163.54 "# blacklist candidate 2008-03-09 = block administrative attacks "
deny from 84.122.143.99 "# blacklist candidate 2008-04-27 = block clam store loser "
</Limit>
Installation and Usage
Before using the 3G Blacklist, check the following system requirements:
- Linux server running Apache
- Enabled Apache module:
mod_alias - Enabled Apache module:
mod_rewrite - Ability to edit your site’s root htaccess file (or)
- Ability to modify Apache’s server configuration file
With these requirements met, copy and paste the entire 3G Blacklist into either the root htaccess file or the server configuration file. After uploading, visit your site and check proper loading of as many different types of pages as possible. For example, if you are running a blogging platform (such as WordPress), test different page views (single, archive, category, home, etc.), log into and surf the admin pages (plugins, themes, options, posts, etc.), and also check peripheral elements such as individual images, available downloads, and alternate protocols (FTP, HTTPS, etc.).
While the 3G Blacklist is designed to target only the bad guys, the regular expressions used in the list may interfere with legitimate URL access. If this happens, the browsing device will display a 403 Forbidden error. Don’t panic! Simply check the blocked URL, locate the matching blacklist string, and disable the directive by placing a pound sign ( # ) at the beginning of the associated line. Once the correct line is commented out, the blocked URL should load normally. Also, if you do happen to experience any conflicts involving the 3G Blacklist, please leave a comment or contact me directly. Thank you :)
Wrap Up..
As my readers know, I am serious about site security. Nothing gets my adrenaline pumping more than the thought of a giant meat grinder squirting out endless chunks of mangled cracker meat. Spam and other exploitative activity on the web has grown exponentially. Targeting and blocking individual agents and IP is no longer a viable strategy. By recognizing and immunizing against the broadest array of common attack elements, the 3G Blacklist maximizes resources while providing solid defense against malicious attacks.
Updates
- 2008/05/14 — removed “
RedirectMatch 403 \/scripts\/” from the first part of the blacklist due to conflict with Mint Statistics. - 2008/05/18 — removed the following three directives to facilitate Joomla functionality (hat tip: Don):
RedirectMatch 403 \/modules\/RedirectMatch 403 \/components\/RedirectMatch 403 \/administrator\/ - 2008/05/31 — removed “
RedirectMatch 403 config\.php” from the first part of the list to ensure proper functionality with the “visual-editing” feature of the WordPress Admin (hat tip: Sat).
Related articles
- Building the 3G Blacklist, Part 3: Improving Site Security by Selectively Blocking Rogue User Agents
- Building the 3G Blacklist, Part 4: Improving the RedirectMatch Directives of the Original 2G Blacklist
- Series Summary: Building the 3G Blacklist
- Ultimate htaccess Blacklist
- Building the 3G Blacklist, Part 2: Improving Site Security by Preventing Malicious Query-String Exploits
- Building the 3G Blacklist, Part 5: Improving Site Security by Selectively Blocking Individual IPs
- Building the 3G Blacklist, Part 1: Improving Site Security by Recognizing and Exploiting Server Attack Patterns
About this article
This is article #545, posted by Jeff Starr on Tuesday, May 13, 2008 @ 11:06pm. Categorized as Websites, and tagged with apache, blacklist, htaccess, mod_rewrite, security, tips. Updated on May 31, 2008. Visited 13084 times. 23 Responses »
Bookmark • Trackback • Comment • Subscribe • Explore
« Building the 3G Blacklist, Part 5: Improving Site Security by Selectively Blocking Individual IPs • Up • WordPress Tip: Update Email Address in the WordPress Database »
1 • May 14, 2008 at 3:55 am — Louis says:
This is solid gold :)
I’ve had no issue so far with the whole list past into my root .htaccess. I’ll see if it affects the administration of my blog in the futur.
Really great stuff as usual from you. The list is clean and comprehensive.
The formatting syntax with the little spaces before Redirect directives is a little strange; is it your new favorite syntax ? :d