5G Firewall Beta

Posted on February 9, 2011 in HTAccess, Security by

Update 2012/01/25 — Official release version: 5G Blacklist 2012. The beta version provided in this post is now for reference only.

Updating the 4G Blacklist, the new 5G Firewall is now open for beta testing. The new code is better than ever, providing wider protection with less code and fewer false positives. I’ve had much success with this new firewall, but more testing is needed to ensure maximum compatibility and minimal issues.

At this point, the code has been tested extensively with the following WordPress configurations:

  • Default WordPress installation (no plugins)
  • Current WordPress version 3.0.5 (running plugins1)
  • Older WordPress version 2.3.3 (running plugins2)

The 5G Firewall is the result of many months of meticulous request monitoring, analyses, and testing. With this code, my goal is an easy, plug-n-play security firewall that blocks the maximum volume of malicious requests with a minimum number of false positives. It’s also built with compatibility in mind. The 5G Firewall is fine-tuned3 to WordPress, but the directives are designed for general use and should help any site conserve bandwidth and server resources while protecting against malicious activity.

Beta Testers

Only test this code if you are familiar with .htaccess and comfortable with diagnosing and resolving potential issues. The 5G is currently running at Perishable Press and everything seems to be working great. But there are so many different configurations that beta testing is needed to help ensure maximum compatibility. Please leave any issues/resolutions in the comments section (remember to wrap code in <code> tags).

Disclaimer

The 5G Firewall is provided “as-is”, with the intention of helping site administrators protect their sites against bad requests and other malicious activity. The code is open and free to use and modify only if proper attribution is included (e.g., “5G FIREWALL from PerishablePress.com”. By using this code you assume all risk & responsibility for anything that happens, whether good or bad. In short, use wisely, test thoroughly, don’t sue me.

Learn more..

To learn more about the theory and development of the 5G Firewall, check out my article on constructing the 4G Blacklist. A search for “blacklist” in the sidebar should also return much related information.

5G Firewall Beta

# 5G FIREWALL from PerishablePress.com

# 5G:[QUERY STRINGS]
<IfModule mod_rewrite.c>
 RewriteEngine On
 RewriteBase /
 RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]
 RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]
 RewriteCond %{QUERY_STRING} boot\.ini  [NC,OR]
 RewriteCond %{QUERY_STRING} echo.*kae  [NC,OR]
 RewriteCond %{QUERY_STRING} etc/passwd [NC,OR]
 RewriteCond %{QUERY_STRING} \=\\%27$   [NC,OR]
 RewriteCond %{QUERY_STRING} \=\\\'$    [NC,OR]
 RewriteCond %{QUERY_STRING} \.\./      [NC,OR]
 RewriteCond %{QUERY_STRING} \:         [NC,OR]
 RewriteCond %{QUERY_STRING} \[         [NC,OR]
 RewriteCond %{QUERY_STRING} \]         [NC]
 RewriteRule .* - [F]
</IfModule>

# 5G:[USER AGENTS]
<IfModule mod_setenvif.c>
 SetEnvIfNoCase User-Agent ^$ keep_out
 SetEnvIfNoCase User-Agent (casper|cmsworldmap|diavol|dotbot)   keep_out
 SetEnvIfNoCase User-Agent (flicky|ia_archiver|jakarta|kmccrew) keep_out
 SetEnvIfNoCase User-Agent (libwww|planetwork|pycurl|skygrid)   keep_out
 <Limit GET POST PUT>
  Order Allow,Deny
  Allow from all
  Deny from env=keep_out
 </Limit>
</IfModule>

# 5G:[REQUEST STRINGS]
<IfModule mod_alias.c>
 RedirectMatch 403 (https?|ftp|php)\://
 RedirectMatch 403 /(cgi|https?|ima|ucp)/
 RedirectMatch 403 (\=\\\'|\=\\%27|/\\\'/?|\)\.css\()$
 RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\|)
 RedirectMatch 403 \.(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$
 RedirectMatch 403 /(contac|fpw|install|pingserver|register)\.php
 RedirectMatch 403 (base64|crossdomain|localhost|wwwroot)
 RedirectMatch 403 (eval\(|\_vti\_|\(null\)|echo.*kae)
 RedirectMatch 403 \.well\-known/host\-meta
 RedirectMatch 403 /function\.array\-rand
 RedirectMatch 403 \)\;\$\(this\)\.html\(
 RedirectMatch 403 proc/self/environ
 RedirectMatch 403 msnbot\.htm\)\.\_
 RedirectMatch 403 /ref\.outcontrol
 RedirectMatch 403 com\_cropimage
 RedirectMatch 403 indonesia\.htm
 RedirectMatch 403 \{\$itemURL\}
 RedirectMatch 403 function\(\)
 RedirectMatch 403 labels\.rdf
</IfModule>

1 Tested plugins for WP 3.0.5:

  • Akismet
  • All in One SEO Pack
  • BackWPup
  • Clean Options
  • Feed Count
  • Google XML Sitemaps
  • W3 Total Cache
  • WP-phpMyAdmin
  • Contextual Related Posts
  • Customizable Post Listings
  • Custom Query String Reloaded
  • Edit Author Slug
  • FeedStats
  • Google XML Sitemaps
  • Mass Mail
  • No category parents
  • Pierre’s Wordspew
  • Post Editor Buttons
  • Search Everything
  • Secure WordPress
  • Simple:Press Forum
  • TPC! Memory Usage
  • Use Google Libraries
  • Vote the Post
  • WordPress File Monitor
  • WordPress Ultimate Security
  • WP-phpMyAdmin
  • WP-Polls
  • WP-UserOnline
  • WP Favorite Posts
  • WP Hide Dashboard
  • WP Security Scan
  • WP Socializer
  • WPtouch

2 Tested plugins for WP 2.3.3:

  • AddMySite (AMS)
  • Akismet
  • All in One SEO Pack
  • Authenticate
  • Code Auto Escape
  • Compact Archives
  • Contact Coldform
  • Customizable Post Listings
  • Custom Query String Reloaded
  • Dagon Design Sitemap Generator
  • Display Post View Count (Top10)
  • Download Counter
  • Feedburner Feed Replacement
  • Feed Count
  • Full Text Feed
  • Google XML Sitemaps
  • KillNag
  • Plugins Used Plugin
  • Search Everything
  • Simple Recent Comments
  • Simple Tags
  • SimpleTwitter
  • Stealth Publish
  • Subscribe To Comments
  • Theme Switcher
  • the_excerpt Reloaded
  • Yet Another Related Posts Plugin

3 Test Environment:

  • Operating System: Linux
  • Server: Apache/2.2.3 (CentOS)
  • MYSQL Version: 5.0.77-log
  • PHP Version: 5.2.6

4 Example query strings for testing:

http://example.com/path/?../
http://example.com/path/?php://
http://example.com/path/?scanner
http://example.com/path/?boot.ini
http://example.com/path/?echo.*kae
http://example.com/path/?mosconfig
http://example.com/path/?etc/passwd
http://example.com/path/?path=./
http://example.com/path/?=\'
http://example.com/path/?=\%27
http://example.com/path/?environ
http://example.com/path/?menu=
http://example.com/path/?mod=
http://example.com/path/?tag=
http://example.com/path/?ftp:
http://example.com/path/?http:
http://example.com/path/?https:
http://example.com/path/?[
http://example.com/path/?]
http://example.com/path/?

51 Responses

  1. Neo says:

    Hi Jeff. Thanks for sharing such a useful tool. I have just read your 4G post, and have now given 5G a test.

    I’ve tested this on three WP sites, all work fine without a hitch.

    These include:
    WP 3.0.5 site running Genesis (but no other plugins).
    WP 2.9.2 site running WP-Touch, Lightbox 2, WP-Polls
    WP 3.0.1 site without any plugins

    I hope to be able to test this on my Magento e-Commerce sites over the next couple of days. Will keep you posted.

  2. Berry Sizemore says:

    I have deployed the firewall without modification to my WordPress and VBulletin sites. My Worpress site is nested in the tree of VBulletin. When I put the firewall into .htaccess of ~/public_html/ (for VBulletin) and ~/public_html/wordpress/ (for WordPress), some functionality for WordPress gives 500 errors.

    • Jeff Starr says:

      Hmm, you might try one section of the firewall at a time. The user-agent and query-string directives shouldn’t cause any problems, but the request-pattern rules are a bit trickier.

      If you decide to try again, any specific information to help diagnose the issue would be useful. For example, what stopped working, was there a URL request involved, and any ideas for which rule might be causing the issue would be awesome.

  3. Isiah says:

    Great stuff. I shall try and implement this tonight. My site is a non-Word Press environment so I’m guessing 5G should run without causing any problems.

    Cheers
    I

  4. Isiah says:

    Hmmm…. spoke too soon. Getting an Access Denied error when trying to access any page on my site:

    Forbidden
    You don’t have permission to access / on this server.
    Apache Server at http://www.thedarkfortress.co.uk Port 80

    but I don’t know why.

    • Jeff Starr says:

      A great way to diagnose issues is to just include one piece of the 5G at a time. Then once you’ve narrowed it down to a specific section, remove half of the rules and see if it works. If not, remove a few more, and so on until it works. The last chunk of code you remove before it works should contain the code at issue.

      At least, that’s how I do it ;)

  5. Ale Medina says:

    Hi,

    Quick question: This code is supposed to go on top of the wp default code for permalinks right?

    I’ll start adding it to my sites and will let you know about any news.

    Thanks for sharing such a valuable resource!!

    • Jeff Starr says:

      I always place mine after (beneath) the WP permalink directives. But theoretically they can go anywhere, it shouldn’t matter.

      Also, the 5G Firewall may be placed in subdirectories to protect whatever is contained therein.

  6. Ipstenu says:

    After a year of pondering, I finally threw this down onto two WP 3.1 installs and one Invision Power Board. So far so good! And I’ve got WordPress with BuddyPress and a metric ton of weird plugins/tweaks going on!

    • Jeff Starr says:

      That’s great to hear! It also works great with the gigantic Simple:Press Forum plugin, which has tons of crazy stuff happening. So good signs so far – let us know if you find any surprises.

  7. Skye says:

    Looks good Jeff. Can’t wait to try it out.

  8. the 5G firewalls seems to work on a 3.0.5 installation of WordPress with the active plugins:

    • Akismet
    • ALO EasyMail Newsletter
    • Disable wp new user notification
    • Enhanced Meta Widget
    • Shadowbox JS
    • Slickr Flickr
    • Social Media Widget
    • Theme My Login
    • Tweet This
    • User Photo
    • W3 Total Cache
    • WordPress.com Stats
    • WP-Polls
    • WP Coda Slider
    • XML Sitemap Feed
  9. Doug Smith says:

    I’ve been testing on one of my sites with a good deal of traffic and I’ve only run into one problem so far.

    In the user agent section is a rule blocking requests with a blank user agent:

    SetEnvIfNoCase User-Agent ^$ keep_out

    I happen to be using PayPal on that site for purchases along with PayPal’s IPN feature, which POSTs the transaction details back to a URL you specify. That request has no User-Agent set so no transactions were getting through. Here’s a sample of the request, which can come from any of several PayPal IP addresses:

    66.211.170.66 - - [10/Feb/2011:14:50:21 -0600] "POST /mypath/myipnscript.php HTTP/1.0" 200 - "-" "-"

    So those using PayPal with IPN will want to watch out for that.

    • Jeff Starr says:

      Thanks for the tip. PayPal’s pretty huge – I may end up removing that particular directive to avoid issues involving financial transactions and the like. Hmmm..

  10. Doug Smith says:

    I have one more for you. If you run bbPress along with WordPress, the forum registration will be blocked because it does a URL with

    /forumpath/register.php

    which is matched in

    RedirectMatch 403 /(contac|fpw|install|pingserver|register)\.php

    There are nowhere near the number of bbPress sites as there are WordPress so I imagine this one won’t affect too many. It’s good to be aware of though.

  11. Neo says:

    I have found that a program download is blocked.

    An example link that works with 5G:
    http://www.website.com/wp-content/uploads/2010/06/Program-v4.3.2.zip

    But this one doesn’t work with 5G:
    http://www.website.com/wp-content/uploads/2010/06/Program-v4.3-Setup.exe

    I should have done the right thing and removed the periods from within the file names in the first place (my bad), but thought you’d like to know regardless.

    • Doug Smith says:

      It’s being blocked because of the “.exe”. That’s a good thing because there is so much malicious traffic probing sites for those to find Windows server vulnerabilities. If you really need to deliver an exe file it would be best wrap it in a zip file and take “exe” out of the file name. You could also alter the 5G rules but then you would lose out on blocking all those bad guys.

  12. Ken Smith says:

    Jeff I have found much help here at PP over the last 3 years and just wanted to thank you for making this code and other stuff available. I know it takes “plenty of time” to do this and that time is appreciated. BTW, your new site design looks GREAT!!!

  13. Frances says:

    Hi Jeff, thanks for doing all this work, your site is a huge resource for me.

    I noticed with my setup (WordPress 3.0.5, a stack of plugins, a lot of stuff in functions.php, various bits in htaccess, custom php.ini, your latest robots.txt, blackhole … ) that Navigation Menus in WordPress throws an error when swapping between menus. I traced it in my case to this line:

    RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]

    I removed ‘menu’ and it works again.

  14. Berry Sizemore says:

    Jeff,

    Line 15 is poison to MediaWiki due to page names such as Special:Pages and Random:Page and etc. That’s the colon filter.

    I took all 55 lines of 5G and put it into the appropriate part of my .htaccess… I had it in the wrong place and I had to nuke the 3G stuff.

    Thanks

  15. Berry Sizemore says:

    Jeff,

    I received a report that the DROID Incredible by Verizon is having trouble. Hope this report helps.

    Best,
    Berry

  16. Berry Sizemore says:

    Jeff & Co.,

    VBulletin is affected by this filter as well, though it only shows up in the admin. I haven’t tested, but it may affect registrations too.

    RewriteRule .* - [F]

    • Jeff Starr says:

      Thanks for all the help, Berry :)

      For this particular rule, removing it would eliminate the entire query-string block of directives. That line tells Apache to 403 any request that meets any of the RewriteCond requirements. I’m thinking the issue may be with one of the RewriteCond character patterns..

  17. Berry Sizemore says:

    This line:

    RedirectMatch 403 /(contac|fpw|install|pingserverr|register)\.php

    hoses VBulletin’s registration system. Remove the word “register” from this filter and all is well:

    RedirectMatch 403 /(contac|fpw|install|pingserverr)\.php

  18. Bobby Jones says:

    I have the 5G installed on a basic default install of Joomla 1.6 all appears to be working so far. One issue that seems strange though is when I try any of the test query strings you listed above I don’t get a 403 error instead I get the default Joomla article not found page. I do have SEO friendly URLs enabled in the Admin panel so i’m not sure if this is maybe wahts casuing that to happen. I have joomla installed in a subdirectory on my domain, when i try the example query strings on my sites root directory which only has a static html page I do get a 403 error when using the example query strings.

    To recap when I try the following (http://mysite.com/path/?../) I DO get a 403 error page. However when I try the following (http://mysite.com/myJoomlaDirectory/path/?../) I get the default joomla article not found page, not a server 403 error page. Hope this helps.

  19. Isiah says:

    Something that I’ve found very useful in stopping probing attacks is adding “reply” to this line:

    RedirectMatch 403 /(contac|fpw|install|pingserver|register|setup)\.php

    as I seen to get a lot of rogue request strings ending in xxx/setup.php.

    I admit this might not be sensible to add to a lot of sites – but seems fine on mine so far

  20. verbatim says:

    Jeff, great work. Is this a separate initiative from your “Blackhole”?

    http://perishablepress.com/press/2010/07/14/blackhole-bad-bots/

    • Jeff Starr says:

      Thanks, yes this a different technique entirely, although they share some of the same functional principles. Also, the 5G is built with htaccess, and the Blackhole primarily with PHP.

  21. Jeff Byrnes says:

    Jeff, wondering if you could help out with this one. I’ve got Mint set up with the Birdfeeder Pepper, but after putting the 5G Firewall in place, if someone clicks on a seed, the link is a 404.

    Example:

    http://www.jeffbyrnes.net/feeder/?FeederAction=clicked&amp;feed=Articles%20(RSS2)&amp;seed=http://www.jeffbyrnes.net/2008/11/05/yes-we-did-yes-we-can-yes-we-will/&amp;seed_title=A%20wise%20man%20once%20said…

    I’ve narrowed it down to the query strings rules, but after going through & removing them half at a time, then one-by-one, no love. That link will only work correctly if I hose all the query strings, which kinda defeats the purpose…

    Anyway, just curious if you’ve got any thoughts, I’m sure I’m missing something!

    • Jeff Byrnes says:

      Oh, and I’ve disabled the rules for now, so that link will resolve correctly. What few readers I have subscribed by RSS definitely take precedence over stopping some bots.

    • Jeff Starr says:

      Try removing the following character-string from the first QUERY-STRING directive:

      https?|

      I’m thinking that should work, but there may another match happening somewhere. Let me know, we’ll get it.

      • Jeff Byrnes says:

        The only https?| I see is in one of the “Request Strings” lines, which is a RedirectMatch bit. There’s no https?| in any of the QUERY-STRING directives.

        Regardless, I tried removing both of the ones in the RedirectMatch rules, no dice.

      • Jeff Starr says:

        Right, my mistake – I was looking at the wrong section. Try removing this line from the query-string rules:

        RewriteCond %{QUERY_STRING} \: [NC,OR]

        That may not be it either, but the troubleshooting process is really just a few iterations of the halving method, where you remove half the code, test, remove another chunk of code, etc, until the offending match is identified.

      • Jeff Byrnes says:

        And shazam it worked! Man, I thought I’d gone through each line one-by-one, but obviously I missed that one somehow. And yeah, split-half search is the way to go for things like this, but that didn’t reveal that was the problem either.

        Regardless, seems to be working now, thanks a ton!

  22. aleSub says:

    Hi,

    Just wanted to inform about a bug already reported: switching between menus (Appareance -> Menus) or trying to add a new one won’t work because of this:

    RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]

    The menus section does include a “menu” parameter which is the id of the menu. So far the fix would be to remove the menu condition, I’m not that good on htaccess to provide an exception applied to that single area only, but maybe someone over here can :)

    Thanks for sharing this great resource!

  23. Jon B says:

    This is awesome. I can’t believe how generous you are to the web community. So great.

    Works for me, but had to comment out one section:

    RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\|)

    It was conflicting with W3 Total Cache, giving this error:

    You don't have permission to access /wp-content/w3tc/pgcache/contact//_index.html.gzip on this server.

    Maybe I am doing something wrong in my w3 settings?

    WP 3.1

    • Jeff Starr says:

      It looks like it may just be the double slashes that are matched/blocked. To keep the protection offered by the other patterns in that line, remove the following characters:

      //|

      That should do it, let me know if not so I can look into it. I also use W3TC here at this site along with the 5G and haven’t detected any issues. Do you know what link/action triggered the error?

      • Jon B says:

        That worked great, thank you!

        Basically, when I include the //| in that line, any URLs get blocked, including the homepage of the site. This fix definitely helped.

      • Jon B says:

        Ah – one thing I should have noticed before now – it’s only specific to one of my sites using W3TC, so it doesn’t seem to be a widespread bug at all. Probably something in my settings that I need to resave or revisit.

  24. Ken Dawes says:

    Hi Jeff,
    Great work on 5G!!

    I’ll be trying it out and let you know how it goes!

    Thanks!
    Ken

  25. Ipstenu says:

    Found a bug using it with MediaWiki.

    If there’s a COMMA in the page title, it kicks a 404.

    I pulled out this line and it’s fine, though obviously that’s not a great thing.
    RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\|)

    • Jeff Starr says:

      You could also just remove the first pattern in that line:

      \,|

      That will prevent matches against commas in URLs while enabling the line to protect against the other character strings.

  26. Ipstenu says:

    Unrelated to my wiki thing, I’ve noticed a whole lot of “client denied by server configuration” messages in my error_log.

    Sadly, there was a server upgrade the day or so after I implemented this, so I’ve taken the blacklist OFF one domain just to see if that has an effect. It’s only off on my WordPress only site (very few plugins).

    • Jeff Starr says:

      Those messages mean that the list is working.. Your server configuration includes directives passed by htaccess. Whenever something is blocked by one of the blacklist rules, that is the message you’ll see in your error logs (depending on server/host). Seeing “a whole lot” means it’s working great.

  27. Scott Cariss says:

    Found one thing in the 5g firewall that breaks default wordpress functionality.

    When under Appearence->Menus->Change Menu Selection this URL is generated:

    /wp-admin/nav-menus.php?action=edit&menu=4

    menu in RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR] breaks this.

    • vale says:

      I’ve got the same issue. Does anyone have found the proper regular expression?

    • Jeff Starr says:

      Thanks for the report, Scott. The final version of 5G will remove the matching menu pattern.

      @vale: To resolve the issue, simply remove the following characters from the line mentioned by Scott:

      menu|

      After removing it, the line should look like this:

      RewriteCond %{QUERY_STRING} (mod|path|tag)\=\.?/? [NC,OR]

  28. Seogeeker says:

    Nice thanks for updating this, been using 3g for a long time. 5g works great on the few I have just tried it on!

  29. Kai Gittens says:

    I just installed the 5G Firewall on my .htaccess file. All is working great except for an issue with WP Security Scan v3.0.7. When I try to access the plugin’s Scanner section, I get a permission denied/403 error.

    Under the QUERY STRINGS section, I commented out the following line and all worked great:

    RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]

    I’m assuming that it has to do with the word ‘scanner’ in there but I was just wondering if doing stuff like this was all OK.

    Also, another quick question: If I intsall the full 5G Firewall, is there still a need for the WordPress Firewall 2 plugin?

    Thanx in advance!

  30. Jeff Starr says:

    Hey Kai, thanks for the heads up. That’s totally okay to remove or comment out patterns that interfere with normal functionality. The firewall still works great, just doesn’t block requests that contain that particular string.

    Not sure about the Firewall 2 plugin – running 5G, I haven’t felt the need, although it does provide some additional security.