5G Blacklist 2012
![[ 5G (2012) ]](https://perishablepress.com/wp/wp-content/images/2012/5G-Blacklist-2012.png)
The 5G Blacklist helps reduce the number of malicious URL requests that hit your website. It’s one of many ways to improve the security of your site and protect against evil exploits, bad requests, and other nefarious garbage. If you’re tired of all the slow, bloated security plugins and expensive 3rd-party services, the 5G Blacklist is a solid solution to help protect your Apache-powered site.
Evolution
After extensive beta testing, the 5G Blacklist/Firewall is solid and ready to help secure sites hosted on Apache servers. In addition to beta testing for the 5G, this is the 5th major update of my “G”-series blacklists. Here is a quick overview of its evolution.
- Ultimate htaccess Blacklist (Compressed Version)
- 2G Blacklist: Closing the Door on Malicious Attacks
- Perishable Press 3G Blacklist
- The Perishable Press 4G Blacklist
- 5G Firewall (Beta)
Along the way, I’ve explored a wide variety of different blacklist techniques. The 5G is the culmination of all these efforts, and will eventually be replaced by the imminent 6G Blacklist/Firewall.
What it does
The 5G Blacklist is a simple, flexible blacklist that checks all URI requests against a series of carefully constructed HTAccess directives. This happens quietly behind the scenes at the server level, saving resources for stuff like PHP and MySQL for all blocked requests.
How it works
Blacklists can block just about any part of a request: IP, user agent, request string, query string, referrer, and everything in between. But IP addresses change constantly, and user agents and referrers are easily spoofed. As discussed, request strings yield the best results: greater protection with fewer false positives.
The 5G works beautifully with WordPress, and should help any site conserve bandwidth and server resources while protecting against malicious activity.
How to use
To install the 5G Firewall, append the following code to your site’s root .htaccess:
# 5G BLACKLIST/FIREWALL
# @ https://perishablepress.com/5g-blacklist-2012/
# 5G:[QUERY STRINGS]
<ifModule mod_rewrite.c>
RewriteEngine On
RewriteBase /
RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]
RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]
RewriteCond %{QUERY_STRING} boot\.ini [NC,OR]
RewriteCond %{QUERY_STRING} echo.*kae [NC,OR]
RewriteCond %{QUERY_STRING} etc/passwd [NC,OR]
RewriteCond %{QUERY_STRING} \=\\%27$ [NC,OR]
RewriteCond %{QUERY_STRING} \=\\\'$ [NC,OR]
RewriteCond %{QUERY_STRING} \.\./ [NC,OR]
RewriteCond %{QUERY_STRING} \? [NC,OR]
RewriteCond %{QUERY_STRING} \: [NC,OR]
RewriteCond %{QUERY_STRING} \[ [NC,OR]
RewriteCond %{QUERY_STRING} \] [NC]
RewriteRule .* - [F]
</ifModule>
# 5G:[USER AGENTS]
<ifModule mod_setenvif.c>
SetEnvIfNoCase User-Agent ^$ keep_out
SetEnvIfNoCase User-Agent (casper|cmsworldmap|diavol|dotbot) keep_out
SetEnvIfNoCase User-Agent (flicky|ia_archiver|jakarta|kmccrew) keep_out
SetEnvIfNoCase User-Agent (libwww|planetwork|pycurl|skygrid) keep_out
SetEnvIfNoCase User-Agent (purebot|comodo|feedfinder|turnit) keep_out
SetEnvIfNoCase User-Agent (zmeu|nutch|vikspider|binlar|sucker) keep_out
<limit GET POST PUT>
Order Allow,Deny
Allow from all
Deny from env=keep_out
</limit>
</ifModule>
# 5G:[REQUEST STRINGS]
<ifModule mod_alias.c>
RedirectMatch 403 (https?|ftp|php)\://
RedirectMatch 403 /(cgi|https?|ima|ucp)/
RedirectMatch 403 /(Permanent|Better)$
RedirectMatch 403 (\=\\\'|\=\\%27|/\\\'/?|\)\.css\()$
RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\||\\\"\\\")
RedirectMatch 403 \.(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$
RedirectMatch 403 /(contac|fpw|install|pingserver|register)\.php$
RedirectMatch 403 (base64|crossdomain|localhost|wwwroot|e107\_)
RedirectMatch 403 (eval\(|\_vti\_|\(null\)|echo.*kae|config\.xml)
RedirectMatch 403 \.well\-known/host\-meta
RedirectMatch 403 /function\.array\-rand
RedirectMatch 403 \)\;\$\(this\)\.html\(
RedirectMatch 403 proc/self/environ
RedirectMatch 403 msnbot\.htm\)\.\_
RedirectMatch 403 /ref\.outcontrol
RedirectMatch 403 com\_cropimage
RedirectMatch 403 indonesia\.htm
RedirectMatch 403 \{\$itemURL\}
RedirectMatch 403 function\(\)
RedirectMatch 403 labels\.rdf
RedirectMatch 403 /playing.php
RedirectMatch 403 muieblackcat
</ifModule>
# 5G:[BAD IPS]
<limit GET POST PUT>
Order Allow,Deny
Allow from all
# uncomment/edit/repeat next line to block IPs
# Deny from 123.456.789
</limit>That’s the golden ticket right there. The 5G Firewall is serious protection for your website: extensively tested, plug-n-play, and completely free. “Grab, gulp, n go” as they say. For more information, see the beta article (and comments).
Troubleshooting
Remember, test thoroughly. If something stops working when the 5G is installed, try removing the 5G. If things start working normally again, you can either pass on the 5G or investigate further. Investigating further is straightforward using something like the halving method, where you remove chunks of the 5G until isolating and identifying the issue. Here is a quick example:
- I’ve installed the 5G, thanks Jeff.
- Uh-oh, the page at
http://example.com/indonesia.htmlstopped loading - Hmm, the URL contains the phrase “indonesia”, so let’s check the 5G for it
- Yep, there’s a rule that blocks
indonesia\.htm - Removing that line resolves the issue, thanks me.
Is it okay to remove rules that are blocking your own pages? Yes, the only downside is that malicious requests that would have otherwise been blocked will now get through. The 5G will continue to block a massive volume of malicious requests — it’ll just be a bit less effective. The protective effect is cumulative, not dependent on any one rule. So customization is encouraged. Once you dial it in, you’re all set.
Disclaimer
The 5G Firewall is provided “as-is”, with the intention of helping site administrators protect their sites against bad requests and other malicious activity. The code is open and free to use and modify as long as the first two credit lines remain intact. By using this code you assume all risk & responsibility for anything that happens, whether good or bad. In short, use wisely, test thoroughly, don’t sue me.
Learn more..
To learn more about the theory and development of the 5G Firewall, check out my articles on building the 3G, 4G and 5G Blacklist. A search for “blacklist” in the sidebar should also yield many results.
Happy securing!
223 responses to “5G Blacklist 2012”
Paul,
Websitedefender is running on my site without trouble and I have 5G (plus additions) running.
Hi,
I was wondering if someone might be of help. I’d like to keep as much of this code as I can. I am using wordpress 3.3.1 (not wordpress.org) I am trying to use the nrelate plugin and they need to be able to pull my post images. Is there a way to change the blacklist so that only they can get through..in essence whitelisting them?
I’d appreciate any help at all.
Thanks!
Hi Jeff- I was playing with BuddyPress and I found that after adding this to my HTACCESS it stopped allowing me to accept users to groups in the requests panel of a group admin. I removed the code and it worked again fine. No idea why, but thought I should let you know just in case.
Hello, has anyone been able to get this script to work with Paypal, I also received an email from PayPal “ Please check your server that handles PayPal Instant Payment Notifications
(IPN). IPNs sent to the following URL(s) are failing:”
Please let me know what I need to add to the script to get it to work with Paypal. Thank you.
I think it blocks the addresses containing parameters with dash (-). I cannot get to edit this plugin’s settings and cannot event deactivate it as the link looks like this:
http://domain.tld/wp-admin/plugins.php?action=deactivate&plugin=timthumb-vulnerability-scanner%2Ftimthumb-vulnerability-scanner.php&plugin_status=all&paged=1&s&_wpnonce=05cea08e75I get wordpress “Page not found” message.
I clicked the admin and other links can run ok:
http://domain.tld/wp-admin/plugin-editor.php?file=subscribe-to-comments/subscribe-to-comments.phpI changed the path to plugin files and a long parameter (both containing multiple dashes) and now it works. That seems to be the problem.
Hi,
I have been trying out the 5G and have noticed the following semi-frequent error messages in my error log (on HostMonster)
[Wed Feb 08 16:23:44 2012] [warn] RewriteCond: NoCase option for non-regex pattern '-f' is not supported and will be ignored.[Wed Feb 08 16:24:16 2012] [error] [client 127.0.0.1] (104)Connection reset by peer: ap_content_length_filter: apr_bucket_read() failedUse of uninitialized value in concatenation (.) or string at /usr/local/cpanel/cgi-sys/fourohfour.cgi line 31.[Wed Feb 08 16:24:16 2012] [error] [client 127.0.0.1] (104)Connection reset by peer: ap_content_length_filter: apr_bucket_read() failedUse of uninitialized value in concatenation (.) or string at /usr/local/cpanel/cgi-sys/fourohfour.cgi line 31.Are these errors something I should be commenting out in 5G?
Are they anything that should be added to 5G?
This inquiring mind wants to know! ;-)
In general, I am seeing *way* fewer “errors” from bozos and bots trying to sniff around my site.
Great Work!
Thank you!
Ken
Hey, I got here from watching your security for WordPress video on lynda.com. I have not uploaded my website yet, I’m just about to, I am using bluehost. I put this code in my .htaccess, but certain things don’t work.I figured out it was this line
RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]. I guess because I’m using localhost to access my site right now? After I upload it should I stick this piece of code back in?Also there seems to be so much security I need, In the lynda course wordpress basics they show you how to make a self hosted site and upload it, but they don’t talk much about security, I only found out about it after making my site so I couldn’t do things like change the wp_ rows. If I just did what they showed me in that course and didn’t add security features that I learned in your course, would my wordpress site even be safe on bluehost?
Sorry for so many questions, thanks again!
Thanks!
Had to remove “path” from:
RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]In order for it to work with Joomla! 2.5.2 Administrator back-end that utilized the MissionControl RocketTheme template. PROBLEM: Component “Options” pop-up window would not display.
Had to remove “\,” and “/\,/” strings from:
RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\||\\\"\\\")for Magento “Add to Cart” to function to work properly.
Hi Jeff.
I’m new here. I have been using some great tricks of yours in my blog. And I wanted to thank your tricks.
Also I am writing to ask if the list 5G influences for inbound links from the wall of pinterest. It also influences if you add a link in the description but does not influence the main link with the photo published in pinterest. It is very funny but in my case it returns a 403 if I try to access the link from the wall of my pinterest or any link placed on the description under the photo.
There is another curious thing happened to me installed on my server your 5G list . Since I installed 5G list, Google did not share the thumbnail when I post on Google +.
I have tried to investigate and I cannot find the solution.
If you can solve one of these problems would be very grateful to you.
Best regards.