5G Blacklist 2012

♦ Posted by Jeff Starr in .htaccess, Security

Updated July 19, 2024 • 223 comments

[ 5G (2012) ] The 5G Blacklist helps reduce the number of malicious URL requests that hit your website. It’s one of many ways to improve the security of your site and protect against evil exploits, bad requests, and other nefarious garbage. If you’re tired of all the slow, bloated security plugins and expensive 3rd-party services, the 5G Blacklist is a solid solution to help protect your Apache-powered site.

Update: Check out the new and improved 6G Firewall »

Evolution

After extensive beta testing, the 5G Blacklist/Firewall is solid and ready to help secure sites hosted on Apache servers. In addition to beta testing for the 5G, this is the 5th major update of my “G”-series blacklists. Here is a quick overview of its evolution.

Along the way, I’ve explored a wide variety of different blacklist techniques. The 5G is the culmination of all these efforts, and will eventually be replaced by the imminent 6G Blacklist/Firewall.

What it does

The 5G Blacklist is a simple, flexible blacklist that checks all URI requests against a series of carefully constructed HTAccess directives. This happens quietly behind the scenes at the server level, saving resources for stuff like PHP and MySQL for all blocked requests.

How it works

Blacklists can block just about any part of a request: IP, user agent, request string, query string, referrer, and everything in between. But IP addresses change constantly, and user agents and referrers are easily spoofed. As discussed, request strings yield the best results: greater protection with fewer false positives.

The 5G works beautifully with WordPress, and should help any site conserve bandwidth and server resources while protecting against malicious activity.

How to use

To install the 5G Firewall, append the following code to your site’s root .htaccess:

# 5G BLACKLIST/FIREWALL
# @ https://perishablepress.com/5g-blacklist-2012/

# 5G:[QUERY STRINGS]
<ifModule mod_rewrite.c>
 RewriteEngine On
 RewriteBase /
 RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]
 RewriteCond %{QUERY_STRING} (menu|mod|path|tag)\=\.?/? [NC,OR]
 RewriteCond %{QUERY_STRING} boot\.ini  [NC,OR]
 RewriteCond %{QUERY_STRING} echo.*kae  [NC,OR]
 RewriteCond %{QUERY_STRING} etc/passwd [NC,OR]
 RewriteCond %{QUERY_STRING} \=\\%27$   [NC,OR]
 RewriteCond %{QUERY_STRING} \=\\\'$    [NC,OR]
 RewriteCond %{QUERY_STRING} \.\./      [NC,OR]
 RewriteCond %{QUERY_STRING} \?         [NC,OR]
 RewriteCond %{QUERY_STRING} \:         [NC,OR]
 RewriteCond %{QUERY_STRING} \[         [NC,OR]
 RewriteCond %{QUERY_STRING} \]         [NC]
 RewriteRule .* - [F]
</ifModule>

# 5G:[USER AGENTS]
<ifModule mod_setenvif.c>
 SetEnvIfNoCase User-Agent ^$ keep_out
 SetEnvIfNoCase User-Agent (casper|cmsworldmap|diavol|dotbot)   keep_out
 SetEnvIfNoCase User-Agent (flicky|ia_archiver|jakarta|kmccrew) keep_out
 SetEnvIfNoCase User-Agent (libwww|planetwork|pycurl|skygrid)   keep_out
 SetEnvIfNoCase User-Agent (purebot|comodo|feedfinder|turnit)   keep_out
 SetEnvIfNoCase User-Agent (zmeu|nutch|vikspider|binlar|sucker) keep_out
 <limit GET POST PUT>
  Order Allow,Deny
  Allow from all
  Deny from env=keep_out
 </limit>
</ifModule>

# 5G:[REQUEST STRINGS]
<ifModule mod_alias.c>
 RedirectMatch 403 (https?|ftp|php)\://
 RedirectMatch 403 /(cgi|https?|ima|ucp)/
 RedirectMatch 403 /(Permanent|Better)$
 RedirectMatch 403 (\=\\\'|\=\\%27|/\\\'/?|\)\.css\()$
 RedirectMatch 403 (\,|//|\)\+|/\,/|\{0\}|\(/\(|\.\.\.|\+\+\+|\||\\\"\\\")
 RedirectMatch 403 \.(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$
 RedirectMatch 403 /(contac|fpw|install|pingserver|register)\.php$
 RedirectMatch 403 (base64|crossdomain|localhost|wwwroot|e107\_)
 RedirectMatch 403 (eval\(|\_vti\_|\(null\)|echo.*kae|config\.xml)
 RedirectMatch 403 \.well\-known/host\-meta
 RedirectMatch 403 /function\.array\-rand
 RedirectMatch 403 \)\;\$\(this\)\.html\(
 RedirectMatch 403 proc/self/environ
 RedirectMatch 403 msnbot\.htm\)\.\_
 RedirectMatch 403 /ref\.outcontrol
 RedirectMatch 403 com\_cropimage
 RedirectMatch 403 indonesia\.htm
 RedirectMatch 403 \{\$itemURL\}
 RedirectMatch 403 function\(\)
 RedirectMatch 403 labels\.rdf
 RedirectMatch 403 /playing.php
 RedirectMatch 403 muieblackcat
</ifModule>

# 5G:[BAD IPS]
<limit GET POST PUT>
 Order Allow,Deny
 Allow from all
 # uncomment/edit/repeat next line to block IPs
 # Deny from 123.456.789
</limit>

That’s the golden ticket right there. The 5G Firewall is serious protection for your website: extensively tested, plug-n-play, and completely free. “Grab, gulp, n go” as they say. For more information, see the beta article (and comments).

Troubleshooting

Remember, test thoroughly. If something stops working when the 5G is installed, try removing the 5G. If things start working normally again, you can either pass on the 5G or investigate further. Investigating further is straightforward using something like the halving method, where you remove chunks of the 5G until isolating and identifying the issue. Here is a quick example:

I’ve installed the 5G, thanks Jeff.
Uh-oh, the page at http://example.com/indonesia.html stopped loading
Hmm, the URL contains the phrase “indonesia”, so let’s check the 5G for it
Yep, there’s a rule that blocks indonesia\.htm
Removing that line resolves the issue, thanks me.

Is it okay to remove rules that are blocking your own pages? Yes, the only downside is that malicious requests that would have otherwise been blocked will now get through. The 5G will continue to block a massive volume of malicious requests — it’ll just be a bit less effective. The protective effect is cumulative, not dependent on any one rule. So customization is encouraged. Once you dial it in, you’re all set.

Disclaimer

The 5G Firewall is provided “as-is”, with the intention of helping site administrators protect their sites against bad requests and other malicious activity. The code is open and free to use and modify as long as the first two credit lines remain intact. By using this code you assume all risk & responsibility for anything that happens, whether good or bad. In short, use wisely, test thoroughly, don’t sue me.

Learn more..

To learn more about the theory and development of the 5G Firewall, check out my articles on building the 3G, 4G and 5G Blacklist. A search for “blacklist” in the sidebar should also yield many results.

Happy securing!

About the Author

Jeff Starr = Creative thinker. Passionate about free and open Web.

223 responses to “5G Blacklist 2012”

frances 2012/03/30 6:03 am

Hi Jeff,

I came across one issue which might be server-specific, with WordPress Database Backup on Strato.

The DB fails to backup unless the following IfModule line is commented out:

RewriteCond %{QUERY_STRING} \:

I haven’t tested this on my usual host.

jose 2012/04/10 8:10 am

I get a lot of attack attempts targeting the “admin” user (which I don’t have, by the way). Every time I get the IP address where it originated I add it to the BAD IP list, but it keep growing… do I need to worry and keep adding them?

Jeff Starr 2012/04/10 8:57 am • Post Author

Blocking IPs is useful for specific cases, but in general it’s just not worthwhile. A better method would be to block the request string they are going after, which in this case looks like admin or similar. You can set it up to block any admin requests or just block any that aren’t from your machine, location, etc.
- jose 2012/04/10 9:05 am
  
  Is this the way to do this:
  
  Adding RedirectMatch 403 admin at the end of [REQUEST STRINGS] section?
- Jeff Starr 2012/04/10 9:29 am • Post Author
  
  Yes that should do it, but you may want to be more specific with additional URL info (eg, /wp-admin/ or similar), that will help to reduce the number of false positives, just in case. And test thoroughly before calling it good ;)

jose 2012/04/10 11:23 am

mmm, how do I do that? I’m honestly not familiar with .htaccess syntax

Jeff Starr 2012/04/10 1:19 pm • Post Author

Here is an even better approach, I think, whereby all admin/login requests are redirected to the actual admin/login page:

<IfModule mod_rewrite.c>
RewriteCond %{REQUEST_URI} !^/wp-admin/ [NC]
RewriteCond %{REQUEST_URI} !^/wp-login.php [NC]
RewriteCond %{REQUEST_URI} (wp\-login|login)\.php [NC,OR]
RewriteCond %{REQUEST_URI} /(wp\-admin|admin|wp\-login|login)/?$ [NC]
RewriteRule .* http://example.com/wp-login.php [R=301,L]
</IfModule>

Note: this code assumes you have WP installed in the root directory. You should only need to edit the “example.com” with your actual domain name. Also, if you use this method, remove any “admin” or “login” rules from the 5G.

If you just want to block them instead of redirecting them, replace the RewriteRule line with this one:

RewriteRule .* - [F]

Remember to test thoroughly :)

Buddy-R 2012/04/13 8:31 pm

@ Jose

I cannot agree more with Jeff!

More and more people are using shared Ip’s (like VPN for example) and if you ban one IP because of 1 user messing (but passing by), you might block hundreds of other faithful users.
Hackers are moving from 1 address to the other and it is difficult to keep up an IP blacklist.

I would suggest you add to Jeff’s awesome firewall a plugin like “Limit Login Attempt” and lockout users for 45 minutes after 3 wrong tries.

With both the 5G Blacklist firewall and the plugin your site will be VERY well protected.
Congrats to Jeff’s always great advises and scripts!

Cheers!
Buddy

Steve 2012/04/20 7:03 am

I left this comment on the 5G Beta page and never got a response. I just tested it on this released version, and have the same symptom:

My main dir .htaccess file starts with “Options -Indexes” to block file listing by default in all directories. Works great.

But I have a few sub-dirs for which I want the file listing to work. In those dirs I add a simple .htaccess file that starts with “Options +Indexes”. Works great until I add the 5G Blacklist.

This line in the [REQUEST STRINGS] section breaks it, causing a 403 error whenever someone tries to navigate to one of those dirs:

RedirectMatch 403 .(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$

If I remove “cgi|” from the line, the problem goes away.

Am I breaking anything or doing myself any harm by removing the cgi thingy?

Jeff Starr 2012/04/20 8:22 am • Post Author

Removing the cgi match is fine, even encouraged if it’s causing issues. The 5G will continue to block many bad requests, but those that target “cgi” stuff will not be blocked. So it’s a trade-off, but not a huge sacrifice.
- Steve 2012/04/20 8:41 am
  
  Thanks.
  
  And a big thanks for all you do.

Ken Dawes 2012/04/26 8:36 am

Hi Jeff,
I assume there would be no changes necessary to 5g if you had WordPress installed to a folder, but did the “run WP from a folder” thing of having the .htaccess and index.php files in the root.

However, what if WordPress was installed to a folder and run from that folder? i.e. www.xyz.com/blog/ – a blog attached to a regular html site?

Or would it be still be put 5g in the root and be done with it?

Thanks for the hard work in coming up with this gem!

Jeff Starr 2012/04/26 11:31 am • Post Author

Or would it be still be put 5g in the root and be done with it?

Exactly that, as .htaccess rules are applied recursively down the directory cascade, so put it in root and everything is covered. If 5G is placed only in a subdirectory, that’s fine too, but only it’ll only block requests made to URLs within that directory.

Ken Dawes 2012/04/26 8:47 am

I wanted to put this in a separate note from my previous one.

There are a number of “specific” edits to 5g to allow certain things not to be blocked in WordPress.

For instance, I have found that the Query String
RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]
needed “scanner” removed in order for the “TimThumb Vulnerability Scanner” plugin to be able to be updated

I’m thinking that it would be helpful for those of us using 5g to have a repository somewhere of the “exceptions” people have found. If could save a lot of time over the “half and then half again…” search for the 5g value that’s problematic.

Just a thought.

Jeff Starr 2012/04/26 11:38 am • Post Author

I think it’s a great idea, and something I’ve tried to do with each release by updating the article according to the feedback received via comments, email, twitter, wherever. It would be great to streamline the process and funnel all of it to an official exceptions page.. thanks for the idea :)

David Radovanovic 2012/05/01 7:13 am

Is there any possibility that legitimate traffic especially search engine bots will be deterred or some way depreciate SEO or rankings? I just got hit with numerous malicious attacks and the Pharma Hack. I only knew about it when Google sent me the dreaded “unnatural links” notice.

Jeff Starr 2012/05/01 9:20 am • Post Author

Absolutely, when Google has you in their crosshairs, both SEO rank and legit traffic can suffer. Eliminating the hacks quickly is essential. Good luck.

Regina Smola 2012/05/01 10:11 am

Awesome info Jeff! Testing it on a couple of WordPress sites now.

I have a client that has WP in the root and VBulletin in a subdir. I have copied the 5G Blacklist 2012 and the WordPress Add-on for 5G Blacklist in the root .htaccess. There is also a .htaccess in the VB directory with some simple security settings. Will the settings still re-curse into the forum and do you have any additional suggestions for blacklisting VB?

Jeff Starr 2012/05/01 10:56 am • Post Author

My pleasure :)

Yes the blacklist rules (and other .htaccess rules) should be applied recursively thru the directory structure. You can test by appending any of the blocked terms to any URL and it should return a 403 Forbidden error. If you can provide some examples of the types of VB requests that you would like to block, I can investigate and possibly integrate into the upcoming 6G Blacklist. You can also test the strings directly using the 6G Dev page.
- Regina Smola 2012/05/01 11:36 am
  
  Thanks Jeff.
  
  Actually I am seeing an issue with IP address 66.211.170.66 and 173.0.81.1, which belong to PayPal. The ips was denied by the server after adding 5G when trying to reach payment_gateway.php.
  
  Any ideas how to accept those IPs or to make that page not blocked?
- Jeff Starr 2012/05/01 12:03 pm • Post Author
  
  I’m not seeing anything in the 5G that would block the string, payment_gateway.php.. perhaps there is a query string associated with such requests? If so, that could be it, as the 5G blocks a variety of potentially malicious query-strings.
Caroline 2012/12/30 9:26 am

Hi Jeff,

I hate to waste your time if this has already been addressed. I looked through your comments and don’t see it addressed. So…

Something in 5G blocks PayPal IPNs for people processing PayPal payments on their site. We got a message like this from PayPal:

IPNs sent to the following URL(s) are failing: http://domain-name.com/?paypalListener=paypal_standard_IPN

This causes ecommerce (I’m using Woocommerce) orders to stay marked as “pending” rather than “processing” (no notification is coming in from PayPal to adjust the status of the order to “paid/processing”). This is a common problem I’m seeing in forums on the ‘Net, but so far nobody else has linked it to 5G. I have. Since I don’t have a sandbox and don’t get enough PayPal orders to make the halving method go quickly, I thought I’d post here meanwhile to share my (pretty much confirmed since I deleted 5G and IPN worked again) hunch. I suspect it’s a query string but don’t know which yet.
Thank you.
- Jeff Starr 2012/12/31 3:57 pm • Post Author
  
  Hi Caroline, this is most likely due to PayPal’s use of a “blank user agent” for certain requests. To fix, comment out the following line (in 5G:[USER AGENTS] section):
  
  SetEnvIfNoCase User-Agent ^$ keep_out
  
  Note that this rule is already excluded in the next version of the blacklist (6G).

David Radovanovic 2012/05/01 12:24 pm

I should have rephrased my question: will any of the rules you have shown inadvertently block search engine bots/crawlers?

Jeff Starr 2012/05/01 12:46 pm • Post Author
That’s a great question, and always important to consider when blocking any sort of traffic. In general, blacklists use regular expressions to block potential bad requests. Any string that you are matching has the potential to block both good and bad requests. But there are known patterns and strings that are almost always used for exploit scanning and other malicious behavior. Those are the ones I go after with rules in the 5G et al. Careful testing and analysis is required to keep false positives at a minimum, whether the requests are search-engine or otherwise. I think I have achieved this with the g-series (and other) blacklists, but can’t say that it’s 100% accurate. I can say the 5G is widely used by people who let me know when issues arise. To date, no reports of major search-engine blockage.

There’s much more to it, of course, but that’s some of the thinking behind it. I’ve written extensively on this topic if you’re interested, here are two immediately relevant articles with more info:
- https://perishablepress.com/building-the-5g-blacklist/
- https://perishablepress.com/building-the-perishable-press-4g-blacklist/
I hope this helps!

William 2012/05/05 12:26 pm

the line:

RedirectMatch 403 \.(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$

caused all sorts of problems for me.

I have WP in the webroot, with a subfolder and FancyIndexes enabled for a subroot. This is also password protected.

I have just removed that line.

Jeff Starr 2012/05/05 2:01 pm • Post Author

Thanks for the feedback, William. That line is aimed at blocking requests for various types of files, basically matching any requests for any of those types of files (e.g., .cgi, .asp, etc.). The patterns are very specific and should only block strings that end with one of the listed file types. It shouldn’t be interfering with any other type of request, afaik. That said, the other rules in the 5G will continue to work at protecting your site. Glad to hear you got it resolved.

Cristian Castellari 2012/05/07 1:46 am

Great list, but in my WP installation i’m having problems. When i’m writing a post, and i click on the Preview a first time, i can see the preview correctly.

Any further time, i get a 403 :(

Can you help me in this issue?

Jeff Starr 2012/05/07 10:23 am • Post Author

Hi Cristian,

Try removing the 5G code and see if that helps. If so, try just adding one of the modules, say the request strings section, and see if the problem returns. If it does, then repeat the process by “halving” the code until you locate the offending match.

If the problem persists after removing the 5G, it’s probably something else causing the issue.
- Cristian Castellari 2012/05/07 10:28 am
  
  The problem is related to 5G code. Without it, the Preview works very well as the rest of the years i’ve blogged on this site :D
  
  Tomorrow morning i’ll do some tests :)
  
  I’ve enabled your blackhole script too (i wrote an email about a question to the code i add in wp-load.php), but it has no responsabilties.
- Jeff Starr 2012/05/07 10:39 am • Post Author
  
  Sounds good. I only raise doubts because I use 5G here at Perishable Press and Preview seems to be working properly.. could be something though :)
- brasofilo 2012/07/11 2:32 pm
  
  I had the same problem and believe this is the offending line:
  
  # RewriteCond %{QUERY_STRING} ? [NC,OR]
  # provoking 404 in Post/Page previews

Comments are closed for this post. Something to add? Let me know.

frances 2012/03/30 6:03 am

Hi Jeff,

I came across one issue which might be server-specific, with WordPress Database Backup on Strato.

The DB fails to backup unless the following IfModule line is commented out:

RewriteCond %{QUERY_STRING} \:

I haven’t tested this on my usual host.
jose 2012/04/10 8:10 am

I get a lot of attack attempts targeting the “admin” user (which I don’t have, by the way). Every time I get the IP address where it originated I add it to the BAD IP list, but it keep growing… do I need to worry and keep adding them?
- Jeff Starr 2012/04/10 8:57 am • Post Author
  
  Blocking IPs is useful for specific cases, but in general it’s just not worthwhile. A better method would be to block the request string they are going after, which in this case looks like admin or similar. You can set it up to block any admin requests or just block any that aren’t from your machine, location, etc.
  - jose 2012/04/10 9:05 am
    
    Is this the way to do this:
    
    Adding RedirectMatch 403 admin at the end of [REQUEST STRINGS] section?
  - Jeff Starr 2012/04/10 9:29 am • Post Author
    
    Yes that should do it, but you may want to be more specific with additional URL info (eg, /wp-admin/ or similar), that will help to reduce the number of false positives, just in case. And test thoroughly before calling it good ;)
jose 2012/04/10 11:23 am

mmm, how do I do that? I’m honestly not familiar with .htaccess syntax
- Jeff Starr 2012/04/10 1:19 pm • Post Author
  
  Here is an even better approach, I think, whereby all admin/login requests are redirected to the actual admin/login page:
  
  <IfModule mod_rewrite.c>
  RewriteCond %{REQUEST_URI} !^/wp-admin/ [NC]
  RewriteCond %{REQUEST_URI} !^/wp-login.php [NC]
  RewriteCond %{REQUEST_URI} (wp\-login|login)\.php [NC,OR]
  RewriteCond %{REQUEST_URI} /(wp\-admin|admin|wp\-login|login)/?$ [NC]
  RewriteRule .* http://example.com/wp-login.php [R=301,L]
  </IfModule>
  
  Note: this code assumes you have WP installed in the root directory. You should only need to edit the “example.com” with your actual domain name. Also, if you use this method, remove any “admin” or “login” rules from the 5G.
  
  If you just want to block them instead of redirecting them, replace the RewriteRule line with this one:
  
  RewriteRule .* - [F]
  
  Remember to test thoroughly :)
Buddy-R 2012/04/13 8:31 pm

@ Jose

I cannot agree more with Jeff!

More and more people are using shared Ip’s (like VPN for example) and if you ban one IP because of 1 user messing (but passing by), you might block hundreds of other faithful users.
Hackers are moving from 1 address to the other and it is difficult to keep up an IP blacklist.

I would suggest you add to Jeff’s awesome firewall a plugin like “Limit Login Attempt” and lockout users for 45 minutes after 3 wrong tries.

With both the 5G Blacklist firewall and the plugin your site will be VERY well protected.
Congrats to Jeff’s always great advises and scripts!

Cheers!
Buddy
Steve 2012/04/20 7:03 am

I left this comment on the 5G Beta page and never got a response. I just tested it on this released version, and have the same symptom:

My main dir .htaccess file starts with “Options -Indexes” to block file listing by default in all directories. Works great.

But I have a few sub-dirs for which I want the file listing to work. In those dirs I add a simple .htaccess file that starts with “Options +Indexes”. Works great until I add the 5G Blacklist.

This line in the [REQUEST STRINGS] section breaks it, causing a 403 error whenever someone tries to navigate to one of those dirs:

RedirectMatch 403 .(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$

If I remove “cgi|” from the line, the problem goes away.

Am I breaking anything or doing myself any harm by removing the cgi thingy?
- Jeff Starr 2012/04/20 8:22 am • Post Author
  
  Removing the cgi match is fine, even encouraged if it’s causing issues. The 5G will continue to block many bad requests, but those that target “cgi” stuff will not be blocked. So it’s a trade-off, but not a huge sacrifice.
  - Steve 2012/04/20 8:41 am
    
    Thanks.
    
    And a big thanks for all you do.
Ken Dawes 2012/04/26 8:36 am

Hi Jeff,
I assume there would be no changes necessary to 5g if you had WordPress installed to a folder, but did the “run WP from a folder” thing of having the .htaccess and index.php files in the root.

However, what if WordPress was installed to a folder and run from that folder? i.e. www.xyz.com/blog/ – a blog attached to a regular html site?

Or would it be still be put 5g in the root and be done with it?

Thanks for the hard work in coming up with this gem!
- Jeff Starr 2012/04/26 11:31 am • Post Author
  
  Or would it be still be put 5g in the root and be done with it?
  
  Exactly that, as .htaccess rules are applied recursively down the directory cascade, so put it in root and everything is covered. If 5G is placed only in a subdirectory, that’s fine too, but only it’ll only block requests made to URLs within that directory.
Ken Dawes 2012/04/26 8:47 am

I wanted to put this in a separate note from my previous one.

There are a number of “specific” edits to 5g to allow certain things not to be blocked in WordPress.

For instance, I have found that the Query String
RewriteCond %{QUERY_STRING} (environ|localhost|mosconfig|scanner) [NC,OR]
needed “scanner” removed in order for the “TimThumb Vulnerability Scanner” plugin to be able to be updated

I’m thinking that it would be helpful for those of us using 5g to have a repository somewhere of the “exceptions” people have found. If could save a lot of time over the “half and then half again…” search for the 5g value that’s problematic.

Just a thought.
- Jeff Starr 2012/04/26 11:38 am • Post Author
  
  I think it’s a great idea, and something I’ve tried to do with each release by updating the article according to the feedback received via comments, email, twitter, wherever. It would be great to streamline the process and funnel all of it to an official exceptions page.. thanks for the idea :)
David Radovanovic 2012/05/01 7:13 am

Is there any possibility that legitimate traffic especially search engine bots will be deterred or some way depreciate SEO or rankings? I just got hit with numerous malicious attacks and the Pharma Hack. I only knew about it when Google sent me the dreaded “unnatural links” notice.
- Jeff Starr 2012/05/01 9:20 am • Post Author
  
  Absolutely, when Google has you in their crosshairs, both SEO rank and legit traffic can suffer. Eliminating the hacks quickly is essential. Good luck.
Regina Smola 2012/05/01 10:11 am

Awesome info Jeff! Testing it on a couple of WordPress sites now.

I have a client that has WP in the root and VBulletin in a subdir. I have copied the 5G Blacklist 2012 and the WordPress Add-on for 5G Blacklist in the root .htaccess. There is also a .htaccess in the VB directory with some simple security settings. Will the settings still re-curse into the forum and do you have any additional suggestions for blacklisting VB?
- Jeff Starr 2012/05/01 10:56 am • Post Author
  
  My pleasure :)
  
  Yes the blacklist rules (and other .htaccess rules) should be applied recursively thru the directory structure. You can test by appending any of the blocked terms to any URL and it should return a 403 Forbidden error. If you can provide some examples of the types of VB requests that you would like to block, I can investigate and possibly integrate into the upcoming 6G Blacklist. You can also test the strings directly using the 6G Dev page.
  - Regina Smola 2012/05/01 11:36 am
    
    Thanks Jeff.
    
    Actually I am seeing an issue with IP address 66.211.170.66 and 173.0.81.1, which belong to PayPal. The ips was denied by the server after adding 5G when trying to reach payment_gateway.php.
    
    Any ideas how to accept those IPs or to make that page not blocked?
  - Jeff Starr 2012/05/01 12:03 pm • Post Author
    
    I’m not seeing anything in the 5G that would block the string, payment_gateway.php.. perhaps there is a query string associated with such requests? If so, that could be it, as the 5G blocks a variety of potentially malicious query-strings.
- Caroline 2012/12/30 9:26 am
  
  Hi Jeff,
  
  I hate to waste your time if this has already been addressed. I looked through your comments and don’t see it addressed. So…
  
  Something in 5G blocks PayPal IPNs for people processing PayPal payments on their site. We got a message like this from PayPal:
  
  IPNs sent to the following URL(s) are failing: http://domain-name.com/?paypalListener=paypal_standard_IPN
  
  This causes ecommerce (I’m using Woocommerce) orders to stay marked as “pending” rather than “processing” (no notification is coming in from PayPal to adjust the status of the order to “paid/processing”). This is a common problem I’m seeing in forums on the ‘Net, but so far nobody else has linked it to 5G. I have. Since I don’t have a sandbox and don’t get enough PayPal orders to make the halving method go quickly, I thought I’d post here meanwhile to share my (pretty much confirmed since I deleted 5G and IPN worked again) hunch. I suspect it’s a query string but don’t know which yet.
  Thank you.
  - Jeff Starr 2012/12/31 3:57 pm • Post Author
    
    Hi Caroline, this is most likely due to PayPal’s use of a “blank user agent” for certain requests. To fix, comment out the following line (in 5G:[USER AGENTS] section):
    
    SetEnvIfNoCase User-Agent ^$ keep_out
    
    Note that this rule is already excluded in the next version of the blacklist (6G).
David Radovanovic 2012/05/01 12:24 pm

I should have rephrased my question: will any of the rules you have shown inadvertently block search engine bots/crawlers?
- Jeff Starr 2012/05/01 12:46 pm • Post Author
  That’s a great question, and always important to consider when blocking any sort of traffic. In general, blacklists use regular expressions to block potential bad requests. Any string that you are matching has the potential to block both good and bad requests. But there are known patterns and strings that are almost always used for exploit scanning and other malicious behavior. Those are the ones I go after with rules in the 5G et al. Careful testing and analysis is required to keep false positives at a minimum, whether the requests are search-engine or otherwise. I think I have achieved this with the g-series (and other) blacklists, but can’t say that it’s 100% accurate. I can say the 5G is widely used by people who let me know when issues arise. To date, no reports of major search-engine blockage.
  
  There’s much more to it, of course, but that’s some of the thinking behind it. I’ve written extensively on this topic if you’re interested, here are two immediately relevant articles with more info:
  
  https://perishablepress.com/building-the-5g-blacklist/
  
  https://perishablepress.com/building-the-perishable-press-4g-blacklist/
  
  I hope this helps!
William 2012/05/05 12:26 pm

the line:

RedirectMatch 403 \.(cgi|asp|aspx|cfg|dll|exe|jsp|mdb|sql|ini|rar)$

caused all sorts of problems for me.

I have WP in the webroot, with a subfolder and FancyIndexes enabled for a subroot. This is also password protected.

I have just removed that line.
- Jeff Starr 2012/05/05 2:01 pm • Post Author
  
  Thanks for the feedback, William. That line is aimed at blocking requests for various types of files, basically matching any requests for any of those types of files (e.g., .cgi, .asp, etc.). The patterns are very specific and should only block strings that end with one of the listed file types. It shouldn’t be interfering with any other type of request, afaik. That said, the other rules in the 5G will continue to work at protecting your site. Glad to hear you got it resolved.
Cristian Castellari 2012/05/07 1:46 am

Great list, but in my WP installation i’m having problems. When i’m writing a post, and i click on the Preview a first time, i can see the preview correctly.

Any further time, i get a 403 :(

Can you help me in this issue?
- Jeff Starr 2012/05/07 10:23 am • Post Author
  
  Hi Cristian,
  
  Try removing the 5G code and see if that helps. If so, try just adding one of the modules, say the request strings section, and see if the problem returns. If it does, then repeat the process by “halving” the code until you locate the offending match.
  
  If the problem persists after removing the 5G, it’s probably something else causing the issue.
  - Cristian Castellari 2012/05/07 10:28 am
    
    The problem is related to 5G code. Without it, the Preview works very well as the rest of the years i’ve blogged on this site :D
    
    Tomorrow morning i’ll do some tests :)
    
    I’ve enabled your blackhole script too (i wrote an email about a question to the code i add in wp-load.php), but it has no responsabilties.
  - Jeff Starr 2012/05/07 10:39 am • Post Author
    
    Sounds good. I only raise doubts because I use 5G here at Perishable Press and Preview seems to be working properly.. could be something though :)
  - brasofilo 2012/07/11 2:32 pm
    
    I had the same problem and believe this is the offending line:
    
    # RewriteCond %{QUERY_STRING} ? [NC,OR]
    # provoking 404 in Post/Page previews

« Previous Comments • 1…34567…10 • Newer Comments »