Perishable Press

2 • October 16, 2007 at 9:45 am — Perishable says:

Good question.. adding it to the list was suggested by a fellow forum member awhile ago. I sniffed around Google a bit but found no evidence of maleficence, so I have removed it from the list. Thanks for pointing it out to us ;)

4 • November 10, 2007 at 6:53 pm — Perishable says:

John, rest assured that there are absolutely no googlebots mentioned in the blacklist. We are targeting the bad guys here, not legitimate agents such as Google, Yahoo!, and MSN. No worries! ;)

6 • December 11, 2007 at 10:22 pm — Perishable says:

Lisa you are wearing me out! :)

Anyway, to answer your question, yes, you can list multiple IP addresses in one line when using a “deny from ip” directive. Here is the syntax that you would use:

deny from 123.123.123 456.456.456 789.789.789

..etc. Notice that we are using spaces to separate the individual IPs — no commas allowed ;)

8 • January 22, 2008 at 8:57 am — Perishable says:

Hei bjarbj78,

RewriteBase specifies the default URL for a directory. It is often used to remove the local directory prefix in order for the rewrite rules to act upon the correct file path. Don’t worry, if it sounds confusing, it is — I still find myself referring to the source when trying to explain it to people :)

Regards,
Jeff

10 • January 23, 2008 at 10:38 pm — Perishable says:

The question is, even if you could use htaccess to block over 9,000 domains, would you really want to? If you consider the potential performance hit and excessive load on server resources associated with the perpetual processing of such a monstrous list, it may inspire you to seek a healthier, perhaps more effective alternative..

For example, here is the code that I use for stopping 99% of the proxies that attempt to access one of my sites:

# prevent proxy access
RewriteEngine on
RewriteCond %{HTTP:VIA} !^$ [OR]
RewriteCond %{HTTP:FORWARDED} !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA} !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR} !^$ [OR]
RewriteCond %{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP} !^$
RewriteRule .* - [F]

Place in root htaccess and take ’er for a spin via any proxy service(s) of your choice.. - it may not be perfect, but it’s lightweight, concise, and very effective ;)

13 • January 24, 2008 at 1:10 pm — Perishable says:

Hi Bjørn,

Glad it worked for you! I have found it to be much more efficient than trying to block the endless swarm of proxy servers individually — things change waay too quickly for that. As for the additional rewrite conditions you mention, I have not used them.. perhaps you could test their effectiveness and share the results with us? ;)

Regards,
Jeff

15 • January 27, 2008 at 9:24 am — Perishable says:

Hi Lisa :)

Using htaccess, wildcards are not necessary when specifying specific ranges of IP addresses. Simply truncate the address to the target range, for example:

deny from 74.54.

..would block every IP beginning with the 74.54. prefix:

74.54.1
   .
   .
   .
74.54.255
74.54.255.1
   .
   .
   .
74.54.255.255

..etc. Of course, you could also use wildcards if so desired:

deny from 74.54.*

..which would block the exact same range as before. It’s totally your choice! :)

16 • January 27, 2008 at 9:32 am — Perishable says:

Ah, one more thing that I should point out.. Be careful when using the “dot” (.) in your blocked IP ranges. If we omit the dot from the example in the previous comment, we would block a different set of IP addresses:

74.540
   .
   .
   .
74.549
74.540.1
   .
   .
   .
74.549.255

..etc. Inclusion or exclusion of the dot can make all of the difference!

20 • January 30, 2008 at 4:04 pm — Perishable says:

Pádraig Brady,

Yes, that is an excellent idea. I am currently in the process of building the next version of the Ultimate htaccess Blacklist (v.3), and will definitely post an article once it is finished. As you can see, this post features the second version of the Blacklist, and I continue to post many useful tips and tricks concerning all things htaccess, PHP, XHTML, CSS, and all of the other web-related acronyms ;) Thus, my advice to people who want to stay current with the Ultimate htaccess Blacklist as well as other security and anti-spam information is to subscribe to my feed — I guarantee you won’t regret it!

Regards,
Jeff_

22 • March 5, 2008 at 10:12 am — Perishable says:

Hi Peter, you are correct — you must remove the RewriteBase / directive when using the blacklist in your httpd.conf file. It is meant for per-directory htaccess files and is not needed when working with httpd.conf.

25 • March 15, 2008 at 5:54 pm — Perishable says:

Thanks for the tip, Max. I will definitely include this modification in the next iteration of the blacklist. Cheers! ;)

27 • April 21, 2008 at 7:51 am — Perishable says:

Sure, I am more than willing to add it the next incarnation, provided some sort of explanation as to why “anonymouse” (or any other user agent for that matter) deserves to be blacklisted. Drop a link and we’ll go from there.. ;)

29 • April 23, 2008 at 8:33 am — Perishable says:

Yes, I have also read elsewhere that blocking anonymouse is a wise move. I will add the directive to the next incarnation of the blacklist. Thanks for helping to improve the list! :)

31 • April 29, 2008 at 2:14 pm — Perishable says:

Hi John, I’m glad you enjoy the article. As for the blacklist, are you sure that it is the cause of the problem? It may very well be, however, many people use the list without any similar issues. Nonetheless, if you have determined that the source of the conflict involves the blacklist rules, a bit of further testing should help you target the problematic user agents. First, remove the first half of the user agents and test if things are working. If so, replace half of the removed agents and test again. Likewise, if the issue did not resolve after removing the first half, repeat the process with the second half. By repeating this process a few times (it shouldn’t take too long), you should be able to identify the conflicting agents. I hope that makes sense. I would do this exercise myself, but I am literally swamped with a million other tasks and chores that I have to get done. In any case, I hope this information is useful, as I definitely want to help you implement successfully the ultimate htaccess blacklist. Finally, if you do find problematic user agent(s), please drop a comment and share with the community here at Perishable Press.

Cheers,
Jeff

33 • April 30, 2008 at 1:32 pm — Perishable says:

Hi John, thank you for taking the time to share this information. I have removed the DA character string from the list and updated the article so that others may benefit from your work (and generosity). Thank you for helping to improve the quality of the blacklist! :)

Kind regards,
Jeff

35 • May 1, 2008 at 5:28 am — Perishable says:

Doh! I completely forgot about Max’s comment concerning the DA string! Apparently age is directly proportional to forgetfulness (I should have scrolled thru the comments before responding) — would’ve saved everyone some time. Max’s solution also enables us to continue blocking the DA agent while allowing access to the various validation services. Once I return to the computer (this weekend), I will update the article and the blacklist with this information. Huge apologies to Max — thanks for the gentle reminder ;)

36 • May 4, 2008 at 8:30 am — Perishable says:

The blacklist and article have been updated to (re)include the DA$ character string. Many thanks to Max for pointing this out. Cheers!