How to Block Proxy Servers via htaccess

By Jeff Starr on Sunday, April 20, 2008 — 18 Responses

Post #534 categorized as Function, last updated on May 28, 2008
Tagged with blacklist, htaccess, list, mod_rewrite, proxy, security, spam, tricks

Not too long ago, a reader going by the name of bjarbj78 asked about how to block proxy servers from accessing her website. Apparently, bjarbj78 had taken the time to compile a proxy blacklist of over 9,000 domains, only to discover afterwards that the formulated htaccess blacklisting strategy didn’t work as expected:

deny from proxydomain.com proxydomain2.com

Blacklisting proxy servers by blocking individual domains seems like a futile exercise. Although there are a good number of reliable, consistent proxy domains that could be blocked directly, the vast majority of such sites are constantly changing. It would take a team of professionals working around the clock just to keep up with them all.

As explained in my reply to bjarbj78’s comment, requiring Apache to process over 9,000 htaccess entries for every request could prove disastrous:

The question is, even if you could use htaccess to block over 9,000 domains, would you really want to? If you consider the potential performance hit and excessive load on server resources associated with the perpetual processing of such a monstrous list, it may inspire you to seek a healthier, perhaps more effective alternative..

A better way to block proxy servers

Rather than attempt to block proxy servers by who they are (i.e., via their specified domain identity), it is far more expedient and effective to block proxy servers by what they do. By simply blacklisting the various HTTP protocols employed by proxy servers, it is possible to block virtually all proxy connections. Here is the code that I use for stopping 99% of the proxies that attempt to access certain sites:

# block proxy servers from site access
# http://perishablepress.com/press/2008/04/20/how-to-block-proxy-servers-via-htaccess/

RewriteEngine on
RewriteCond %{HTTP:VIA}                 !^$ [OR]
RewriteCond %{HTTP:FORWARDED}           !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA}       !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR}     !^$ [OR]
RewriteCond %{HTTP:PROXY_CONNECTION}    !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION}   !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP}      !^$
RewriteRule ^(.*)$ - [F]

To use this code, copy & paste into your site’s root htaccess file. Upload to your server, and test it’s effectiveness via the proxy service(s) of your choice. It may not be perfect, but compared to blacklisting a million proxy domains, it’s lightweight, concise, and very effective ;)

Copyright2009PerishablePress

Subscribe to Perishable Press

47048 visits

18 Responses

Top • Leave a comment

[ Gravatar Icon ]

#1 — Gabry

Hello was reading your page about the htaccess file to block proxy servers from surfing my site, very interesting, but my host said that since I use FrontPage editor it might block me from editing my site, is there a way to avoid this? Thank you in advance

[ Gravatar Icon ]

#2 — H5N1

Is this already effective? :)
I tried to read this arcticle via web-proxy without problem! :D
I thought this limitation was already implemented here for web-proxy…

[ Gravatar Icon ]

#3 — Perishable

Hi H5N1 :)

No, I do not block proxy servers from Perishable Press. There are a number of readers who (for whatever reason) visit this site via proxy. It is important to me to facilitate site access for this select group of individuals, even at the expense of malicious spam and other attacks. Maybe someday I will change this policy, but for now, it is my hope that the code provided in this article will prove useful to other site owners and webmasters.

[ Gravatar Icon ]

#4 — Willard

I was hoping you’d eventually create an article by itself on this subject! Good advice I must say. =)

I do have one question though.. there seems to be a small difference between what you posted the other day, and what you posted here.. specifically:

RewriteCond %{HTTP:XROXY_CONNECTION} !^$ [OR]

vs.

RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]

Notice the additional P? I’m just wondering if that was added on purpose.

Thanks for the input if you have any!

[ Gravatar Icon ]

#5 — Perishable

@Gabry: I am unfamiliar with FrontPage protocols, however you could always try uploading the code and checking for access. Then, if FrontPage is blocked, try removing one line at a time until access is achieved. If successful, this method of removing a line (or two) will reduce the overall effectiveness of the htaccess blocking rules to some degree, but should still provide a significant amount of protection. Also, a cursory search of the required FrontPage protocol indicates that the required header may in fact be X_FORWARDED_FOR or even X-FORWARDED-FOR, which isn’t on the list. So, try the code as-is first and if you are blocked, then try removing the X_FORWARDED_FOR first. Finally, if that fails, try removing different lines one at a time and checking the results. Sorry I couldn’t provide more specific advice, but hopefully these clues will help get you going in the right direction.

[ Gravatar Icon ]

#6 — Perishable

@Willard: The reason for the change is based on research that suggests that XPROXY is the correct protocol for this purpose. However, after reading your comment and looking into it further, it seems that XROXY is also a commonly employed protocol/header for proxy servers. So, to be honest, I am considering adding the XROXY condition to the htaccess code just to be safe. Further, I am also considering adding three more common proxy protocols to the list as well, which, when added to the XROXY case, would give the following four additions:

RewriteCond %{HTTP:XROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:X-FORWARDED-FOR} !^$ [OR]
RewriteCond %{HTTP:FORWARDED-FOR} !^$ [OR]
RewriteCond %{HTTP:X-FORWARDED} !^$ [OR]

I am thinking that these additional directives will help improve the overall effectiveness of this proxy-blocking technique. I am not going to edit the article just yet, however, as I am hoping that someone with some deeper knowledge of the subject will chime in with some definite information on the topic. I apologize for any confusion in the matter. Thanks for sharing your concerns with us! :)

[ Gravatar Icon ]

#7 — prislea

how I can exclude some ip’s/proxy’s from the filter?

tks.

[ Gravatar Icon ]

#8 — Perishable

Hi prislea, have you tried including an additional rewrite condition targeting the specific domain, for example:

RewriteCond %{HTTP_REFERER} !.*allowedproxydomain.com.*

I haven’t tried this yet, but it may help you to get going in the right direction :)

[ Gravatar Icon ]

#9 — sam

are these conditions suppose to block sites like hidemyass.com or similar sites?
because I tried and its not blocking it.

[ Gravatar Icon ]

#10 — David

I’m looking at this page on blocking proxy servers, using a proxy server.

I tried the code, it doesn’t seem to work for proxylord.com.

[ Gravatar Icon ]

#11 — Jeff Starr

@David: see comment #3 for an explanation as to why you are able to surf this site while using a proxy. Also, this code blocks proxies by targeting associated HTTP protocols. The block list is not comprehensive, so proxies using unlisted methods will not be blocked.

[ Gravatar Icon ]

#12 — David

Yes, I see #3.
Is there a way to block the anonymous proxy server with the .htaccess codes?

Maybe it’s a go Daddy thing.

[ Gravatar Icon ]

#13 — Perishable

Hi David, it should be easy to block the anonymous proxy server via htaccess. Add the following code to your root htaccess file:

# deny domain access
RewriteEngine on
RewriteBase /
RewriteCond %{HTTP_USER_AGENT} anonymous [NC]
RewriteRule ^.* - [F,L]

..of course, this method blocks by targeting the user agent, which may or may not be the same as the domain name. Another way to block a specific proxy is to target the domain itself, as identified via referrer:

RewriteCond %{HTTP_REFERER} ^http://.*anonymous.*$ [NC]

This line should replace the RewriteCond line in the previous code. Remember to test thoroughly!

[ Gravatar Icon ]

#14 — Eric

Will this also block Paypal IPN? Untested on my end.. waiting for a payment to come through rather than converting all my ipn stuff to sandbox.

[ Gravatar Icon ]

#15 — Perishable

Thanks for the feedback, Eric — keep us updated on the results..

[ Gravatar Icon ]

#16 — air force ones

Hey Perishable。 I have a good idea about how to block proxy server. Cause the operation system of most proxy server are Linux but the operation system of most visitor are windows. So If we can block Linux, maybe can block most proxy server.

[ Gravatar Icon ]

#17 — Perishable

Are you kidding? A good number of my visitors are Linux users. I definitely do not want to block them. I appreciate the idea, but think it would be an unwise move. The last thing I want to do is cater specifically to Windows users..

[ Comments are closed for this post. ]

If you have or need further information, contact me.

Search Perishable Press
Standards-based web design
WordPress, blogging & security
  • WordPress / Plugins, Themes, Tutorials
  • Blogging / Social media, social networking
  • Security / Anti-spam, blacklists, firewalls
Featured Content

Explore Perishable Press

Join 2428.41+ Subscribers
Perishable Resources
Perishable Projects
Random Friends
© 2009 Perishable Press • Design by Monzilla Media
Site Credits • Site Map • Linkage • Top