Controlling Proxy Access with HTAccess
In my recent article on blocking proxy servers, I explain how to use HTAccess to deny site access to a wide range of proxy servers. The method works great, but some readers want to know how to allow access for specific proxy servers while denying access to as many other proxies as possible.
Fortunately, the solution is as simple as adding a few lines to my original proxy-blocking method. Specifically, we may allow any requests coming from our whitelist of proxy servers by testing Apache’s HTTP_REFERER
variable, like so:
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-01.domain.tld(.*)
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-02.domain.tld(.*)
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-03.domain.tld(.*)
Notice the pattern here. Each line matches against the specified proxy server in the referrer variable. Once integrated into the original method, each of the three specified URI’s will be allowed access to your site. Thus, by editing these directives to match the name and number of your whitelisted proxy servers, you can allow access to any list of proxies or other referrers while blocking many of the others.
Integration
To integrate your customized whitelist RewriteCond
itions with the original proxy-block method, simply place them near the end of the existing conditions, directly above the RewriteRule
, like so:
<IfModule mod_rewrite.c>
RewriteEngine on
RewriteCond %{HTTP:VIA} !^$ [OR]
RewriteCond %{HTTP:FORWARDED} !^$ [OR]
RewriteCond %{HTTP:USERAGENT_VIA} !^$ [OR]
RewriteCond %{HTTP:X_FORWARDED_FOR} !^$ [OR]
RewriteCond %{HTTP:PROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:XPROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:HTTP_PC_REMOTE_ADDR} !^$ [OR]
RewriteCond %{HTTP:XROXY_CONNECTION} !^$ [OR]
RewriteCond %{HTTP:X-FORWARDED-FOR} !^$ [OR]
RewriteCond %{HTTP:HTTP_CLIENT_IP} !^$ [OR]
RewriteCond %{HTTP:FORWARDED-FOR} !^$ [OR]
RewriteCond %{HTTP:X-FORWARDED} !^$
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-01.domain.tld(.*)
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-02.domain.tld(.*)
RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-03.domain.tld(.*)
RewriteRule ^(.*)$ - [F]
</IfModule>
Just slap that bad boy into your server’s httpd.conf
file or the HTAccess file of your choice (generally the root HTAccess file), and you’re golden. Note that not all proxies reveal the information targeted in these directives, but many of them continue to do so. Thus, with this code in place, you will enjoy protection against unwanted proxies while allowing open access to the proxy servers or other referring domains of your choice.
Comprehension
Here at Perishable Press, we’re all about understanding how these types of methods actually work. Comprehension is important, especially when it comes to this type of black-magic Apache voodoo. So let’s enjoy a few explanatory spoonfuls, shall we?
As usual, we first check for the required Apache module, which in this case is the inimitable mod_rewrite. Then, after initializing the rewrite engine, the next twelve lines test different variables for any (non-empty) value. These twelve variables are associated with proxy servers and may contain data if present. The [OR]
flags appended to the first eleven of the RewriteCond
itions cumulatively tells Apache something to the effect of “if any of these variables contain any value whatsoever, then invoke the specified RewriteRule
”. And this is where the original proxy-blocking directives end. The rewrite is then applied for any matched cases, and the unwanted proxy visits are subsequently denied.
In this updated method, however, we also want to allow our choice of specific proxy servers. So, by appending the previously discussed whitelist directives to the list of RewriteCond
itions, we are qualifying the first twelve conditions as follows:
if the client is not being sent via this proxy method OR this proxy method OR this proxy method OR this proxy method OR … this proxy method, AND the referrer is not
allowed-proxy-01
AND the referrer is notallowed-proxy-02
AND the referrer is notallowed-proxy-03
, then invoke the specifiedRewriteRule
.
And so, if the entire collection of conditions prove true, the specified RewriteRule
will be applied and the proxy request will be blocked. Conversely, if the referrer is on the whitelist, they will be granted access — regardless of whether or not any of the previous proxy variables contain values.
Sayanora
That’s it for this fine tutorial. Please let me know if you have any questions, concerns or criticisms regarding this method. Hopefully, the explanation is clear, but if not, please let me know!
15 responses to “Controlling Proxy Access with HTAccess”
looks great man; a few questions.
Would it be possible to have he ‘white list’ saved at location ‘x’ as a .txt file? and further save all the proxies that are blocked to a blacklist file?
So making it essentially the same as you have except the last bit saying
AND the referrer is not in ‘whitelist.txt’ THEN save refer in ‘blacklist.txt’ AND invoke the specified RewriteRule.
This will then allow you to keep an eye on who is being blocked just against you forget to whitelist a proxy service.
Yes php can do that (after a few hours of trialling I go tit to work :p) just been contemplating making a ‘gateway’ script…i.e. all traffic must pass through this script before it hits the site.
Will file this away with my notes…will prob get round to it soon…but when is 4G out?!
Not sure if HTAccess directives provide that kind of flexibility, however, PHP could certainly be used to incorporate that sort of functionality.
Very soon.. the 4G is ready to go, I just need to finish the articles that go along with its release. I have removed the user-agent blacklist and the referrer blacklist and will present them as optional, independent (and extremely beefed up) blacklists along with the 4G. The 4G still contains multiple parts, but user-agents and referrers will not be targeted.
Just wondering why one would want to block proxy servers yet allow a few.
Could someone give an example of proxy servers you want to have access while blocking others?
@James: One example that comes readily to mind involves company intranets, where proxy servers are frequently employed throughout the network. This article was written because numerous people were asking how it could be done, and intranet proxies were one of the reasons why it was necessary.
I been useing the blocking proxy servers, users using cell phones are getting 503 errors.
What would I add to allow cell phone users access?
I missed typed on that last comment, thats posta be 403 error.
@James
you have “bad”(blocked) users who uses anonymous proxy’s and you have “good” users who uses proxy’s from theyr work.
is there a way to allow access by IP?
“prislea: is there a way to allow access by IP?”
Thats what I’m asking. If I added something like RewriteCond %{HTTP_REFERER} !(.*)allowed-proxy-01.verizon.tld(.*)
Would cell phone users on Verizon have access?
@James: It’s all in how you specify the rewrite logic. To ensure that the proxy conditions are only matched when the referrer is not Verizon, drop the
[OR]
flags and use the implicit[AND]
flags instead.