Redirect All Requests for a Nonexistent File to the Actual File

Published Tuesday, August 12, 2008 @ 11:47 pm • 19 Responses

In my previous article on redirecting 404 requests for favicon files, I presented an HTAccess technique for redirecting all requests for nonexistent favicon.ico files to the actual file located in the site’s web-accessible root directory:

# REDIRECT FAVICONZ
<ifmodule mod_rewrite.c>
 RewriteCond %{THE_REQUEST} favicon.ico [NC]
 RewriteRule (.*) http://domain.tld/favicon.ico [R=301,L] 
</ifmodule>

As discussed in the article, this code is already in effect here at Perishable Press, as may be seen by clicking on any of the following links:

Clearly, none of these URL requests target the “real” favicon.ico file, yet thanks to the previous method they are all happily redirected to the proper location. This is useful for a variety of reasons, including preventing excessive and unnecessary server strain due to malicious scripts.

Shortly after posting this method, Chris Coyier of CSS Tricks discovered that the code was causing an infinite redirection loop when applied to his site. After some experimentation, I have resolved the infinite loop issue, improved the syntax, and adapted the code to work from virtually any location.

The previous favicon redirection method works as advertised, but only if placed in a subdirectory, such as would be the case for WordPress installations located in their own non-root directory. For example, this WordPress installation for Perishable Press is located in a subdirectory named “press”. Within the HTAccess for the press subdirectory, the first version of this redirection technique works perfectly. When placed in the root directory, however, the code fails by creating the infinite loop. Thus, a better solution is needed..

Universal Redirect for Nonexistent Files

The following technique works in any directory — subdirectory or site root:

# REDIRECT 404 FILES
<ifmodule mod_rewrite.c>
 RewriteEngine on
 RewriteCond %{REQUEST_URI} !^/favicon\.ico [NC]
 RewriteCond %{REQUEST_URI} favicon\.ico [NC]
 RewriteRule (.*) http://domain.tld/favicon.ico [R=301,L] 
</ifmodule>

To implement this technique, replace both instances of “favicon.ico” with any file that you would like to redirect. The favicon is a good a choice for reasons discussed in the previous article, but other good candidates would be your site’s robots.txt file or any other file that seems impossible for people and bots to find. Further, if the location of the “real” file is located somewhere other than the root directory, indicate such in the first RewriteCond (e.g., !^/images/favicon\.ico). Then, after the rewrite conditions are taken care of, edit the RewriteRule to reflect the full URL of the actual file.

A couple of notes.. First, it is recommended that you place this code as far up the directory structure as possible. Begin with the site root HTAccess file. In most configurations, that should work fine; however, in some cases, such as when WordPress is installed in a subdirectory but is serving permalinks written without the subdirectory showing in the URL, the redirect may simply not happen. If this is the case, or if the redirect does not occur for some other reason (conflicting rewrite rules, etc.), try placing the code in the subdirectory. Bottom line: place the code in the root HTAccess and if the redirects don’t kick in, place the code in the blog subdirectory. The good news here, and the main reason for this improved method, is that the code will no longer trigger an infinite loop and crash your site; it is much safer, much cleaner, and more widely applicable.

Having bored you to tears with all of that detail, allow me to deliver the death blow with an explanation as to what’s actually happening when this code is executed. I think a nice, numbered list will help simplify the process:

  1. Check for the required Apache module, mod_rewrite
  2. Enable rewriting by activating the RewriteEngine
  3. Check if the requested URI is the actual file
  4. Match any nonexistent URI that contains the target file
  5. Redirect any matching URI to the actual file
  6. Close the module check container

Conclusion

Hopefully, this technique will be useful for anyone needing to stop those relentless 404 requests for nonexistent files. Doing so will not only help increase the effectiveness of your error-log investigations, but also help reduce the unwanted server load associated with automated directory prowling via arbitrary file candidates. Or something ;)


Dialogue

19 Responses Jump to comment form

1Louis

August 13, 2008 at 2:21 am

I wonder if it’s a good idea to redirect the request to the favicon. That won’t make the automatic requests stop, and that will burn some bandwidth that would not have been burned if you just ignore the request.

My point would be that your idea is cleaner, but will waste ressources :/

2Jeff Starr

August 13, 2008 at 9:42 am

Let’s think this through. With the rewrite rule in place, all requests for favicon.ico require a redirect and an image request. Without the rewrite rule in place, the htaccess file is still parsed yet instead of an image, a web page is served. Which of these cases consumes more resources? I don’t know for sure, but if I remember correctly, servers deliver images at much lower cost than web documents. It also may depend on the size of the image or file being requested. Favicons and robots.txt files are pretty small, but larger files would likely have a greater impact on performance.

3IMaggie

August 14, 2008 at 1:08 pm

I am pretty new to this, exactly where do I put the script. Is in the htaccess file?

4Jeff Starr

August 17, 2008 at 7:06 am

Hi IMaggie, technically the code presented in the article is not a script, but yes, if you are planning on using it, you would place it in an htaccess file, as described in the post itself. I hope that helps :)

5play games

August 24, 2008 at 3:15 pm

Great Work Perishable

I’ve customised your code to meet my specific reqs, namely to protect against hotlinkers linking directly to my content.

RewriteCond %{HTTP_REFERER} xx\.xxxxx\.com [NC,OR]
RewriteCond %{HTTP_REFERER} xxxxx\.org [NC]
RewriteRule \.(swf)$ redirect.swf [NC,L]

If you were in the UK I’d buy you a beer.

6Jeff Starr

September 1, 2008 at 9:29 am

Thanks for the feedback — I’ll be taking you up on that beer next time I’m in the UK! ;)

7htaccess redirect

September 11, 2008 at 7:37 pm

I was looking for this rewrite code for Wordpress for quite awhile, couldn’t get it to work myself. Thanks!

8Piller

September 17, 2008 at 3:46 pm

Hello,

I am having some trouble redirecting URL on my site. I used Expression Engine to mount it, and suddenly something happened and this kind of redirectioning are not working anymore:

http://loscuchillos.com/letras should go to http://loscuchillos.com/index.php/letras

So, what I basically need to perform is an automatic redirection to /index.php/NAME of whatever NAME typed as a subfolder. Without showing the whole address.

So the code the Expression Engine provides to get this kind of redirectioning (http://expressionengine.com/wiki/Remove_index.php_From_URLs/) is not working anymore.

It was working before and now I wounder what could be wrong.

Rewrite Module is enabled, same as .httaccess

Is there anythins else I should consider?

I’d really appreciate any comments on this issue.

Thanks,

Piller

9Gabriel H.

September 28, 2008 at 12:32 am

Hello,

Be careful with the redirection, you can have an infinite loop.

To avoid that, you can rename the favicon.ico file by something else.

Eg :

RewriteCond %{THE_REQUEST} favicon.ico [NC]
RewriteRule (.*) /favicon_all.ico [R=301,L]

If I don’t do this, I get infinite redirections…

10Jeff Starr

September 28, 2008 at 8:01 am

Hi Gabriel, I am wondering if you read the entire article, or happened to copy & paste that first chunk of code. The reason I say this is because this article is essentially about preventing infinite loops. The htaccess code presented at the beginning of the article was given in a previous article and may indeed cause infinite loops, which is precisely why I wrote this article. If you read through, you will see that the infinite loop issue is resolved with a new and improved version of the code. I know of at least several sites that use the technique with no problems whatsoever. With the proper technique there should be no need to rename any files. I hope this helps!

11Gabriel H.

September 28, 2008 at 7:27 pm

Hello Jeff Starr,

My apologies, you are right, I didn’t read the whole article and just copy-pasted blindly the code… Now that works much better with the new code ;)

Have a nice day.

12JK

December 20, 2008 at 6:17 am

Hi Jeff,

First of all, you have a great site.

I too get tons of 404s with favicon.ico missing like your example post URLs from your site shown above. I followed your steps and read several times to get this correct. For some reason, the redirect does not happen at all. Let me explain further on my site configuration.

I have the main site in the root directory which is not blog (plain static site). In the sub-directory I have a blog with it’s own domain name. So I have a .htaccess file specifc to my blog in this sub-directory. Currently that’s where I have all rewrite commands. So I added your rewrite commands to this file towards the bottom. Still I get 404s for favicon.ico & .gif files.

What am I missing?

Thanks for your help.

13JK

December 20, 2008 at 6:41 am

Ok. I found the problem.

The set of lines that you suggested to be in put in .htaccess file should go all the way to the top of the file - only then it kicks in. Otherwise it has been ignored.

Thanks.

14Jeff Starr

December 21, 2008 at 9:49 am

@JK: yes, that may be the case, depending on the contents of your htaccess file(s), it might be necessary to arrange the various rewrite rules in such a way as to avoid potential conflicts. In any case, I am glad to hear you got it working! :)

15Kalle

December 29, 2008 at 2:48 am

I just discovered all these favicon 404’s after installing CyStats. Rather annoying. Thanks for the code, it seems to help. But what ARE all these 404’s? Who or what is looking for my favicon in all possible places, and above all - why??

16Jeff Starr

January 1, 2009 at 9:51 am

Hi Kalle, Glad to hear the method is helping you. I recently participated in a comment thread over at CSS Tricks that discusses the technique in greater detail and even gets into “why” these types of requests occur in the first place. Check it out at:

http://css-tricks.com/weird-file-requests-and-easing-server-stress-with-htaccess/

Subscribe to comments on this post


Share your thoughts..

TopRead official comment policy

Contact Perishable Press

  • Contact Jeff via form

Search Perishable Press

About Perishable Press

Perishable Press is the virtual playground of Jeff Starr — visionary, founder and lead developer of Monzilla Media, a small web and graphic design company in the lush desert oasis of Moses Lake, Washington. Perishable Press features articles and tutorials on many aspects of digital design..

Read more..

Perishable on Twitter

Google tells users to drop support for IE6! @ http://www.tgdaily.com/content/view/40785/140/

Perishable on Tumblr

WordPress Tip for Multiple Themes

Sunday, 4 January 2009, 5:16 pm

If your site makes available multiple themes for users to choose from, remember to include the JavaScript (or any other required code) for any statistical applications that you might be using, such as Mint, Google Analytics, and so forth. I am not sure about the various WordPress statistics plugins, but they may need to be included as well. A good way to check if your stats plugin is tracking data across all themes is to either visit a few pages that you know others aren’t hitting, or else activate each of the alternate themes and check the source code of each one for the required code.

Earlier today, I realized that only several of my most recent themes included the required JavaScript for Mint and Google Analytics. I am now in the process of editing each of the 18 themes available for users at Perishable Press. Haven’t decided on whether or not both statistics apps are needed for all themes, but I will certainly be using at least one of them to keep an eye on everything.

Insane Christmas

Monday, 22 December 2008, 9:47 pm

For as long as I can remember, Christmas has always been a relatively peaceful affair. Sure there’s the usual holiday stress — traffic, shopping, presents, relatives, and all that goes with the preparation of a traditional celebration, but when it’s all said and done, you get to relax and enjoy the peace and harmony of gathering together and basking in the reason for the season: the birth of Christ.

This year, however, the stress factor has been kicked up a few notches, making for a rather insane Christmas if I do say so myself. In addition to the usual holiday chaos, we are currently purchasing a brand new home, and quickly realizing the incredible amount of work involved in the process. If you’ve ever bought a newly built home, you know exactly what I am talking about here.

Plus, as if all the paperwork, inspections, insurance, costs, and anxious anticipation weren’t enough to confound the usual holiday stress, we are also packing up everything, dealing with kids, working full-time jobs, and — beginning on Christmas Eve — moving into our new house.

It certainly is all a great joy and blessing to have such amazing things going on, but combined with the work that I do on the Web — blogging, designing, projects, helping people, and so on — it really becomes all too much rather quickly. We are doing are best to get through everything with our sanity intact, but I have to admit that this is the most insane Christmas I have ever experienced.

New (4G) Blacklist Now in Beta

Monday, 22 December 2008, 9:27 pm

Just a quick note to anyone interested in securing their websites against malicious activity, spam, and other nonsense. Several months after releasing my 3G Blacklist, I have finally begun work on the next incarnation of the blacklist: the 4G Firewall!

The first part of the blacklist is now ready for testing, and I plan on setting it up on Perishable Press within the next few days. While testing on my own site, I thought it would beneficial to also invite a few “beta” testers to run the code on their own site(s) as well.

So, if you have a site that receives its share of malicious attacks, and cracker exploits, drop me a line via the contact form at Perishable Press and I will send you the initial block of HTAccess directives. This version of the Blacklist is looking better than ever, and I look forward to releasing the complete version to the public early in 2009.

Thanks for the Free Traffic and Link Juice

Sunday, 7 December 2008, 1:26 pm

Just wanted to thank the fine folks at fafich.ufmg.br for all the free traffic and link juice. Thanks to their misapplication of my comprehensive canonicalization code, every non-canonical version of their 21,700 indexed pages points directly to my site, Perishable Press. This means that every one of their permalink URLs that is mistyped, lacks the “www” prefix, or contains the superfluous “index.php” file name is directed via permanent redirect directly to the home page of my site.

I have tried contacting the site owner(s) about this situation, but it has been over a week and I have yet to hear anything back. Hopefully, they will take notice soon and correct the issue by properly configuring their htaccess file, but in the meantime, I certainly don’t mind the extra link juice and free traffic! :)

No Plugin Needed for Feed Delay

Monday, 24 November 2008, 10:01 am

I recently saw a WordPress plugin that was designed to delay the publication of your WordPress feed by any specified time interval. While it is a good idea to carefully proofread your content before posting it, a plugin certainly is not required to do so.

As savvy WordPress users already know, WordPress has a built-in post-preview feature that enables authors to view their unpublished content as a published post. This enables authors to do any amount of proofreading and browser checking until they are satisfied with the results.

To do this, simply write your post as usual, and then click on the “Preview this post” button on the right-hand side of the screen. In older versions of WordPress (less than 2.5, I think), you actually need to save (without publishing!) the post first and then re-open it as if to continue editing. You will then see a “Preview »” link sort of hidden (due to poor CSS design) in the upper-right corner near the edit post field. Right-click on that link to open in new tab and you are good to go.

No extra plugin needed! :)

Read more on Tumblr..

Subscribe to Comments Recent Dialogue

  • Harry Roberts: Yeah, as above, removing the outline is a pretty bad idea in terms of accessibility. Maybe removing it from large image/navigational ...
  • Mark: There we go! That's the way to do it! Thanks, Jeff!...
  • Jeff Starr: Well said, Mark! Here is some news that I find ...
  • Jeff Starr: Thank you all for the great feedback! I wrote this article as a way to purge some of my thoughts on Twitter, but now see that some of...
  • Jeff Starr: Thank you so much for the thoughtful feedback, Adrian. It has been a good year indeed, and I certainly hope that 2009 brings many ble...
  • Jeff Starr: Hi heywho, glad to hear you are doing well! ;) I wish I could join in the festivities.. it has been so long that I almost have forgot...
  • Rob Barrett: Thanks for posting about the Stealth Publish plugin -- just what I needed for my site. Works perfectly!...
  • Jeff Starr: Hi Chiwan, I got your email and have sent some information that may help you with this. Cheers, Jeff...
  • Chiwan: Hi. This is cool. So I can I replace the clock that comes with your Apathy theme with this clock? If that's not possible, how do ...
  • Brass Engraved: Thankyou very much for this, worked like a dream!...

Read more recent comments..

Attention: Do NOT follow this link!