Latest TweetsHeads up: Blasty (DMCA service) MIA: perishablepress.com/avoid-blas…
Perishable Press

Comprehensive URL Canonicalization via htaccess for WordPress-Powered Sites

[ Secrets Revealed! ] Permalink URL canonicalization is automated via PHP in WordPress 2.3+, however, for those of us running sites on pre-2.3 versions or preferring to deal with rewrites directly via Apache, comprehensive WordPress URL canonicalization via htaccess may seem impossible. While there are several common methods that are partially effective, there has not yet been available a complete, user-friendly solution designed specifically for WordPress. Until now..

In this article, I share my “secret” htaccess URL canonicalization formula. I originally developed this method in July of 2007, and have been using it successfully on a variety of WordPress-powered sites since that time. Thus, having verified the effective performance of this technique, I feel confident in sharing it with the open-source community. But first, a bit of context..

What is URL canonicalization?

Technically speaking, a canonical URL is the definitive URL for any given web page. For example, let’s say you have a web page located at the following URL:

http://domain.tld/subdirectory/canonicalized/

Assuming that this URL is the preferred, official URL, it is referred to as the canonical URL for that particular web page. As the canonical URL, that is the address that you want listed in the search engines, bookmarked by visitors, and so on. Unfortunately, on sites running WordPress versions less than 2.3, that web page would also be accessible at the following URLs:

http://domain.tld/subdirectory/canonicalized/
http://domain.tld/subdirectory/canonicalized
http://domain.tld/subdirectory/index.php/canonicalized/
http://domain.tld/subdirectory/index.php/canonicalized
http://domain.tld/subdirectory/index.php?p=77
http://domain.tld/subdirectory/?p=77

http://www.domain.tld/subdirectory/canonicalized/
http://www.domain.tld/subdirectory/canonicalized
http://www.domain.tld/subdirectory/index.php/canonicalized/
http://www.domain.tld/subdirectory/index.php/canonicalized
http://www.domain.tld/subdirectory/index.php?p=77
http://www.domain.tld/subdirectory/?p=77

And so on.. Each of these non-canonical URLs is accessible by visitors and search engines, creating many potential sources of duplicate content. Duplicate content is undesirable as it may damage the search engine placement of the associated web page. This happens when the search engines index multiple versions of the same page and effectively divide the attributed link equity among the duplicates. Consolidating the worth of your pages by serving only a singular instance of each of them is beneficial to you, your visitors, and your search engine rankings.

Should you use this canonicalization method?

As previously mentioned, WordPress 2.3+ provides a built-in URL canonicalization technique via PHP. Thus, if you are using WP 2.3+, you technically don’t need to use this method, however, if you prefer to handle URL rewriting at the server level, then you may indeed benefit from its use.

For those of us running WP versions less than 2.3, WordPress includes no comprehensive canonicalization method. In fact, this technique was developed before WordPress 2.3, and is specifically designed to provide complete URL canonicalization for WordPress-powered sites. If this sounds like you, and you are concerned about duplicate content issues, I would definitely recommend implementing this solution1.

Comprehensive URL Canonicalization for WordPress

Now that we’re all on the same page (so to speak), let’s have a look at the complete htaccess canonicalization code, as used here at Perishable Press:

# Comprehensive URL Canonicalization for WordPress
RedirectMatch permanent index.php/(.*) https://perishablepress.com/press/$1

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.perishablepress\.com$ [NC]
RewriteRule ^(.*)$ https://perishablepress.com%{REQUEST_URI} [R=301,L]

RewriteBase /press/

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.(html|php)\ HTTP/
RewriteRule ^(([^/]+/)*)index\.(html|php)$ https://perishablepress.com/press/$1 [R=301,L] 

RewriteCond %{REQUEST_URI} /+[^\.]+$
RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /press/index.php [L]
</IfModule>

To verify the effectiveness of this method, try accessing a few non-canonical URLs for this site. For example, entering any of the non-canonical URL formats previously presented will return the official, canonical URL for any given page.

The Breakdown

Before presenting a generalized, copy-&-paste version of this htaccess canonicalization solution, let’s examine the functionality involved with each of its various directives:

Remove all instances of index.php from URLs
In the first line, we call upon RedirectMatch to match and remove all instances of index.php occurring within our permalink URLs.
Check and initialize the mod_rewrite module
In the second line, we check for the presence of the mod_rewrite module via an <IfModule mod_rewrite.c> container. Then, in the third line, we initialize the rewrite engine with RewriteEngine On.
Specify www prefix preference for all URLs
In the fourth and fifth lines, we effectively match and eliminate all instances of the www prefix from our URLs. With a simple modification we could ensure that the www prefix was included in our canonical URLs. For more information on this, check out our 2-for-1 special.
Specify the base directory for the rewrite rules
In the sixth line, we specify the base directory for the rewrite rules using the RewriteBase directive. For those of us familiar with the htaccess rules used for WordPress permalinks, this line should look familiar.
Remove all instances of index.php from the end of URLs
In the seventh and eighth lines, we are matching and removing all instances of index.php (or index.html) from the end of our permalinks.
Ensure a trailing slash at the end of all URLs
In the 9th and 10th lines, we ensure that all permalink URLs include a trailing slash.
Execute the remaining portion of the default permalink directives
The next three lines constitute the remainder of the default WordPress htaccess permalink rules. These should look familiar, and are required in order for WordPress to utilize its native permalink functionality.
Wrap it up by closing the IfModule container
Back in the second line, we opened a <IfModule mod_rewrite.c> container to check for the presence of the required mod_rewrite module. Now that we have finished specifying our complete set of rewrite directives, we no longer require the rewrite module and thus may close the container.

Generalized Examples

Now that we have seen a site-specific version of this technique, and have gained a clearer picture as to its underlying functionality, let’s wrap things up a two generalized examples: one for root-installations of WordPress, and the other for subdirectory installations. After each example, I will describe the few edits required for proper implementation and usage.

WordPress installed in the ROOT directory

# Comprehensive URL Canonicalization for WordPress in Root
RedirectMatch permanent index.php/(.*) http://domain.tld/$1

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.domain\.tld$ [NC]
RewriteRule ^(.*)$ http://domain.tld%{REQUEST_URI} [R=301,L]

RewriteBase /

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.(html|php)\ HTTP/
RewriteRule ^(([^/]+/)*)index\.(html|php)$ http://domain.tld/$1 [R=301,L] 

RewriteCond %{REQUEST_URI} /+[^\.]+$
RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /index.php [L]
</IfModule>

To use, replace your existing root htaccess WordPress permalink rules with this code. Then, replace all instances of “domain.tld” with your domain name. That’s it. Upload and test like crazy. If everything seems like it is working, you can have some fun by entering various non-canonical URL variations and watching it work.

WordPress installed in a SUB-directory

# Comprehensive URL Canonicalization for WordPress in Subdirectory
RedirectMatch permanent index.php/(.*) http://domain.tld/subdirectory/$1

<IfModule mod_rewrite.c>
RewriteEngine On
RewriteCond %{HTTP_HOST} ^www\.domain\.tld$ [NC]
RewriteRule ^(.*)$ http://domain.tld%{REQUEST_URI} [R=301,L]

RewriteBase /subdirectory/

RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /([^/]+/)*index\.(html|php)\ HTTP/
RewriteRule ^(([^/]+/)*)index\.(html|php)$ http://domain.tld/subdirectory/$1 [R=301,L] 

RewriteCond %{REQUEST_URI} /+[^\.]+$
RewriteRule ^(.+[^/])$ %{REQUEST_URI}/ [R=301,L]

RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . /subdirectory/index.php [L]
</IfModule>

To use, replace your existing subdirectory htaccess WordPress permalink rules with this code. Then, replace all instances of “domain.tld” with your domain name. Finally, ensure that all instances of “subdirectory” with the name of your own. That’s it. Upload and test like spider pig. If everything seems like it is working, you can entertain the kids by entering various non-canonical URL variations and watching as they “magically” switch back to their correct locations.

Modifications & Variations

Canonical URL for root domain name

If you are running WordPress in a subdirectory, you may also want to ensure canonicalization of your site’s root URL. To do this, place the following code to your site’s root htaccess file:

# Remove index.php from root URL
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /index\.php [NC]
RewriteRule ^index\.php$ http://domain.tld/ [R=301,L] 

# Permanently redirect from www domain to non-www domain
RewriteCond %{HTTP_HOST} ^www\.domain\.tld$ [NC]
RewriteRule ^(.*)$ http://domain.tld/$1 [R=301,L]

Once in place, this code will ensure that only the non-www, non-index.php versions of your site’s root domain URL are served. Remember, you only need this code if you are running WordPress in a directory other than root.

Serve www version instead of non-www version of URLs

If you prefer to use the www prefix in your URLs (as opposed to omitting them), simply replace these two lines:

RewriteCond %{HTTP_HOST} ^www\.domain\.tld$ [NC]
RewriteRule ^(.*)$ http://domain.tld%{REQUEST_URI} [R=301,L]

..with these two lines:

RewriteCond %{HTTP_HOST} ^domain\.tld$ [NC]
RewriteRule ^(.*)$ http://www.domain.tld%{REQUEST_URI} [R=301,L]

..and, again, remember to replace all instances of “domain.tld” with your domain name.

Conclusion

The canonicalization solution presented in this article is both comprehensive and effective. As mentioned before, I have been using this technique on myriad sites since July of 2007 with great success. Since employing this method, I have virtually eliminated all opportunity for duplicate content to be served from my site. Of course, there are other duplicate content issues not associated with canonical URLs, but we will save that for another article :)

Hopefully, sharing these rules will open the doors to improvements. If you see something that could be improved, optimized, or otherwise enhanced, please share your wisdom with us. As an open-source community, we only benefit when everyone shares, and we all benefit when somebody does.

Footnotes

Jeff Starr
About the Author Jeff Starr = Creative thinker. Passionate about free and open Web.
Archives
36 responses
  1. The only issue with the automatic redirect that wordpress has done to “helps with” seo is that they have made it a 302 redirect when it should be a 301… so they are not helping.

    I would suggest working with your htaccess file like this post says and not leaving it up to wordpress to give you the 302 redirect

  2. Jeff Starr

    There are other reasons to use the htaccess method as well, including the fact the WordPress fails to address certain types of canonicalization issues. For example, WordPress does not redirect permalink URLs containing the index.php string to their canonical address; the HTAccess method described in my article canonicalizes all instances of index.php-laced URLs.

  3. Hi, I’ve found your article very useful but still can’t do something that seems very simple. I hope you can help because I’ve spent hours and every variation I try doesn’t seem to work!

    I have WP in the sub-directory /blog. My .htaccess file is in my root.

    Having moved over from a REALLY dated Movable Type installation, my permalinks are in this format:
    /blog/archives/123456.php (where 123456 is the postid MT assigned)

    I’ve moved to WP (v2.5.1) and want to use this format:
    /blog/archives/123456

    I’m getting errors when I try to access the old permalinks with the php extension. What rewrite rule should I add to my .htaccess? Please, please help!

  4. Update – I found a solution that worked, but it didn’t use the rewrite rules.

    RedirectMatch permanent /blog/archives/([0-9]{6}).php$ http://domain.com/blog/archives/$1

  5. Jeff Starr

    Very nice! RedirectMatch is a powerful tool for handling sensitive redirects. Thanks for posting your solution; I am glad you resolved the issue! :)

  6. Hi guys…hope someone can help me…

    I’ve got wordpress running in /htdocs/wordpress/
    and I’d like to get rid of the “/wordpress” part in the displayed url “http://www.mysite/wordpress/”, turning it in “http://www.mysite.com/”

    As i can’t edit the httpd.conf to change the DocumentRoot, I must solve the issue through the .htaccess file.
    I’ve tried a couple of solutions but none worked: i get “Permission denied” or 404 errors.

    Can anyone help?

    tnx in advance.

    F.

  7. Jeff Starr

    Hi federico: check out the first few paragraphs of this post:

    Working with Multiple Themes Outside of the WordPress Installation Directory

    ..hopefully that will give you some ideas.

  8. Hi Jeff…
    tnx so much…it worked perfectly…

    c u

    Federico

  9. Deb Phillips December 10, 2008 @ 7:53 am

    Jeff – I want to say “Thanks a million!” for this great detailed and easy-to-understand article. This has so simply solved several issues I’ve wanted to fix for my WordPress site. Thank you, thank you, thank you!

  10. Deb Phillips December 10, 2008 @ 9:13 am

    Help! I’m suddenly getting errors such as the following browser error (Safari) when clicking on all internal links (navigation, tags, categories):

    Too many redirects occurred trying to open “http://lewisvillephotos.com/tag/shallowford-square”. This might occur if you open a page that is redirected to open another page which then is redirected to open the original page.

    =====

    I’m not sure how to use the above error info. I haven’t set a separate 301 redirect on the above-referenced tag or the other links I’ve just now checked.

    Has the URL Canonicalization code caused this type of error, or is there something else going on? If so, can you suggest what might be occurring?

    Thank you!

  11. Deb Phillips December 10, 2008 @ 9:19 am

    I failed to mention in my previous “Help!” post that I’ve temporarily resorted back to a previous version of my .htaccess file. So all links on my site should be working properly at the moment. Thanks.

  12. Jeff Starr

    Hi Deb, I haven’t encountered this issue before. There are several things to look at, however, that will help diagnose the issue. First, as you suggest, thoroughly investigate any additional redirects that may be coming from elsewhere on your server. Second, does the error message appear for all pages, and if so, for which non-canonical URLs is it happening? Lastly, which version of WordPress are you using? The code has not been tested on the latest versions, so the conflict could be there as well. Oh, and one more question.. are you experiencing the error in other browsers as well, or only Safari?

[ Comments are closed for this post ]