Block Bad Bots with Blackhole Pro + Save 25% with code: CENTAURUS Get plugin »
Tag Archive

Redirecting URLs that Include Numbers

Redirecting stuff with .htaccess generally is pretty straightforward, but there can be a lot of confusion when it comes to targeting patterns that include numbers. I think this largely is due to the syntax used for matching numbers in regular expressions. It’s sort of unintuitive until you get the hang of it. So to help in that regard, this tutorial explains the basics of matching numbers with .htaccess, and then provides some useful examples that should get you there. Read more »

Redirecting Hash Fragments with .htaccess

During this year’s site redesigns, I noticed in the server logs some 404 errors for various WordPress comments. These 404 requests each involved a fragment identifier (i.e., character string beginning with a pound sign, #) being interpreted as its HTML entity hex equivalent, %23. It may not seem like a big deal, but these days every detail counts, so it’s wise to clean up as many 404 errors as possible. Thus, here is a simple .htaccess technique for redirecting hash-fragment requests to their proper destination. Read more »

Stop 404 Requests for Mobile Versions of Your Site

If you’ve been keeping an eye on your 404 errors recently, you will have noticed an increase in requests for nonexistent mobile files and directories, especially over the past year or so. The scripts and bots requesting these files from your server seem to be looking for a mobile version of your site. Unfortunately, they are wasting bandwidth and resources in the process. It has become common to see the following 404 errors constantly repeated in your log files: http://domain.tld/apple-touch-icon.png http://domain.tld/iphone http://domain.tld/mobile http://domain.tld/mobi http://domain.tld/m So some bot comes along, assumes that your site includes a mobile version, and then tries […] Read more »

Pimp Your 404: Presentation and Functionality

I have been wanting to write about 404 error pages for quite awhile now. They have always been very important to me, with customized error pages playing a integral part of every well-rounded web-design strategy. Rather than try to re-invent the wheel with this, I think I will just go through and discuss some thoughts about 404 error pages, share some useful code snippets, and highlight some suggested resources along the way. In a sense, this post is nothing more than a giant “brain-dump” of all things 404 for future reference. Hopefully you will find it useful in pimping your […] Read more »

Best Practices for Error Monitoring

Given my propensity to discuss matters involving error log data (e.g., monitoring malicious behavior, setting up error logs, and creating extensive blacklists), I am often asked about the best way to go about monitoring 404 and other types of server errors. While I consider myself to be a novice in this arena (there are far brighter people with much greater experience), I do spend a lot of time digging through log entries and analyzing data. So, when asked recently about my error monitoring practices, I decided to share my response here at Perishable Press, and hopefully get some good feedback […] Read more »

Plenty of Errors to Chew On..

Alrighty then! Looks like recent changes to site structure have really dropped a bomb on quite a few regular visitors out there. After switching over to the new default theme last night, I had setup an email notification system to alert me of all errors encountered at this domain (i.e., the main site and all test sites). Of course, I knew that at least a few errors would be inevitable, but I had no idea that I would receive nearly 300 of them! After examining the nature of these errors, it appears that the bulk of them are the result […] Read more »

Eliminate 404 Errors for PHP Functions

Recently, I discussed the suspicious behavior recently observed by the Yahoo! Slurp crawler. As revealed by the site’s closely watched 404-error logs, Yahoo! had been requesting a series of nonexistent resources. Although a majority of the 404 errors were exclusive to the Slurp crawler, there were several instances of requests that were also coming from Google, Live, and even Ask. Initially, these distinct errors were misdiagnosed as existing URLs appended with various JavaScript functions. Here are a few typical examples of these frequently observed log entries: Fortunately, an insightful reader named Bas pointed out that the […] Read more »

Suspicious Behavior from Yahoo! Slurp Crawler

Most of the time, when I catch scumbags attempting to spam, scrape, leech, or otherwise hack my site, I stitch up a new voodoo doll and let the cursing begin. No, seriously, I just blacklist the idiots. I don’t need their traffic, and so I don’t even blink while slamming the doors in their faces. Of course, this policy presents a bit of a dilemma when the culprit is one of the four major search engines. Slamming the door on Yahoo! would be unwise, but if their Slurp crawler continues behaving suspiciously, I may have no choice. Check out the […] Read more »

Standards-Compliance Throwdown: MS-IE5/6 DNS/404 Error-Page Redesign

Default DNS Error page for Internet Explorer First of all, congratulations if you are geeky enough to understand the title of this article. Many would be like, "CSS, MS.. IE, error ..what..?" Whatever. If you get the title, you will get the point of this utterly pointless exercise. If that is the case, prepare for a delightful romp through geekland. Otherwise, save your precious time and stop reading here (exit strategy). Well, okay, for the seriously unenlightened, let us explain the object of our present focus: The default "DNS Error" page for Internet Explorer unfortunately remains a familiar sight for millions […] Read more »

Disobedient Robots and Company

In our never-ending battle against spammers, leeches, scrapers, and other online undesirables, we have implemented several powerful security measures to improve the operational integrity of our perpetual virtual existence. Here is a rundown of the new behind-the-scenes security features of Perishable Press. Read more »

Latest Tweets How to block bad bots:…