Switching servers & migrating sites can be a HUGE deal (or not), depending on things like:

• Number of sites to transfer
• Size and complexity of sites
• Who is hosting your sites
• Experience

I recently did this, switching from a 3-year run at ASO to my new home at Media Temple. Total of 24 properties, with WordPress running on around 10 sites. Past experience with VPS servers really had me paranoid about running out of memory. A few years ago, Perishable Press alone gobbled up 256MB of RAM at WiredTree, so add another 23 sites on top of that and needless to say I was extremely concerned about the migration from a shared-hosting environment to a (dv) Base account limited to 512MB RAM.

Continue Reading

Just a note that PerishablePress.com will be migrating to its new home at Media Temple. Thanks for your patience as the DNS propagates. If you encounter anything seriously weird, like unexpected errors or anything, please comment on this post or send an email. As always, thank you for your help.

Okay – here goes nothin’!

Continue Reading

Getting the best possible web hosting is for obvious reasons something that all website owners aim to do. However, you might know what type of web hosting you are looking to get but still do not know what provider to partner up with. Many of the web hosting top lists you find on the web today look the same as they all take the same things into account. These guides usually take into account similar factors and cost usually is weighted highly. While many people do simply look for the cheapest host with decent reviews, there are others who seek slightly more from their provider. Below are some of the most sought after hosting types (dedicated, shared, vps, green) and some of the providers within those categories who may be overlooked by users but provide some excellent service.

Continue Reading

I really hate bad robots. When a web crawler, spider, bot — or whatever you want to call it — behaves in a way that is contrary to expected and/or accepted protocols, we say that the bot is acting suspiciously, behaving badly, or just acting stupid in general. Unfortunately, there are thousands — if not hundreds of thousands — of nefarious bots violating our websites every minute of the day.

For the most part, there are effective methods available enabling us to protect our sites against the endless hordes of irrelevant and mischievous bots. Such evil is easily blocked with virtually zero side-effects because their presence is simply irrelevant.

But what about bad bots that aren’t exactly irrelevant, such as Yahoo’s mindless Slurp crawler? By disobeying the robots.txt protocol as promised, Yahoo’s Slurp clearly falls into the “bad-bot” category. Unlike typical “nonsense” bots, Slurp is not exactly irrelevant (yet), so simply blocking them is not a reasonable solution.

Continue Reading

There are two possibilities here: Yahoo!’s Slurp crawler is broken or Yahoo! lies about obeying Robots directives. Either case isn’t good. Slurp just can’t seem to keep its nose out of my private business. And, as I’ve discussed before, this happens all the time. Here are the two most recent offenses, as recorded in the log file for my blackhole spider trap:

Continue Reading

As my readers know, I spend a lot of time digging through error logs, preventing attacks, and reporting results. Occasionally, some moron will pull a stunt that deserves exposure, public humiliation, and banishment. There is certainly no lack of this type of nonsense, as many of you are well-aware.

3G Blacklist

Even so, I have to admit that I am very happy with my latest strategy against crackers, spammers, and other scumbags, namely, the 3G Blacklist. Since implementing this effective HTAccess security method, I have seen a dramatic decrease in the overall volume of malicious activity recorded in my Apache, PHP, and 404 error logs. Thankfully, the 3G Blacklist has kept things pretty quiet around here, at least as far as malicious activity is concerned. In fact, it has been awhile since I’ve seen anything worthy of sharing with you — until now.

Continue Reading

Hmmm.. Let’s see here. Google can do it. MSN/Live can do it. Even Ask can do it. So why oh why can’t Yahoo’s grubby Slurp crawler manage to adhere to robots.txt crawl directives? Just when I thought Yahoo! finally figured it out, I discover more Slurp tracks in my Blackhole trap for bad spiders:

Continue Reading

I need your help! I am losing my mind trying to solve another baffling mystery. For the past three or four months, I have been recording many 404 Errors generated from msnbot, Yahoo-Slurp, and other spider crawls. These errors result from invalid requests for URLs containing query strings such as the following:

• http://perishablepress.com/press/page/2/?tag=spam
• http://perishablepress.com/press/page/3/?tag=code
• http://perishablepress.com/press/page/2/?tag=email
• http://perishablepress.com/press/page/2/?tag=xhtml
• http://perishablepress.com/press/page/4/?tag=notes
• http://perishablepress.com/press/page/2/?tag=flash
• http://perishablepress.com/press/page/2/?tag=links
• http://perishablepress.com/press/page/3/?tag=theme
• http://perishablepress.com/press/page/2/?tag=press

..plus hundreds and hundreds more ¹. The URL pattern is always the same: a different page number followed by a query string containing one of the tags used here at Perishable Press, for example: “/?tag=something”. The problem is that there are no such links anywhere on the site. The site employs permalink format for all WordPress-generated links (e.g., post/page links, search queries, tag queries, etc.). In an effort to locate the source of these URLs, I have performed multiple, thorough searches through every aspect of my site (files, database, code, examples, etc.), and the results are always the same: nothing. UGH! What is the source of these misguided URLs? Hopefully, you will be able to help shed some light on this unexplained crawl behavior (please, I beg you!!).

Continue Reading

During the recent ASO server debacle, I raced frantically to restore functionality to Perishable Press. Along the way, one of the many tricks that I tried while trying to fix the dreaded “white screen of death” syndrome involved increasing the amount of PHP memory available to WordPress. This fix worked for me, but may not prove effective on every installation of WordPress. If you are unsure as to whether or not you need to increase your PHP memory, consult with your host concerning current available memory ¹ and overall compatibility with a localized increase. Note that if your blog is running fine, there is probably no need to employ this solution. I recommend this change only if you are experiencing PHP-related and/or PHP memory issues associated with any of the following:

Continue Reading

Recently, while dealing with the dreaded white screen of death, I found myself unable to login to the WordPress Admin area to manually disable all of the plugins used here at Perishable Press. In the past, I have dealt with this situation by simply deleting all plugin files from the server, however this time, time was of the essence — I had only a few minutes with which to troubleshoot, diagnose, and ultimately resolve the deadly white-screen syndrome. Fortunately, after a few minutes of digging through the WordPress Codex, I had discovered enough information to successfully complete my mission. Now that the fiasco is over, I want to share a simple technique for quickly disabling and (re-)enabling your entire set of WordPress plugins.

Continue Reading

Just a note to announce a site upgrade to WordPress 2.3.3. The upgrade went well, but overall server performance continues to suffer. I am aware that some people are experiencing difficulties leaving comments and even accessing the site in general. Rest assured, I am working with my hosting company, A Small Orange, to get everything back on track and running smooth. In the meantime, I appreciate your patience as we work to resolve the issues, restore full functionality, and return to reliable performance.

Please share any helpful observations regarding the site here. — Thanks!

In my previous articles on PHP error handling, I explain the process whereby PHP error handling may be achieved using htaccess. Handling (logging, reporting) PHP errors via htaccess requires the following:

• Access/editing privileges for htaccess files
• A server running PHP via Apache, not CGI (e.g., phpSuExec) ¹
• Ability to edit/change permissions for files on your server

If you are having trouble handling PHP errors using htaccess, these three items are the first things to check. If it turns out that you are unable to use htaccess to work with PHP errors, don’t despair — this article explains how to achieve the same goals using local php.ini files. To implement this strategy, the following is required:

Continue Reading

In my previous article on logging PHP errors, How to Enable PHP Error Logging via htaccess, we observed three fundamental aspects of preventing, preserving, and protecting your site’s PHP errors:

Prevent public display of PHP errors via htaccess

# supress php errors
php_flag display_startup_errors off
php_flag display_errors off
php_flag html_errors off
php_value docref_root 0
php_value docref_ext 0

Preserve (log) your site’s PHP errors via htaccess

# enable PHP error logging
php_flag  log_errors on
php_value error_log  /home/path/public_html/domain/PHP_errors.log

Protect your site’s PHP error log via htaccess

# prevent access to PHP error log
<Files PHP_errors.log>
 Order allow,deny
 Deny from all
 Satisfy All
</Files>

Now, in this article, we will explore these operations ² in greater depth, provide additional functionality, and examine various implications. First we will explore PHP error handling for production environments (i.e., for websites and applications that are online, active, and public), then we will consider error handling for development environments (i.e., for projects that are under development, testing, private, etc.).

Continue Reading

Yup, ‘ol Slurp is at it again, flagrantly disobeying specific robots.txt rules forbidding access to my bad-bot trap, lovingly dubbed the “blackhole.” As many readers know, this is not the first time Yahoo has been caught behaving badly. This time, Yahoo was caught trespassing five different times via three different IPs over the course of four different days. Here is the data recorded in my site’s blackhole log (I know, that sounds terrible):

Continue Reading

Just when I thought I had finally solved my web-hosting woes by transferring to a virtual private server, I am slapped in the face by the cold realities of server memory limitations. Apparently, WordPress-powered sites are extremely resource-intensive, requiring insane amounts of random access memory (RAM), something which does not concern those of us working from shared hosting accounts.

On a shared server, system resources are shared among the various accounts that reside on a particular server. When one of these sites takes a hit and requires extra bandwidth, it “borrows” it from the total amount of bandwidth available for the other shared sites, leaving them with limited memory and other server resources. This is one of the reasons that shared hosting can suck so badly at times. If you happen to be located on a server that hosts a few resource-intensive or unstable sites, chances are high that your site will not perform as well as it might if its neighbors weren’t such stinking pigs.

With shared servers, it is this sharing of server memory that enables WordPress-powered sites to enjoy their resource-hogging plugins without too many issues. Sure, sharing memory can sometimes be a drain when you are fighting with hundreds of other sites for that extra megabyte of precious memory, but at least your site doesn’t shut down if you happen to exceed the predefined memory limits. This is exactly what happened after I setup Perishable Press on my new virtual private server at WiredTree. To be honest, since this was my first move away from shared hosting, I really had no idea how much memory my site was using. Turns out that just this one site — with its reduced number of plugins and optimized content — was enough to gobble up every drop of the 256 MB of allocated RAM without even blinking. It was like, okay, site now online — oops, not any more — you just exceeded your memory limits and crashed Apache. Again. Ugh.

Continue Reading

..And we’re back. After an insane week spent shopping for a new host, dealing with some Bad Behavior, and transferring Perishable Press to its new home on a virtual private server (VPS), everything is slowly falling back into place. Along the way, there have been some interesting challenges and many lessons learned. Here are a few of the highlights..

The tide may be turning for A Small Orange

I am certainly not alone when I say that shopping for a new hosting provider and transferring websites is one of my least favorite aspects of web development. In my experience, switching hosts requires waay too much time and rarely unfolds without significant problems. Nonetheless, when service and/or support turns sour, upgrading to a better host is well worth the effort. In my case, A Small Orange just wasn’t working out.

Everything was going fine for the first several months — excellent service, fantastic support, and consistent, reliable server uptime. However, during the last several months, server uptime frequently dipped below the 98% level, an unacceptable amount of downtime, especially since it generally happened during critical times: peak hours or while I was trying to work on the site. When I finally submitted a support ticket addressing the “unacceptable levels of downtime,” ASO support staff put my mind at ease by moving my site to a “a more stable server.”

Continue Reading

Hello! If you are seeing this page via the default WordPress theme (i.e., blue header, based on Kubrick), welcome to the new server! I recently upgraded my hosting to a virtual private server, and finally reassigned the domain name servers to resolve to the new address. So, if you are here and everything seems to be working, then everything went according to plan and I will be able to complete the transfer on Saturday (Lord willing). If, on the other hand, something is busted, throwing an error, or otherwise causing problems, then Saturday may involve more troubleshooting and fixing things.. Either way, if you are reading this, welcome! There are great things in store here at our new home ;)

Hello!

As many of you already know, the popular WordPress anti-spam plugin, Bad Behavior, caused some problems yesterday, and as a result many bloggers and users were locked out of their favorite sites, including this one. As for now, the problem seems to be fixed, however, the experience of being locked out of my own site has left a rather unpleasant taste in my mouth.

Needless to say, I will be reconsidering the continued use of Bad Behavior as a part of my long-term anti-spam strategy. In the meantime, everything is up and running fine again (with Bad Behavior disabled) — comments are working, and I can even log in to admin to write new posts :)

In other news, Perishable Press is moving to a new server! I just switched the DNS to my new VPS nameservers, so hopefully everything will propagate and transfer without a hitch. Nonetheless, if something does happen to go wrong, Perishable Press may be offline for a spell, but rest assured, I am all over it..

So — here goes nothing — I am pulling the plug on the old server — see you on the other side ;)

Okay, I realize that the title sounds a bit odd, but nowhere near as odd as my recent discovery of Slurp ignoring explicit robots.txt rules and digging around in my highly specialized bot trap, which I have lovingly dubbed “the blackhole”. What is up with that, Yahoo!? — does your Slurp spider obey robots.txt directives or not? I have never seen Google crawling around that side of town, neither has MSN nor even Ask ventured into the forbidden realms. Has anyone else experienced such unexpected behavior from one the four major search engines? Hmmm.. let’s dig a little further..

Here is the carefully formulated, highly specific, properly placed robots.txt rule that explicitly and strictly forbids all agents from accessing my blackhole bot trap:

Continue Reading

My apologies for recent periods of site unavailability. Apparently, my hosting company is having a difficult time with the server that happens to host Perishable Press. As far as I can tell, the server problems began Wednesday morning (due to a “traffic spike”). By Thursday morning, Perishable Press (as well as a few of my other sites) had been down for a period exceeding eight nonconsecutive hours.

According to my log files, Thursday itself was even worse, with server uptime struggling at an embarrassing 78.7%. And, although Friday looked considerably better, the site continues to experience periods of downtime that inspire me to begin (once again) the arduous process of switching to a more reliable host..

Currently, the site is up, but only after a lengthy period of downtime earlier this morning. I figured I had better post an explanation while I had the chance.. who knows when stability will be restored. To be fair, up until Wednesday, my hosting provider has delivered reliable, consistent uptime. I sure hope they get back into the groove.

Continue Reading

Recently, I discussed the suspicious behavior recently observed by the Yahoo! Slurp crawler. As revealed by the site’s closely watched 404-error logs, Yahoo! had been requesting a series of nonexistent resources. Although a majority of the 404 errors were exclusive to the Slurp crawler, there were several instances of requests that were also coming from Google, Live, and even Ask. Initially, these distinct errors were misdiagnosed as existing URLs appended with various JavaScript functions. Here are a few typical examples of these frequently observed log entries:

http://perishablepress.com/press/category/websites/feed/function.opendir
http://perishablepress.com/press/category/websites/feed/function.array-rand
http://perishablepress.com/press/category/websites/feed/function.mkdir
http://perishablepress.com/press/category/websites/feed/ref.outcontrol

Fortunately, an insightful reader named Bas pointed out that the errors were actually PHP functions. Bas explains:

The two functions (array_rand and opendir) you define as javascript functions are PHP functions. Some servers generate clickable links to the php manual (which uses function.NAMEOFFUNCTION in their URL’s) in php scripting error messages. Maybe that’s also the cause of these problems.

Continue Reading

Most of the time, when I catch scumbags attempting to spam, scrape, leech, or otherwise hack my site, I stitch up a new voodoo doll and let the cursing begin. No, seriously, I just blacklist the idiots. I don’t need their traffic, and so I don’t even blink while slamming the doors in their faces.

Of course, this policy presents a bit of a dilemma when the culprit is one of the four major search engines. Slamming the door on Yahoo! would be unwise, but if their Slurp crawler continues behaving suspiciously, I may have no choice. Check out the following records, pulled directly from one of my error logs, where Yahoo! exhibits some extremely questionable behavior.

Continue Reading

Perishable Press switched to A Small Orange [ASO] in March of 2007. At the time, I was looking for highly recommended shared hosting with several key features:

Update 2011/02/05: ASO is no longer my host. As this article explains, ASO service was great at the start, but after three years quality of service has declined considerably. There are some great people at ASO, but I can no longer recommend them for serious web hosting. For more information, check out my post on switching to Media Temple.</update>

• Solid customer service and extremely reliable server uptime
• Unlimited domains with plenty of disk space and bandwidth
• Unlimited Email accounts, MySQL databases and everything else

A Small Orange delivers all of the above in every shared hosting plan. ASO provides hosting plans perfect for any size operation:

Continue Reading

In this article, we extrapolate our favorite CSS-compression technique for JavaScript. Below, we outline the steps required to auto-compress your JavaScript documents via gzip and PHP. Two different compression methods are presented. The first method does not require htaccess, but rather involves the manual editing of JavaScript files. The second method employs htaccess to do all the work for you, thus requiring much less effort to implement. In either case, the result is the same: automatically compressed content delivered only to supportive browsers, resulting in reduced bandwidth, faster loading times, and smiling visitors :)

Continue Reading

Several months ago, we encountered some problems with our hosting company and decided to switch servers. Then, after spending countless hours transferring our army of domains, the new server crashed and our databases were deleted. Further, after the transfer we quickly realized the inferior technological quality of our new host. Thus disgusted, we transferred everything back to our old server and hoped for the best.

For the past several months since then, our original server has been running as well as could be expected, all things considered. All functions were running smooth, the error logs were empty, and all was well and good in cyberspace. Then, suddenly, about two weeks ago, someone gained access and indiscriminately hacked every index file on the server. Expectedly, this created chaos, crashed websites, and left our hard-working server techs scrambling to fix the hole and secure the server.

Immediately after the attack, we began repairing our websites, uploading files, restoring databases, and troubleshooting errors. Then, just as we began to make some progress, the entire server crashed, wiping out all traces of every domain on the server. Deluged with "help tickets" from many customers, our hosting company responded with a form letter indicating the problem and reaffirming us of their support (copied verbatim):

Dear Velued Customer,

today we have experienced 4 hours of downtime due to broken cPanel/RHEL update. Everyday we receive updates from cPanel and RedHat, they are automatically installed on the server. Those are critical patches, software updates etc. Today's nightly upgreade broken whole server due to incompatibility in Bind (Name servers) library. All techs have been working on the issue, it took us some time to locate the problem. In the meantime we find out that more hosting companies has such problems. Finally we were able to fix the issue and the servers are back to normal. If you experience any problems accessing your domain names it may be because you tried to access the server when DNS was down and your local ISPs DNS server couldn't cache the IP address. It may take few hours until your local ISP's DNS server refresh the DNS zone.

You can check that your web site is up and working properly through 3rd party proxy server ie. www.the-cloak.com

The issue affected ALL hosting companies which uses cPanel, for more information regarding the issue please check cPanel forums at:

http://forums[…].com (edited)

We understand your frustration and how it harmed your business however we would like to assure you that we are here 24 hours a day and 7 days a week and if there is anything wrong we will do our best to fix the issue as soon as possible.

Please accept our appology and we hope to offer you best hosting services possible.

Best regards,
Customer Service Manager

Apparently, during the process of cleaning up the aftermath of the server attack, it became necessary to upgrade various components of cPanel and other server software. Unfortunately, the upgrade produced conflicts and subsequently crashed the entire system. Ahhh yeah. Thanks for that form letter.

Several days later, after great stress and concern, the domains were once again online and accessible, enabling customers access to (once again) begin work on the restoration process. Things were finally looking up..

Well almost. After all of our websites had been restored and the dust had settled, several key applications were no longer functional. After an unsuccessful troubleshooting session, we broke down and submitted a help ticket. As it turns out, two vital PHP functions, passthru() and exec(), had been disabled due to security concerns. In other words, thanks to the cracking exploits some mindless showoff, the generous scripting privileges customers once enjoyed have now been restricted.

The good news is that, aside from the loss of a few key functions, everything else is once again up and running considerably well. Looking back, we see how the difficult, stressful, even frustrating events serve as priceless learning experiences. Indeed, managing websites is definitely a challenging endeavour, requiring great patience, flexibility, and determination.