Perishable Press

Perishable Press 3G Blacklist

After much research and discussion, I have developed a concise, lightweight security strategy for Apache-powered websites. Prior to the development of this strategy, I relied on several extensive blacklists to protect my sites against malicious user agents and ...

Building the 3G Blacklist, Part 5: Improving Site Security by Selectively Blocking Individual IPs

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. Wrapping up the series with this article, I provide the final key to our comprehensive blacklist strategy: selectively blocking individual IPs. ...

Building the 3G Blacklist, Part 4: Improving the RedirectMatch Directives of the Original 2G Blacklist

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this fourth article, I build upon previous ideas and techniques by improving the directives contained in the original, 2G Blacklist. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and ...

Building the 3G Blacklist, Part 3: Improving Site Security by Selectively Blocking Rogue User Agents

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this third article, I discuss targeted, user-agent blacklisting and present an alternate approach to preventing site access for the most prevalent and malicious user agents. Subsequent articles will focus on key blacklist strategies designed to protect your ...

Building the 3G Blacklist, Part 2: Improving Site Security by Preventing Malicious Query-String Exploits

In this continuing five-article series, I share insights and discoveries concerning website security and protecting against malicious attacks. In this second article, I present an incredibly powerful method for eliminating malicious query string exploits. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion ...

Building the 3G Blacklist, Part 1: Improving Site Security by Recognizing and Exploiting Server Attack Patterns

In this series of five articles, I share insights and discoveries concerning website security and protecting against malicious attacks. In this first article of the series, I examine the process of identifying attack trends and using them to immunize against future attacks. Subsequent articles will focus on key blacklist strategies designed to protect your site transparently, effectively, and efficiently. At the conclusion of the series, the five articles will culminate in ...

Universal www-Canonicalization via htaccess

During my previous rendezvous involving comprehensive canonicalization for WordPress, I offer my personally customized technique for ensuring consistently precise and accurate URL delivery. That particular method targets WordPress exclusively (although the logic could be manipulated for general use), and requires a bit of editing to adapt the code to each particular configuration. In this follow-up tutorial, I present ...

Blacklist Candidate Number 2008-04-27

Welcome to the Perishable Press “Blacklist Candidate” series. In this post, we continue our new tradition of exposing, humiliating and banishing spammers, crackers and other worthless scumbags.. Since the implementation of my 2G Blacklist, I have enjoyed ...

How to Block Proxy Servers via htaccess

Not too long ago, a reader going by the name of bjarbj78 asked about how to block proxy servers from accessing her website. Apparently, bjarbj78 had taken the time to compile a proxy blacklist of over 9,000 domains, only to discover afterwards that the formulated htaccess blacklisting strategy didn’t work as expected: deny from proxydomain.com proxydomain2.com Blacklisting proxy servers by blocking individual domains seems like a futile exercise. Although there are ...

Three Unsolved WordPress Mysteries

After several years of using WordPress, I have at least three unanswered questions: What’s up with the WordPress PHP Memory Error? Why do certain phrases trigger “Forbidden” errors when saving or publishing posts? What happened to the Plugin Pages in the WordPress Codex? Let’s have a look at each one of these baffling mysteries.. Unsolved Mystery #1: What’s up with the WordPress PHP Memory Error? Every single day, WordPress generates hundreds of these errors: [20-Feb-2008 19:49:42] PHP ...

Content Negotiation for XHTML Documents via PHP and htaccess

In this article, I discuss the different MIME types available for XHTML and explain a method for serving your documents with the optimal MIME type, depending on the capacity of the user agent. Using either htaccess or PHP for content negotiation, we can serve complete, standards-compliant markup for our document’s header information. This ...

Redirect WordPress Feeds to Feedburner via htaccess (Redux)

In a previous article, I explain how to redirect your WordPress feeds to Feedburner. Redirecting your WordPress feeds to Feedburner enables you to take advantage of their many freely provided, highly useful tracking and statistical services. Although there are a few important things to ...

Custom HTTP Errors via htaccess

We all know how important it is to deliver sensible, helpful 404 error pages to our visitors. There are many ways of achieving this functionality, including the well-known htaccess trick used to locally redirect users to custom error pages: # htaccess custom error pages ErrorDocument 400 /errors/400.html ErrorDocument 401 /errors/401.html ErrorDocument 403 /errors/403.html ErrorDocument 404 /errors/404.html ErrorDocument 500 /errors/500.html ..and so on. These directives basically tell Apache to deliver the designated documents for their associated error ...

Blacklist Candidate Number 2008-03-09

Welcome to the Perishable Press “Blacklist Candidate” series. In this post, we continue our new tradition of exposing, humiliating and banishing spammers, crackers and other worthless scumbags.. Imagine, if you will, an overly caffeinated Bob Barker, hunched over his favorite laptop, feverishly scanning his server access files. Like some underpaid ...

2G Blacklist: Closing the Door on Malicious Attacks

Since posting the Ultimate htaccess Blacklist and then the Ultimate htaccess Blacklist 2, I find myself dealing with a new breed of malicious attacks. It is no longer useful to simply block nefarious user agents because they are frequently faked. Likewise, ...

Over 150 of the Worst Spammers, Scrapers and Crackers from 2007

Over the course of each year, I blacklist a considerable number of individual IP addresses. Every day, Perishable Press is hit with countless numbers of spammers, scrapers, crackers and all sorts of other hapless turds. Weekly examinations ...

Improve Site Performance by Increasing PHP Memory for WordPress

During the recent ASO server debacle, I raced frantically to restore functionality to Perishable Press. Along the way, one of the many tricks that I tried while trying to fix the dreaded “white screen of death” syndrome involved increasing the amount of PHP memory available to WordPress. This fix worked for me, but may not prove effective ...

WordPress Error Fix(?): Increase PHP Memory for cache.php

This trick isn’t guaranteed to prevent all WordPress-generated PHP memory errors, but it certainly seems to help reduce their overall occurrence. For some reason, after my host upgraded their servers to Apache 1.3.41, I began logging an extremely high number of fatal PHP “memory exhausted” errors resulting from the WordPress cache.php script. Here is an example of the countless errors that are generated: [17-Feb-2008 19:26:38] PHP Fatal error: Allowed memory size ...

Blacklist Candidate Number 2008-02-10

Welcome to the Perishable Press “Blacklist Candidate” series. In this post, we continue our new tradition of exposing, humiliating and banishing spammers, crackers and other worthless scumbags.. Scumbag number 2008-02-10, “COME ON DOWN!!” — you’re the next baboon to get banished from the site! Like many bloggers, I ...

Permalink Evolution: Customize and Optimize Your Dated WordPress Permalinks

How to streamline and maximize the effectiveness of your WordPress URLs by using htaccess to remove extraneous post-date information: years, months, and days.. Recently, there has been much discussion about whether or not to remove the post-date information from WordPress permalinks 1. Way back during the WordPress ...

Advanced PHP Error Handling via PHP

In my previous articles on PHP error handling, I explain the process whereby PHP error handling may be achieved using htaccess. Handling (logging, reporting) PHP errors via htaccess requires the following: Access/editing privileges for htaccess files A server running ...

1-Minute Tutorial: Permanent (301) Redirect via PHP or htaccess

Here is an example of one of the most frequently asked PHP/htaccess-related questions I receive here at Perishable Press: How do I redirect a specific page/URL using PHP/htaccess? So common is this inquiry that I have decided to just post a quick, “one-minute” tutorial describing the technique. Permanent (301) Redirect via PHP To permanently redirect a ...

Comprehensive URL Canonicalization via htaccess for WordPress-Powered Sites

Permalink URL canonicalization is automated via PHP in WordPress 2.3+, however, for those of us running sites on pre-2.3 versions or preferring to deal with rewrites directly via Apache, comprehensive WordPress URL canonicalization via htaccess may seem impossible. While there are several common methods that are partially effective, there has not yet ...

Advanced PHP Error Handling via htaccess

In my previous article on logging PHP errors, How to Enable PHP Error Logging via htaccess, we observed three fundamental aspects of preventing, preserving, and protecting your site’s PHP errors: Prevent public display of PHP errors via htaccess # supress php errors php_flag display_startup_errors off php_flag display_errors off php_flag html_errors off php_value docref_root 0 php_value docref_ext 0 Preserve (log) ...

Blacklist Candidate Number 2008-01-02

Come one, come all — today we officially begin a new series of posts here at Perishable Press: the public exposure, humiliation, and banishment of spammers, crackers, and other site attackers. Kicking things off for 2008: blacklist candidate number 2008-01-02! Every Wednesday, I take a little time to investigate my ...

Optimize WordPress: Pure Code Alternatives for 7 Unnecessary Plugins

In this article, my goal is to help you optimize WordPress by replacing a few common plugins with their correspondingly effective code equivalents. As we all know, WordPress can be a very resource-hungry piece of software, especially when running a million extraneous plugins. Often, many common plugins are designed to perform relatively simple tasks, such as redirect ...

How to Enable PHP Error Logging via htaccess

In this brief tutorial, I will show Apache users how to suppress PHP errors from visitors and enable PHP error logging via htaccess. Tracking your site’s PHP errors is an excellent way to manage and troubleshoot unexpected issues related to plugins and themes. Even better, monitoring PHP errors behind the scenes ...

A Dramatic Week Here at Perishable Press..

..And we’re back. After an insane week spent shopping for a new host, dealing with some Bad Behavior, and transferring Perishable Press to its new home on a virtual private server (VPS), everything is slowly falling back into place. Along the way, there have been some interesting challenges and many lessons learned. Here are a few of the highlights.. The tide may be turning ...

Three Ways to Allow Hotlinking in Specific Directories

After implementing any of the hotlink-prevention techniques described in our previous article, you may find it necessary to disable hotlink-protection for a specific directory. By default, htaccess rules apply to the directory in which it is located, as well as all subdirectories contained therein. There are (at least) three ways to enable selective hotlinking: Place hotlink images in an alternate directory This method ...

Stupid htaccess Trick: Enable File or Directory Access to Your Password-Protected Site

In this brief tutorial, we are going to enable users to access any file or directory of a site that is password-protected via htaccess. There are many reasons for wanting to employ this technique, including: Share public resources from an otherwise private site Enable visitors to access content during site maintenance Testing and formatting of layout and design during development As a webmaster, I have used this technique on several ...

Creating the Ultimate htaccess Anti-Hotlinking Strategy

When I wrote my article, Stupid htaccess Tricks, a couple of years ago, hotlink-protection via htaccess was becoming very popular. Many webmasters and bloggers were getting tired of wasting bandwidth on hotlinked resources, and therefore turned to the power of htaccess to protect their content. At that time, there were only ...

Ultimate htaccess Blacklist 2 (Compressed Version)

[ Keywords: htaccess, rewrite, blacklist, block, deny, spam, spammers, scrapers, rippers ] In our original htaccess blacklist article, we provide an extensive list of bad user agents. This so-called “Ultimate htaccess Blacklist” works great at blocking many different online villains: spammers, scammers, scrapers, scrappers, rippers, leechers — you name it. Yet, despite its usefulness, there is always room for improvement. For example, as reader ...

How to Verify the Four Major Search Engines

Keeping track of your access and error logs is a critical component of any serious security strategy. Many times, you will see a recorded entry that looks legitimate, such that it may easily be dismissed as genuine Google fare, only to discover upon closer investigation a fraudulent agent. There are many such cloaked or disguised agents crawling around these days, mimicking various search engines to hide beneath the radar. Thus, it ...

htaccess Combo Pack: WordPress Permalinks and non-www Redirect

WordPress users employing permalinks via htaccess to optimize their dynamic URLs transform complicated-looking links such as: http://example.com/blog/index.php?page=33 ..into search-engine friendly links such as: http://example.com/blog/post-title/ Every rewritten URL relies on a common set of htaccess rules to transform the links. The htaccess rules for all WordPress permalinks look like this for root WP installations: # BEGIN WordPress RewriteEngine On RewriteBase / RewriteCond %{REQUEST_FILENAME} !-f RewriteCond ...

Eliminate 404 Errors for PHP Functions

Recently, I discussed the suspicious behavior recently observed by the Yahoo! Slurp crawler. As revealed by the site’s closely watched 404-error logs, Yahoo! had been requesting a series of nonexistent resources. Although a majority of the 404 errors were exclusive to the Slurp crawler, there were several instances of requests that were also coming from Google, Live, and even Ask. Initially, these distinct errors were misdiagnosed as existing ...

Temporary Site Redirect for Visitors during Site Updates

[ Keywords: temporary, redirect, htaccess, php, site, maintenance, 403, 503, http, status, code ] In our article Stupid htaccess Tricks, we present the htaccess code required for redirecting visitors temporarily during periods of site maintenance. Although the article provides everything needed to implement the temporary redirect, I think readers would benefit from a more thorough examination of the process — nothing too serious, just ...

Permanently Redirect a Specific IP Request for a Single Page via htaccess

Not the most interesting title, but “oh well”.. Recently, a reader named Alison left a comment requesting help with a particular htaccess trick. She wanted to know how to permanently redirect (301) all requests for a specific page when requested from a specific IP address. In other words, when a visitor coming from 123.456.789 requests the page requested-page.html, the visitor will be redirected to just-for-you.html. All visitors not ...

Allow Google Reader to Access Hotlink-Protected Images

[ google, reader, mobile, htaccess, allow, access, hotlink, protection, images, rewrite ] In our previous article, we explain the process of allowing Feedburner to access your hotlink-protected images. The article details the entire process, which covers the basics of hotlink protection and involves adding several lines of code to your htaccess file. In this article, we skip the detailed explanations and ...

Allow Feedburner Access to Hotlink-Protected Images

[ Keywords: feedburner, htaccess, allow, access, hotlink, protection, feed, images, rewrite ] Recently, we installed and configured the excellent WordPress Feedburner plugin by the venerable Steve Smith 1. The plugin basically redirects our various WordPress-powered content feeds to Feedburner, which then delivers them to subscribers. This method enables us to take advantage of Feedburner’s excellent statistical tools. Further, all of the action ...

How to Block IP Addresses with PHP

[ Keywords: block, deny, ip, address, php, spam, htaccess, redirect ] Figuratively speaking, hunting down and killing spammers, scrapers, and other online scum remains one of our favorite pursuits. Once we have determined that a particular IP address is worthy of banishment, we generally invoke the magical powers of ...

Major Problem with cPanel Hotlink Protection and htaccess

[ Keywords: cpanel, hotlink, protection, htaccess, rewrite, rules, problem, bug, issue ] There is a major problem with the “Hotlink Protection” feature of cPanel. To summarize the issue, allow me to quote a recent email sent to a completely unresponsive tech support department: ...The problem is that if I try to include any rewrite rules for permalinks, hotlinking, or blocking spambots, cPanel automatically enables its “Hotlink Protection” feature. And, even worse, it automatically adds every URL from every ...

Ultimate htaccess Blacklist

[ Keywords: htaccess, rewrite, blacklist, block, deny, spam, spammers, scrapers, rippers ] For those of us running Apache, htaccess rewrite rules provide an excellent way to block spammers, scrapers, and other scumbags easily and effectively. While there are many htaccess tricks involving blocking domains, preventing access, and redirecting traffic, ...

Compressed JavaScript Compression

[ Keywords: javascript, compression, compress, php, htaccess, gzip, ob_gzhandler ] In this article, we extrapolate our favorite CSS-compression technique for JavaScript. Below, we outline the steps required to auto-compress your JavaScript documents via gzip and PHP. Two different compression methods are presented. The first method does not require htaccess, but rather involves the manual editing of JavaScript files. The second method employs htaccess to do ...

Fast, Effective PHP Compression

PHP compression is an excellent method of conserving bandwidth and reducing client download times. We have already discussed an excellent method for CSS compression, and in this article we share a super-easy technique for compressing all PHP content without editing a single file. Using two PHP files and two corresponding lines of .htaccess, it is possible to compress your PHP files via gzip-encoding. Browsers and other user-agents capable of ...

Perishable Press Redirection Lounge

Welcome to the Perishable Press Redirection Lounge!

Invite Only: Visitor Exclusivity via the Opt-In Method

Web developers trying to control comment-spam, bandwidth-theft, and content-scraping must choose between two fundamentally different approaches: selectively deny target offenders (the "blacklist" method) or selectively allow desirable agents (the "opt-in", or "whitelist" method). Currently popular according to various online forums and discussion boards is the blacklist method. The blacklist method requires the webmaster to create and maintain a working list of undesirable agents, usually blocking their access via htaccess or php. The downside of "blacklisting" is that ...

Riding the Wave

Compared to some of the big players out there on the internet, we here at Perishable Press run a relatively small website. We began this project in September of 2005 with nothing but a domain name and a pocketful of inspiration. During the first several months of development, our traffic statistics looked something like: one unique visitor and 10,000 hits (i.e., nobody but us). Well, that went on for the first few months of 2006, and then something ...

Disobedient Robots and Company

In our never-ending battle against spammers, leeches, scrapers, and other online undesirables, we have implemented several powerful security measures to improve the operational integrity of our perpetual virtual existence. Here is a rundown of the new behind-the-scenes security features of Perishable Press: Automated spambot trap, designed to identify bots (and/or stupid people) that disobey rules specified in the site’s robots.txt file. Automated disobedient-robot identification (via reverse IP lookup), admin-notification (via email) and blacklist inclusion (via htaccess). Automated inclusion of ...

Automatic Language Translation Methods

As you may have noticed, Perishable Press recently added automatic language translation to each of our articles. The free, automatic translations are available as a series of image links (via corresponding country flag icons) next to each article’s individual post view. We have found that providing this free service is important as many of our visitors come from countries other than the United States, and therefore may be unable to read our articles as presented in the ...

Roll your own Apache Rewrite Log

Note to self: Here is the .htaccess code for creating an Apache Rewrite log. This is definitely helpful for testing .htaccess rewrite rules, permalinks, etc. Update: as jc points out in the comments section, Apache versions 2.2 or better no longer allow RewriteLog directives in htaccess files. For these versions, it is necessary to coordinate logging through the server configuration file or via virtual host. For more information, check the source. For ...

« Reverse • Up •

Tools & Resources

3G Blacklist finally unleashed!