![[ Black Hole (Figurative) ]](https://perishablepress.com/wp/wp-content/images/2010/blackhole/blackhole-04.jpg)
Welcome to the Quick Start Guide for the standalone PHP version of Blackhole for Bad Bots. This post basically is a condensed summary of the original Blackhole tutorial. So if you are new to the concept of blocking bad bots, check out the original tutorial. Otherwise, for those that are familiar, the following guide should simplify things and help you get started with Blackhole as quickly as possible. Continue reading »
Perishable Press is now over 12 years old. It is a lot of work keeping everything updated, maintained, and well-secured. Fortunately there are a gazillion free online tools for checking your site’s health. Everyone has their favorites. In this quick article, I share mine. Continue reading »
![[ xyCSS ]](https://perishablepress.com/wp/wp-content/images/2015/xycss.png)
Recently I’ve been implementing SSL on my domains and have been streamlining and updating some projects along the way. Consolidating properties is a great way to simplify workflow and boost productivity, so I’ve went ahead and moved xyCSS from its own domain, xy.css, to its new home here at Perishable Press. Continue reading »
![[ 2013 User Agent Blacklist ]](/wp/wp-content/images/2013/user-agent-blacklist.png)
The 2013 User Agent Blacklist blocks hundreds of the worst bots while ensuring open-access for normal traffic, major search engines (Google, Bing, et al), good browsers (Chrome, Firefox, Opera, et al), and everyone else. Compared to blocking threats by IP, blocking by user-agent is more effective as a general security strategy. Although it’s trivial to spoof any user agent, many bad requests continue to report user-agent strings that are known to be associated with malicious activity. For example, the notorious […] Continue reading »
![[ WP-Mix ]](https://perishablepress.com/wp/wp-content/images/2013/wpmix_150x150.png)
Wrapping up 2012, I finally launched xyCSS, which is all about responsive, grid-based design. To showcase xy.css, I used it to design WP-Mix.com, which also serves to house a growing collection of choice code snippets. Currently WP-Mix features over 100 snippets, tutorials, and other useful bits to help with WordPress development and web design in general. The topics are similar to those at Perishable Press (e.g., WordPress, PHP, JavaScript, CSS, etc.), but the posts are less-involved and aimed at intermediate […] Continue reading »
![[ xy.css - Homepage ]](https://perishablepress.com/wp/wp-content/images/2013/xycss-01.jpg)
For the past year or so, I’ve been heavy into responsive, grid-based design. In December, I “soft-launched” my new site, xyCSS with a simple tweet: Bringing it all together: https://perishablepress.com/xycss/ As implied (and explained), xy.css is a lightweight CSS template for creating semantic HTML5 designs on a responsive liquid matrix. Continue reading »
I am in the process of migrating my sites from A Small Orange to Media Temple. Part of that process involves canonicalizing domain URLs to help maximize SEO strategy. At ASO, URL canonicalization required just a few htaccess directives: # enforce no www prefix <ifmodule mod_rewrite.c> RewriteCond %{HTTP_HOST} !^domain\.tld$ [NC] RewriteRule ^(.*)$ http://domain.tld/$1 [R=301,L] </ifmodule> When placed in the web-accessible root directory’s htaccess file, that snippet will ensure that all requests for your site are not prefixed with www. There’s […] Continue reading »
Recently cleared several megabytes of log files, detecting patterns, recording anomalies, and blacklisting gross offenders. Gonna break it down into three sections: User Agents Character Strings IP Addresses User Agents User-agents come and go, and are easily spoofed, but it’s worth a few lines of htaccess to block the more persistent bots that repeatedly scan your site with malicious requests. # Nov 2010 User Agents SetEnvIfNoCase User-Agent "MaMa " keep_out SetEnvIfNoCase User-Agent "choppy" keep_out SetEnvIfNoCase User-Agent "heritrix" keep_out SetEnvIfNoCase User-Agent […] Continue reading »
Chris Coyier of CSS-Tricks recently declared that people should do “nothing” in response to other sites scraping their content. I totally get what Chris is saying here. He is basically saying that the original source of content is better than scrapers because: it’s on a domain with more trust. you published that article first. it’s coded better for SEO than theirs. it’s better designed than theirs. it isn’t at risk for serious penalization from search engines. If these things are […] Continue reading »
![[ 2010 User-Agent Blacklist ]](https://perishablepress.com/wp/wp-content/images/2010/misc-chunks/user-agent-blacklist.png)
The 2010 User-Agent Blacklist blocks hundreds of bad bots while ensuring open-access for the major search engines: Google, Bing, Ask, Yahoo, et al. Blocking bad user-agents is an effective addition to any security strategy. It works like this: your site is getting hammered by rogue bots that waste valuable server resources and bandwidth. So you grab a copy of the 2010 UA Blacklist from Perishable Press, include it in your site’s root .htaccess file, and enjoy better security and performance. […] Continue reading »
Over the course of each year, I blacklist a considerable number of individual IP addresses. Every day, Perishable Press is hit with countless numbers of spammers, scrapers, crackers and all sorts of other hapless turds. Weekly examinations of my site’s error logs enable me to filter through the chaff and cherry-pick only the most heinous, nefarious attackers for blacklisting. Minor offenses are generally dismissed, but the evil bastards that insist on wasting resources running redundant automated scripts are immediately investigated […] Continue reading »
I mean, basically anything except for Internet Explorer, which is a debate in and of itself. Here I’m referring to old versions of good browsers, like Firefox 2, Safari 2, Opera 8, and so on. It seems that older versions of these browsers are not as common as older versions of IE, so should we bother supporting them when designing our websites? Most agree that we shouldn’t support old versions of crappy browsers like IE, but what about older versions […] Continue reading »
There are several ways to instruct Google to stay away from various pages in your site: Robots.txt directives Nofollow attributes on links Meta noindex/nofollow directives X-Robots noindex/nofollow directives ..and so on. These directives all function in different ways, but they all serve the same basic purpose: control how Google crawls the various pages on your site. For example, you can use meta noindex to instruct Google not to index your sitemap, RSS feed, or any other page you wish. This […] Continue reading »
Given my propensity to discuss matters involving error log data (e.g., monitoring malicious behavior, setting up error logs, and creating extensive blacklists), I am often asked about the best way to go about monitoring 404 and other types of server errors. While I consider myself to be a novice in this arena (there are far brighter people with much greater experience), I do spend a lot of time digging through log entries and analyzing data. So, when asked recently about […] Continue reading »
You have seen user-agent blacklists, IP blacklists, 4G Blacklists, and everything in between. Now, in this article, for your sheer and utter amusement, I present a collection of over 8000 blacklisted referrers. Shortcut: skip the article and jump to Disclaimer and Download » Referrer Spam Sucks For the uninitiated, in teh language of teh Web, a referrer is the online resource from whence a visitor happened to arrive at your site. For example, if Johnny the Wonder Parrot was visiting the […] Continue reading »
![[ Image: Inverted Eclipse ]](https://perishablepress.com/wp/wp-content/images/2009/agent-blacklist/inverted-eclipse.jpg)
As discussed in my recent article, Eight Ways to Blacklist with Apache’s mod_rewrite, one method of stopping spammers, scrapers, email harvesters, and malicious bots is to blacklist their associated user agents. Apache enables us to target bad user agents by testing the user-agent string against a predefined blacklist of unwanted visitors. Any bot identifying itself as one of the blacklisted agents is immediately and quietly denied access. While this certainly isn’t the most effective method of securing your site against […] Continue reading »
