Unexplained Crawl Behavior Involving Tagged Query Strings
I need your help! I am losing my mind trying to solve another baffling mystery. For the past three or four months, I have been recording many 404 Errors generated from msnbot, Yahoo-Slurp, and other spider crawls. These errors result from invalid requests for URLs containing query strings such as the following: https://example.com/press/page/2/?tag=spam https://example.com/press/page/3/?tag=code https://example.com/press/page/2/?tag=email https://example.com/press/page/2/?tag=xhtml https://example.com/press/page/4/?tag=notes https://example.com/press/page/2/?tag=flash https://example.com/press/page/2/?tag=links https://example.com/press/page/3/?tag=theme https://example.com/press/page/2/?tag=press Note: For these example URLs, I replaced my domain, perishablepress.com with the generic example.com. Turns out that listing the plain-text […] Continue reading »
Taking Advantage of the X-Robots Tag
Controlling the spidering, indexing and caching of your (X)HTML-based web pages is possible with meta robots directives such as these: <meta name="googlebot" content="index,archive,follow,noodp"/> <meta name="robots" content="all,index,follow"/> <meta name="msnbot" content="all,index,follow"/> I use these directives here at Perishable Press and they continue to serve me well for controlling how the “big bots”1 crawl and represent my (X)HTML-based content in search results. For other, non-(X)HTML types of content, however, using meta robots directives to control indexing and caching is not an option. An […] Continue reading »
WordPress Tip: Quick Hack to Block Spam for the Wordspew Shoutbox Chat Plugin
Recently, I reactivated an older version (1.16) of Jalenack’s Wordspew Shoutbox plugin for the Dead Letter Art Chat Forum. The DLa collective has been working on a new issue of their ‘zine and needed an easy online chat location for impromptu business dealz (ideas, planning, etc.). Almost immediately after reactivating the Shoutbox plugin, the chat forum was flooded with an endless wave of spam. The rate and volume of spam was so high as to render the forum utterly useless. […] Continue reading »
Obsessive CSS Code Formatting: Indentation and Spacing
In the intriguing discussion following the first obsessive CSS formatting article, Jordan Gray brought up the age-old question regarding spacing: tabs or single spaces? I smugly responded that the issue has long-since been resolved, with tabbed spacing as the obvious winner. Let’s take a look at some serious CSS spacing examples.. 1) Strictly Single Spacing Here we have several code blocks showing consistent spacing via single blank space. Three key areas where single spacing is seen in this example: after […] Continue reading »
