New Bookstore! Save 20% on books with discount code: LAUNCH
Web Dev + WordPress + Security
87 posts related to: Cleaning Up Google Search Results

List of All User Agents for Top Search Engines

Here is a working list of all user agents for the major, top search engines. I use this information frequently for my plugins such as Blackhole for Bad Bots and BBQ Pro, so I figured it would be useful to post the information online for the benefit of others. Having the user agents for these popular bots all in one place helps to streamline my development process. Each search engine includes references and a regex pattern to match all known […] Continue reading »

6G Firewall 2020

[ 6G Firewall ]

After three years of development, testing, and feedback, I’m pleased to announce the official launch version of the 6G Firewall (aka the 6G Blacklist). This version of the nG Firewall is greatly refined, heavily tested, and better than ever. Fine-tuned to minimize false positives, the 6G Firewall protects your site against a wide variety of malicious URI requests, bad bots, spam referrers, and other attacks. Blocking bad traffic improves site security, reduces server load, and conserves precious resources. The 6G […] Continue reading »

6G Firewall Beta

[ 6G Blacklist (beta) ]

Since releasing the 5G Blacklist earlier this year, malicious server scans and bad requests have surged with more novel attacks than I’ve seen since first getting into this stuff six years ago. In other words, now is the time to beef up security and lock things down. If you’re into monitoring your server and knowing your traffic, you may be observing the same recent spike in malicious activity. In response to these attacks, I’ve been secretly working on the next […] Continue reading »

Add Google+ Share Button to Any Site

[ Google+ Share Button

g+ Share button Word on the streets is that the new Google+ Share button is the best way yet to benefit from Google’s myriad social-media services and all-important search-engine. And Google makes it SO easy to add the new Share button to your website. This article explains what it is, where it fits in with all the other social-Google stuff, and of course how to add the g+ Share button to any site. Continue reading »

Blank Space / Whitespace Character for .htaccess

Working on the next version of the G-Series Blacklist, I needed a way to match a wide variety of UTF-8-encoded (hex) character strings. Those familiar with their site’s traffic will recognize this particular type of URI request string, which is typically associated with malicious server scanning, exploits, and other malicious behavior. As I explain in this post, pattern-matching and blocking the blank-space, or whitespace character in URL-requests is an effective way to improve the security of your website. Continue reading »

Multiple Sitemaps

[ Screenshot: Settings for Google XML Sitemaps ]

Yes you can have multiple sitemaps for your site. Create the sitemaps you need, and then specify them in your robots.txt file. For example, here are the robots.txt directives for the two sitemaps used here at Perishable Press: Continue reading »

Hacked by Google?

[ WP Cron HTTP Auth ]

The setup: I recently launched a new plugin that included a Demo page. To keep things flexible, I set up the Demo as a page on my experimental “Labs” WordPress installation, which is entirely nofollow, noindex and noarchive, meaning that Google can’t legitimately see what’s there. Continue reading »

Clean Up Malicious Links with HTAccess

[ Screenshot: Google Search Results ]

I recently spent some time analyzing Perishable Press pages as they appear in the search results for Google, Bing, et al. Google Webmaster Tools provides a wealth of information about crawl errors, as well as the URLs of any pages that link to missing content. Combined with your site’s access/error logs, you have everything needed to track down 404 errors and clean up your listings in the search engine results. Continue reading »

Better Robots.txt Rules for WordPress

[ Better Robots.txt Rules for WP ]

Cleaning up my files during the recent redesign, I realized that several years had somehow passed since the last time I even looked at the site’s robots.txt file. I guess that’s a good thing, but with all of the changes to site structure and content, it was time again for a delightful romp through robots.txt. This post summarizes my research and gives you a near-perfect robots file, so you can copy/paste completely “as-is”, or use a template to give you […] Continue reading »

Simple IP-Detection Bad for SEO

In general, Perishable Press enjoys generous ranking in Google’s search-engine results. The site’s many pages bring in lots of traffic for some great keywords, and a direct search for “Perishable Press” returns the first spot, with eight featured site links even. And recently, after switching servers, traffic increased even further. Things were going well, and it seemed like the perfect opportunity to finally renovate and redesign the site. So I dive in.. And then approximately 24-48 hours after beginning work […] Continue reading »

htaccess Redirect to Maintenance Page

Redirecting visitors to a maintenance page or other temporary page is an essential tool to have in your tool belt. Using HTAccess, redirecting visitors to a temporary maintenance page is simple and effective. All you need to redirect your visitors is the following code placed in your site’s root HTAccess: # MAINTENANCE-PAGE REDIRECT <ifmodule mod_rewrite.c> RewriteEngine on RewriteCond %{REMOTE_ADDR} !^123\.456\.789\.000 RewriteCond %{REQUEST_URI} !/maintenance.html$ [NC] RewriteCond %{REQUEST_URI} !\.(jpe?g?|png|gif) [NC] RewriteRule .* /maintenance.html [R=302,L] </ifmodule> That is the official copy-&-paste goodness right […] Continue reading »

Stop 404s for Mobile Versions of Your Site

[ Stop 404 Requests for Mobile Sites ]

If you’ve been keeping an eye on your 404 errors recently, you will have noticed an increase in requests for nonexistent mobile files and directories, especially over the past year or so. The scripts and bots requesting these files from your server seem to be looking for a mobile version of your site. Unfortunately, they are wasting bandwidth and resources in the process. It has become common to see the following 404 errors constantly repeated in your log files: http://domain.tld/apple-touch-icon.png […] Continue reading »

Is it Secret? Is it Safe?

[ Enjoying the Evening ]

Whenever I find myself working with PHP or messing around with server settings, I nearly always create a phpinfo.php file and place it in the root directory of whatever domain I happen to be working on. These types of informational files employ PHP’s handy phpinfo() function to display a concise summary of all of your server’s variables, which may then be referenced for debugging purposes, bragging rights, and so on. While this sort of thing is normally okay, I frequently […] Continue reading »

Tell Google NOT to Index Certain Parts of Your Web Pages

There are several ways to instruct Google to stay away from various pages in your site: Robots.txt directives Nofollow attributes on links Meta noindex/nofollow directives X-Robots noindex/nofollow directives ..and so on. These directives all function in different ways, but they all serve the same basic purpose: control how Google crawls the various pages on your site. For example, you can use meta noindex to instruct Google not to index your sitemap, RSS feed, or any other page you wish. This […] Continue reading »

Dynamic Link Insertion via Unobtrusive External JavaScript

[ Dynamic Flow ]

In my recent guest post at The Nexus, I discuss Google’s new nofollow policy and suggest several ways to deal with it. In that article, I explain how Google allegedly has changed the way it deals with nofollow links. Instead of transferring leftover nofollow juice to remaining dofollow links as they always have, Google now pours all that wonderful nofollow juice right down the drain. This shift in policy comes as a terrible surprise to many webmasters and SEO gurus, […] Continue reading »

Dealing with Google’s New Nofollow Policy

Anyone plugged into the Web these days has heard about how Google has supposedly changed the way it deals with nofollow attributes. According to a number of speculative reports, Google will no longer apply unused nofollow PageRank to other links on the page. So, let’s say that you have some sites that have been PageRank “sculpted” by way of strategically applied nofollow tags. For example, you may have nofollowed all of your comment, footer, or sidebar links. Ever since Google […] Continue reading »

Welcome
Perishable Press is operated by Jeff Starr, a professional web developer and book author with two decades of experience. Here you will find posts about web development, WordPress, security, and more »
WP Themes In Depth: Build and sell awesome WordPress themes.
Thoughts
Take a screenshot with Firefox (no extension required). Open Developer Tools Settings and enable the “Take a screenshot” button. Then click the button :)
Take a screenshot with Chrome (no extension required). Open DevTools, type Cmd + Shift + P, then type screenshot.
After 10 years working on my 2010 iMac, my upgrade finally arrived. Shiny new iMac shipped from Ireland :)
Too much caffeine weirds me out. But I love the taste of coffee. So once in a while I enjoy a small cup of decaf. Hits the spot.
Chris Coyier is a truly awesome person. One of the finest people I've ever worked with. Just #gottasayit
Excel won't open CSV file because SYLK format? Open it with text editor and add an apostrophe ' at the beginning of the file, save changes, done.
Displaying too many social media buttons and links all over the place imho makes you look desperate and frankly kinda sad.
Newsletter
Get news, updates, deals & tips via email.
Email kept private. Easy unsubscribe anytime.