Latest TweetsDifference between mod_alias and mod_rewrite perishablepress.com/difference…
Perishable Press
Found 1 matching result
Page 1 of 1

Yahoo! in my Blackhole

Okay, I realize that the title sounds a bit odd, but nowhere near as odd as my recent discovery of Slurp ignoring explicit robots.txt rules and digging around in my highly specialized bot trap, which I have lovingly dubbed “the blackhole”. What is up with that, Yahoo!? — does your Slurp spider obey robots.txt directives […] Continue »