While the email and calendar facilities offered by Google Apps are superb and the web page building tools are easy to use, there is (or was see UPDATE below) one major flaw. Google Apps applies a robots.txt file that prohibits all search engines (including Google’s) from indexing the site. This wasn’t always the case, it appears to have first raised its ugly head sometime in November.
Where’s the logic in this ? As it turns out none, it’s a bug, well it may be a bug as a comment I left on Peter Coopers blog prompted him to email a contact he had within Google, who contacted somebody else within Google, who confirmed it was a bug. Makes you worry about handing over too much of your infrastructure to “faceless companies”. Who do you call, or email? Where are the bug databases? Questions on the support forum, remained unanswered. I guess we can always ask guys like Peter Cooper, thanks again Peter.
The bug may already be fixed, at least for my apps.gleesonIT.com domain. Still seeing the offending robots.txt appearing occasionally on this side of the Atlantic, but accessing the apps sub-domain via an Amazon EC2 image (hosted in Washington State, USA) the robot is now consistently blank.
Now I have to make a decision, will I move the site back onto Google Apps (had moved the “www” domain to Zimki )? Better do something fast before the duplicate content on apps / www.gleesonit.com ends up being penalised!