I've been using this technique for more than a year I believe-- so far, I haven't been contacted by Google about this practice. Nor have I been banned for doing it. But if this post raises their attention to the issue of indexing "cruft" within a weblog site, hopefully it will produce a Google-sanctioned solution that we can all use.

Who knows, maybe Ev could bring this problem to their attention...

If you don't want Google to index your comments and trackbacks together with the content of a page, isn't the best solution to not put it there? Instead you could have a link at the bottom of each story to a index of comments and trackbacks for that page.

But as to putting the comments/trackbacks on a different page? No, I'd rather not. Besides, there's more than just comments and trackback that I want to exclude. I used to get a lot of hits searching for "blah photos" matching for pages all over my site. Even though the page mentions "blah", there were no photos of it/them. It was the site navigation link for "Photos" on the right was causing these false positive search results. So-- should I put my site navigation on a separate page??? Perhaps I should start using frames or something?

Over my dead <body/>.

I doubt that the "cloak detection" process is completely automated, and I'm sure that if they manually reviewed your content to see what you were hiding, they'd understand. Heck, you could even leave an HTML comment for them:

I don't think it's a problem as long as you are just hiding cruft, and not introducing new content or doing something malicious.

On a different note, those using Mozilla Firefox can change their user agent on the fly. Sometimes I like to surf the web "as googlebot." The results can be interesting.

I'd image this would be pretty easy to do since you already have the spider-detecting function.

And unfortunately it's problems like that that cause Google to serve up a lot of results that are nowhere near what a person is searching for and thus waste their time.

You're defintely cloaking here. Though its good to see that they haven't penalized you for it.

Perhaps there is a little human judgement still applied. That would be welcome info.

Thanks for sharing the script.