For more controlled access to Googlebot as far as you URLs are concerned, Google has come up with some changes in its Webmaster Tools, in reference to handling of URLs with parameters. Now, you can decide which page Google should crawl and index, and which pages it should not. This is also a good way to hide any duplicate content, if you have that on your site. Here is a word of caution though, be careful while preventing some of your URLs from being crawled as it would be completely hidden from the search engines in that case.

(more…)
July 25, 2011 at 2:34 am | WebMaster Tool | No comment