Tuesday, March 27, 2012

The Googlebot

For those of you who don't know what a googlebot is, it crawls your site and indexes the content for google searches.  

As I develop I want to make sure that my half-finished product does not appear in any search results, so it's important that googlebot knows to ignore my site.  Here are some sources explaining how the googlebot works, and how to have it ignore your site.

Googlebot,
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=182072

rel=“nofollow”,
http://support.google.com/webmasters/bin/answer.py?hl=en&answer=96569

telling the googlebot no,
http://www.mattwaite.com/posts/2009/apr/12/telling-google-bot-no/

0 Comments:

Post a Comment

Subscribe to Post Comments [Atom]

<< Home