Google found your hidden site?

September 11th, 2007

Sometimes when you make a site you’ll have in mind to keep it “hidden” for a while – meaning not linking to it from any other site or submitting it to the search engines. I usually do this when I want some time to play around with it, test different ideas out, or error check it before it goes “live.” But sometimes Google finds your site anyways – how does this happen? Here are the most common ways that Google can use to find your supposedly isolated site:

1. Your blog is automatically pinging
2. You use Adsense on your site, even if just for a quick test
3. You use Google Analytics
4. Google found your site through a drop list, whois list, or a recently registered domain name list
5. You have the Google Toolbar installed
6. You are using a domain name that was previously registered before it expired

While these are not all the ways that Google can find your site, they are by far the most common. Out of the list, having the Google Toolbar installed is probably the primary reason that your sites are being discovered. So how can you stop this from happening? You have two choices: first, you can develop your site locally. This means installing software on your computer that turns it into a virtual server – Apache, PHP, MySQL, etc – and creating your site independent of the internet. This is not as hard as it seems, since you can download various all-in-one programs that will create this set up in a matter of minutes. Two of the most popular are WampServer and ApacheFriends.

Second, you can use robots.txt in your root directory to prevent robots from indexing certain sections (or all) of your site, like so:

User-agent: *

Disallow: /directory // OR * for all

Again, this would go in robots.txt in your root directory.

If you liked this post, consider subscribing to AbstractPromotion.com's RSS feed.