STILL Paging Google...
Niels Bakker
niels=nanog at bakker.net
Wed Nov 16 13:17:08 UTC 2005
* matthew at elvey.com (Matthew Elvey) [Wed 16 Nov 2005, 01:56 CET]:
>Still no word from google, or indication that there's anything wrong
>with the robots.txt. Google's estimated hit count is going slightly up,
>instead of way down.
robots.txt is about explicitly spidering your site; Google will still
follow links from outside towards your website and index pages linked
that way.
This is common knowledge
-- Niels.
--
"Calling religion a drug is an insult to drugs everywhere.
Religion is more like the placebo of the masses."
-- MeFi user boaz
More information about the NANOG
mailing list