Google Crawler

Mike Damm MikeD at irwinresearch.com
Fri Jan 3 17:07:47 UTC 2003


http://www.google.com/bot.html for issues with the crawler.
mailto:googlebot at google.com will get you a human bean to talk to. Normally
when there is a problem with their robot, they are pretty responsive.

	-Mike

---
Michael Damm, MIS Department, Irwin Research & Development
V: 509.457.5080 x298 F: 509.577.0301 E: miked at irwinresearch.com


-----Original Message-----
From: Andy Ellifson [mailto:andy at ellifson.com] 
Sent: Friday, January 03, 2003 8:45 AM
To: nanog at merit.edu
Subject: Google Crawler


We are a domain registrar and we host/park over 750,000 domain names. 
Every now and then the Google Crawler decides to bury the machines that
host our 'parked' domain pages.  We use robots.txt but that doesn't
help under these circumstances.  I have tried sending a message to
Google using their web site.  They don't have a NOC entry on
puck.nether.net either.  Our only alternative right now is to block the
crawler at the router level.

Does anbody have a contact at Google or is anyone at Google listening?

Thanks!
Andy Ellifson



More information about the NANOG mailing list