http://www.inktomi.com/ might be the responsible for B2K, while Google is the one running here,
http://www.robotstxt.org/wc/norobots.html might be the solution in both cases.