Sudden bandwidth useage increase

[paul]

I woke up to a bandwidth overrun message from my hosting provider this morning. One my old sites (1999) suddenly sucked up almost three normal months of bandwidth useage in one evening. Looking at the stats I see that IP 66.249.65.44 was apparently the issue and as near as I can tell that IP belongs to Google.

This is a static html site with about 125 pages that has been fully indexed for years.

Can anyone tell me what was going on?

[Todd W]

Google spydered your site.

[James]

That's not Googlebot's IP address. Looks like someone was just being an arse.

[Todd W]

If 125 static html pages with no downloads (no movies, big flash) etc, made you go over your BW limit.. maybe your BW limit is to low ??

I didn't look up the IP just assumed you were correct.

[Westech]

That IP does belong to Google (according to http://www.arin.net/whois/ ), so it's most likely one of their various bots. Do you have any logs of the page requests that would show the User Agent for those requests? That could help you figure out exactly what's going on.

[James]

Ah, apologies, I just looked up a list, which I guess was outdated.

[paul]

Here is what the Webalizer stats for user agent show:

1 6756 27.23% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1; .NET
2 2947 11.88% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1; SV1)
3 1016 4.09% Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)

My site normally uses less than 1 gig of bandwidth per month. They used almost 5 in one night.

[r2d2]

Maybe your stats havent updated yet, because that doesn't look like googlebot - I have this in webalizer:

6 2787 3.74% Googlebot/2.1 (+http://www.google.com/bot.html)

also that doesnt look like anywhere near enough traffic to use 5Gb with plain html pages.

The 'Top 10 of X Total Sites By KBytes' stats near the middle could reveal more.

[paul]

Maybe your stats havent updated yet, because that doesn't look like googlebot - I have this in webalizer:

6 2787 3.74% Googlebot/2.1 (+http://www.google.com/bot.html)

also that doesnt look like anywhere near enough traffic to use 5Gb with plain html pages.

The 'Top 10 of X Total Sites By KBytes' stats near the middle could reveal more.

1 504 2.03% 404 2.00% 4836614 95.94% 61 2.00% 66.249.65.44

Thats where I got the Google IP.

[Masetek]

Man, something is really wrong there! Google raped your site bad. Maybe you should email google, give them that data and ask them whats up.

[Westech]

Here's some info you might find useful: http://www.google.com/webmasters/bot.html

[paul]

Thanks for the responses. Things seem to have settled down without me doing anything. Below are the stats for the last few days. The starting and ending bandwidth numbers are what I consider "normal" for my site. As much as I can tell, all that activity has not changed anything. I will await the next update with more than the usual anxiety :)

3 5338 15.97% 4310 15.74% 1854 15.15% 667 16.05% 658 21.37% 50182 0.81%
4 5150 15.40% 4297 15.70% 1917 15.67% 612 14.73% 603 19.58% 1797426 28.93%
5 4111 12.30% 3441 12.57% 1618 13.22% 569 13.69% 547 17.77% 2879848 46.35%
6 4297 12.85% 3417 12.48% 1685 13.77% 552 13.29% 511 16.60% 1349960 21.73%
7 4365 13.06% 3773 13.78% 1612 13.17% 561 13.50% 537 17.44% 54485 0.88%

[James]

I'd suggest that you still ask Google what's up, to see if you can prevent this from happening again; you don't want them doing this every few weeks.

*PS* \/\/007!!!11!1! 1337'th p057!11!!!*asterisk!1!