Bandwidth Stealing ?

[memaggiem]

Hello, I am a neophyte here and am hoping that this is the proper place to post this question. I own a website and am having issues with bandwidth. It appears that a few visitors are taking 600+MB at one visit alone!

The "average" bandwidth used per visitor is about 25MB

Is there anyway that I can see exactly what/where/how they are taking down all that bandwidth?

Thanks so much for your reply!! My website is small and niche specific and is very popular for it's content. It's not very professionally done so I'm shy about having you all visit it - plus, I have "bursted" my bandwidth useage for the month already LOL

[ADAM Web Design]

They could be search engine spiders or other bots. Or they could just be really greedy users. It's kinda hard to say.

Do you have access to your raw logs? If os, get the IPs of the greedy types from those and run checks on them using tools like www.samspade.org and others.

[sysop9999]

Could be you're being hotlinked as well.

Get your hands on your website logfile and see who's repeatedly downloading your stuff.

[ADAM Web Design]

I can't see hotlinking being the issue. It's possible, but if it's the wholesale baby food site then there aren't a lot of large images (and from what I can see, no videos) that would suggest hotlinking.

It's a possibility, and not a bad thought. I just see it as a longshot myself, and would respectfully disagree in favour of some form of crawler, script, or automated program.

Maggie: have you talked this over with your web host? They may have information we don't, and therefore be able to shed some light on the situation as well. If you're paying them for bandwidth, then they should be able to justify its usage and where it's coming from, and block any potential excessive automation.

Which, in typing that, led to a thought that I hate to suggest as a possibility (since it's extremely negative and a BIGtime longshot), but I have seen it happen.

Have you made any enemies online recently? As in, have you done something knowingly or otherwise that might have upset the wrong people? If so, they may be launching a Denial of Service (DoS) attack on you and/or your server. A DoS is when someone (or a bunch of someones) with nothing better to do decide to send a whole bunch of bad requests for information to your server to try and knock out the Internet connection and/or the server itself.

By the same token, if you're not running your own (dedicated) server, you're sharing it with a bunch of other people, one of whom may have upset the wrong people as well.

Like I said, though, this is a severe longshot, and most companies worth their salt have taken every precaution to see that it doesn't happen (since they have to deal with their bandwidth providers on this issue, and those guys can get a little perturbed when things like that happen.)

Anyway, one way or the other, I think we're all just speculating and need some more info.

[memaggiem]

THANK YOU For your input and time!

I have hotlinking disabled and I have run the IPs of the "greedy" users. One of which actually runs back to a highly respectable IT firm (Hmmmm!) I have raw logs however they don't give specifics on what the IPs have viewed, just the bandwidth used.

All of the greedy ones are dedicated IPs so it would be rather easy to "report" them however I am really not wishing to do that; that indeed might make me some enemies!! Also, I am not sure it would be considered "abuse". I know that when I get visitors from an ISP like comcast, that I am unable to track down a "dedicated" ISP. Even visitors from those large ISP do not suck up a lot of bandwidth!

I have gone about reducing the file sizes of the gifs and jpgs but like mentioned, they are not really large files to begin with!

LOL, I am afraid to talk to my ISP because my "burst" rate is VERY high - at this time I am thinking that they are going by averages and not charging me. In the past the site barely broke 20GB bandwidth per month!

Again, thank you very much for taking the time to reply!!!! I really appreciate it!

P.S. - the bots that visit seldom break over 30MB and I have a robots.txt file in place so as to not index many of my directories. I am also not visited by many "unknown" or "unsavory" robots as far as I can tell......

I was wondering if it is possible for the bandwidth to be used because someone is downloading the whole site?

[ADAM Web Design]

It's a possibility. Not likely, but it's possible.

As far as being scared to tell your host, if anything you should be the opposite. Crawl up your host's ass until they help you fix this! And don't let up until you get an answer.

If you don't it'll be you that suffers, not them. Hosts already slash each other's jugular veins as it is, and things like this are one of the few ways they have to make some costs back (unless it's an ISP-based host or some ripoff company like Interland, in which case crawling up their ass goes double because they'll SURELY take you to the cleaners, given a chance.)