Very high "local" traffic usage

[acestuff]

Firstly, apologies if the category for this question isn't perfect but I thought this was the best fit...

My host counts traffic usage as that from sites and from what it calls "local" traffic. Recently I have noticed that "local" traffic has become huge: some days it has been over 25GB (about 20x that of all my site traffic!). It's always 6GB+ now. This is the "In" traffic. "Out" traffic is only about 4mb (which is about what it has always been - the "In" traffic also used to be about this much).

I have contacted the host who has assured me that it is not a problem with their statistics. However, they can't give me any more information as to where all this "local" usage is coming from. All I have found out from them is that: "the local traffic is generated between MySQL ,FTP and Apache servers".

The only FTP account is mine which I might use to upload a few mbs of files each day. The SQL db is ~0.1mb, queried about 500 times a day. The only complicated thing the server does is a Paypal IPN, executed about 4 or so times a day. There has been no major changes to any of the sites lately.

My question is: how do I find out where all this "local" usage is coming from?
Is there any chance this is down to a security problem? (Just to be sure, all the passwords were changed but the problem still persists...)

Thanks in advance for any help,
Colin

[mirmill]

Are you running an analytics program on your site?

[acestuff]

Are you running an analytics program on your site?

There are 4 analytical tools available. They all focus on web traffic though:
Access & Error Logs
Traffic Statistics
Webalizer
Usage Reports

[mirmill]

I don't really understand their definition of local traffic. Are you on a dedicated or shared hosting account?

[acestuff]

I don't really understand their definition of local traffic. Are you on a dedicated or shared hosting account?

I'm on a shared hosting account.

I also don't really know what it is. I asked support last month. Here is the reply I received:
Its traffic coming in/out of your account and includes ftp, outgoing connections using php, etc. Its calculated manually by a program that tracks the processes on the server and it should be accurate though it may not reflect FTP usage, per se.

When I asked if they could find out where this "usage" was coming from:
Unfortunately our administrators were unable to determine where most of the traffic is coming from. We may assure you that all the traffic statistics are (were) properly calculated.

All they could say was:
The local traffic is generated between MySQL ,FTP and Apache servers for your account.


This is all well and good but it doesn't really help in figuring out what's going on. Do you have any suggestions?

[mirmill]

Well, if nothing on your site has changed, I would be concerned that the host has some kind of server issue going on. It bothers me (and you, obviously) that they can't tell you where it's coming from. Is this a bigger company? I know switching hosts is a PITA, but you don't want anything happening on any of the other sites on that hosting account to mess with yours.

[acestuff]

Well, if nothing on your site has changed, I would be concerned that the host has some kind of server issue going on. It bothers me (and you, obviously) that they can't tell you where it's coming from. Is this a bigger company? I know switching hosts is a PITA, but you don't want anything happening on any of the other sites on that hosting account to mess with yours.

I'm with Awardspace which claims to have over 1,000,000 customers. I have been with them now for about 3 years but only on paid hosting since Christmas. I don't really want to move as, looking at their hosting as a whole, I like them and in the past, I've found their support to be very helpful. Shame about this case though...

It's more than likely my fault as all the scripts that are used are written by me. After looking at things lately, this extreme usage doesn't seem to occur for the day before the PayPal IPN is called (which I now think is the most likely source of the problem.)

As the PayPal IPN uses fsockopen and mySQL, I can imagine that many things could be going wrong. I think I'll just have to start going through all the code to see if I can find the issue. Not knowing the ins and outs of PHP and mySQL that great though, it's going to be tricky!

Oh and you say:
Well, if nothing on your site has changed...
the problem is: I change parts of my site(s) every day...

[mirmill]

Ah, I was going by your comment from your OP: "There has been no major changes to any of the sites lately. :)"

I have heard that Awardspace is often under DDoD attack. Other than that, they sound pretty good. That sounds like a good idea, to see if you can find something on your end. If not, ask to speak to a higher up about the local traffic. Someone must know what it is. Good luck!

[acestuff]

Ah, I was going by your comment from your OP: "There has been no major changes to any of the sites lately. :)"

I have heard that Awardspace is often under DDoD attack. Other than that, they sound pretty good. That sounds like a good idea, to see if you can find something on your end. If not, ask to speak to a higher up about the local traffic. Someone must know what it is. Good luck!


Sorry for the confusion: although no major changes have occurred (a couple are still under development), minor changes happen all the time.

I've never really noticed the effects of any DDoS with Awardspace. On my last server on the other hand, they were DDoS'd and went down for a week! Uptime is always good with Awardspace (and the price was very good).

I'll post back when I find the problem...

[acestuff]

Here's what I've found out so far: the usage is directly linked to the number of times the Paypal IPN is used (every time the IPN is called 4GB of "local traffic" is used :eek:)

I've gone through every last bit of the scripts related to the Paypal IPN. I see no issues in any of the database SQL scripts so it must be something to do with the "fsock" stuff (which I'm not too sure how the fsock works/is!)

If anyone sees anything obviously wrong with this piece of code please say. In the meantime, I'm going to do some research into fsock...


// Read the post from PayPal system and add "cmd"
$req = "cmd=_notify-validate";
foreach($_POST as $key => $value) {
$value = urlencode(stripslashes($value));
$req .= "&$key=$value";
}

// Post back to PayPal system to validate
$header = "POST /cgi-bin/webscr HTTP/1.0\r\n";
$header .= "Content-Type: application/x-www-form-urlencoded\r\n";
$header .= "Content-Length: " . strlen($req) . "\r\n\r\n";
$fp = fsockopen("ssl://www.paypal.com", 443, $errno, $errstr, 30);
// $fp = fsockopen("ssl://sandbox.paypal.com", 443, $errno, $errstr, 30); // Testing only

if (!$fp) {
// HTTP ERROR
echo "Connection Error<br/>Error Number: $errno<br/>Error: $errstr";
}
else {
// Connection with Paypal made
fputs($fp, $header . $req);
while(!feof($fp)) {
$res = fgets($fp, 1024);
if(strcmp($res, "VERIFIED") == 0) {
// Creates storage for buyer
$buyer = new Buyer();

// Ensure transaction of at least minimum amount
if($_POST["mc_gross"] < 1.50) {
echo "Minimum amount not exceeded";
}
else {
// Compiles data to be stored
foreach(array_keys($_POST) as $type) {
$buyer->addDetail($type, $_POST[$type]);
}

// Sends the data to be stored
processSale($buyer);
}
}
else if(strcmp($res, "INVALID") == 0) {
// Invalid transaction
echo "Invalid Transaction";
}
}
}
fclose($fp);

[mirmill]

I'm no expert in this, just asking a few more questions that may (hopefully!) lead to an answer for you.

So fsock basically opens a connection for your PayPal IPN and your servers to talk to each other, yes? Two potential issues, a) you're getting all this local traffic because the data running back and forth isn't formatted properly for some reason and it's bouncing back and forth but eventually working, or b) you have some kind of firewall issue, but you would think that would stop it from communicating completely.

Have you contacted PayPal about it?

[acestuff]

I know this thread died quite a while ago but I thought I'd give it some closure.

I didn't actually find out what the problem was in the end, despite discussion with Paypal and Awardspace. However, a couple of weeks ago, this really high "local" bandwidth usage stopped. I hadn't changed any of my code so it wasn't a problem my end. I suspect that it was a problem with Awardspace's system (wouldn't be the first...)

Anyway, I'm happy that it's fixed now :)

[sonyaseo.1987]

why you not used w3c validator tools??? Try it. its a great and said about of your mistake.

[acestuff]

why you not used w3c validator tools??? Try it. its a great and said about of your mistake.

I have no idea what w3c's validation tools have to do with the problem I had... Their tools are for validating markup used client site mainly.

I do agree however, that they are great.

[sonyaseo.1987]

papplu. If you want to local traffic more do cotinuously local listing for your website. You achieve local traffic speedly.

[criterion9]

papplu. If you want to local traffic more do cotinuously local listing for your website. You achieve local traffic speedly.

FYI: You might want to review your filter or actually read the posts above your irrelevant reply...