dcsimg
www.webdeveloper.com
Results 1 to 8 of 8

Thread: FTP upload speed compared to other clients...

Hybrid View

  1. #1
    Join Date
    Jun 2004
    Posts
    220

    FTP upload speed compared to other clients...

    Hi

    I have a really simple script that uploads a file using ftp_put(). This script is about 10 times (!) slower than uploading the same file from the same server to the same server using Filezilla. The PHP script is as far as I can tell doing exactly the same as Filezilla, passive mode, utf-8 opt, etc. We have captured the traffic using Wireshart and the company responsible for the FTP server is currently investigating those, but I thought I'd just check in on this forum. Edit; the question is if anybody has seen this and has some advice

    Thanks
    Lubox
    Last edited by Lubox; 12-18-2012 at 06:46 AM.

  2. #2
    Join Date
    Jul 2010
    Location
    /ramdisk/
    Posts
    865
    You can use the "cp" command from ssh+[ba]sh. That would make the most sense and would be exceptionally faster than using FTP to copy files.

  3. #3
    Join Date
    Jun 2004
    Posts
    220
    Using Filezilla a FTP upload is as fast as the internet lines at the other end is (we have exceptional high bandwidth ourselves, so we're almost always maxing the other end). Of course, with PHP as FTP client we're not doing this since it is 10-20 times slower on all versions I've tried. If you have any examples of "cp" command from ssh+[ba]sh on Windows (!), I'll gladly check it out. I've tried the PHP copy(), but that is as slow as ftp_put() (probably using same methods).

    Edit: Note that this only applies to uploads (ftp_put()). FTP downloads (ftp_get()) with PHP is as fast as Filezilla.

    Thanks
    Last edited by Lubox; 12-20-2012 at 03:16 AM.

  4. #4
    Join Date
    Jul 2010
    Location
    /ramdisk/
    Posts
    865
    Quote Originally Posted by Lubox View Post
    Using Filezilla a FTP upload is as fast as the internet lines at the other end is (we have exceptional high bandwidth ourselves, so we're almost always maxing the other end). Of course, with PHP as FTP client we're not doing this since it is 10-20 times slower on all versions I've tried. If you have any examples of "cp" command from ssh+[ba]sh on Windows (!), I'll gladly check it out. I've tried the PHP copy(), but that is as slow as ftp_put() (probably using same methods).

    Edit: Note that this only applies to uploads (ftp_put()). FTP downloads (ftp_get()) with PHP is as fast as Filezilla.

    Thanks
    wat...

    ok you get putty, log in with your "SFTP" credential (SFTP meaning... ftp over ssh). So ssh will spawn a shell (probably bash or sh), and you can use "cp" the copy command,
    cp <src> <dst>
    it will be hundreds of times faster than trying to copy crap from the server to your computer and then back to the server again, lol.

  5. #5
    Join Date
    Oct 2010
    Location
    Ohio
    Posts
    233
    The reason its faster with filezilla is because it can create multiple streams. You may try forking the process, giving each fork a batch of files to process, this will allow the job to be running with multiple down streams and greatly decrease the total transfer time.

  6. #6
    Join Date
    Jun 2004
    Posts
    220
    Quote Originally Posted by Derokorian View Post
    The reason its faster with filezilla is because it can create multiple streams. You may try forking the process, giving each fork a batch of files to process, this will allow the job to be running with multiple down streams and greatly decrease the total transfer time.
    Multiple streams within the ftp-protocol? Is that even possible? Filezilla are able to transfer multiple files, but the protocol doesn't allow multiple threads for one and the same file (if I understood you correctly) as far as I know.
    I'm already transferring many files at the same time in the system (40-100 simultaneous uploads depending on receiving server), so all in all the speed is not bad. But it could be much, much faster.

    I've also tried to upload using cUrl, but that is equally slow. I will also try with socket operations and see if I manage uploads on that level.

    Note that ftp downloads (ftp_get) is as fast as Filezilla, so there's a huge difference in ftp_get and ftp_put.

    Could it still be the servers that are using som kind of limit if the connection is not from a known client or something? I'm trying to find servers that I do not see this on, but so far they've all been the same.

    To "eval", any script examples would be nice. By the way, why do you think I copy crap from the server to my computer and back to the server?

    Thanks
    Lubox

  7. #7
    Join Date
    Oct 2010
    Location
    Ohio
    Posts
    233
    Quote Originally Posted by Lubox View Post
    Multiple streams within the ftp-protocol? Is that even possible? Filezilla are able to transfer multiple files, but the protocol doesn't allow multiple threads for one and the same file (if I understood you correctly) as far as I know.
    I'm already transferring many files at the same time in the system (40-100 simultaneous uploads depending on receiving server), so all in all the speed is not bad. But it could be much, much faster.
    Ah my misunderstanding. I thought you were uploading a file and moving on to the next when it was complete. So my suggestion was based on that assumption. If you are already doing multiple files at a time then I'm not sure I can give you any pointers to increase the speed. I just know my site back up started finishing much quicker when I spawned multiple processes to handle sets of files (aka pushing multiple files at once) and I thought maybe you were running into the same problem.

    Sorry I'm less than useful sometimes.

  8. #8
    Join Date
    Jun 2004
    Posts
    220
    Quote Originally Posted by Derokorian View Post
    Ah my misunderstanding. I thought you were uploading a file and moving on to the next when it was complete. So my suggestion was based on that assumption. If you are already doing multiple files at a time then I'm not sure I can give you any pointers to increase the speed. I just know my site back up started finishing much quicker when I spawned multiple processes to handle sets of files (aka pushing multiple files at once) and I thought maybe you were running into the same problem.

    Sorry I'm less than useful sometimes.
    Hehe, no worries, I know I didn't make this clear in the first post, I just thought I'd stick to the real issue, the single transfer. So, I've tried ftp_put(), I've tried copy(), and I've tried the cUrl upload. All 10-20 times slower than the same upload from Filezilla. And it's not like there's a limit reached within PHP. I've tried it on fast servers and it may upload at 2500kB/s, I've tried it on slow servers and it may upload at 500kB/s. And Filezilla uploads 10 times faster on both servers. It just seems like there's a throttle somewhere saying "if PHP {10% of max speed}"

    Thanks
    Lubox

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  
HTML5 Development Center



Recent Articles