PhP ftp_put function limitation? -

i working on solution have upload 33,000 or more csv files server.

my script working fine upto 25,000 files breaks after 26,000 files upload or 27,000 files upload etc. following php warning i.e.

php warning: ftp_put(): bind() failed: address in use (98)

i have researched error on internet didn't find solution, therefore, posting here.

i checked directory no. of files limit on ubuntu 14.0.4 lts server ext4 filesystem , couldn't find restricts 33,000 files upload single directory.

i tried upload same 33,000 files upload 2 different directories i.e. 20,000 dira , rest 13,000 dirb after 5,000 successful files transferred dirb, stops , gives above error.

here basic code using upload files reference i.e.

    $dpath = "/var/www/data/";     $sourcedir = $dpath .'test1';     $remotedir = 'test2';      $serverip = "192.168.0.1";     $user = "dataexport";     $pass = "abc";      $connhandle = ftp_connect($serverip);     if($connhandle)     {         $loginflag = ftp_login($connhandle, $user, $pass) or   die('couldnt connect $serverip');          if($loginflag)         {             if(is_dir($sourcedir))             {                 $dirhandle = opendir($sourcedir);                  if($dirhandle)                 {                     while(($file = readdir($dirhandle)) !== false)                     {                         if($file != "." && $file != "..")                         {                             if(is_dir($sourcedir.'/'.$file))                             {                                // code copying directory.. not using it.                             }                             else                             {                                $srcfilepath = $sourcedir.'/'.$file;                                $desfilepath = $remotedir.'/'.$file;                                  if(ftp_put($connhandle, $desfilepath, $srcfilepath, ftp_ascii))                                 {                                     echo "copied path: $desfilepath \n";                                 }                                 else                                  {                                     echo "copying failed \n"; die();                                 }                             }                         }                     }                      closedir($dirhandle);                 }             }             else              {                 echo "not valid directory <br />\n";             }         }         else         {             echo "wrong credentials server ip = $loginflag <br />\n";         }     }     else      {         echo "unable connect server ip = $serverip <br />\n";     }      ftp_close($connhandle); } 

note: modified real code make simple , didn't test may require testing.... getting error reported @ following code line after 25,000 files transferred. i.e.

ftp_put($connhandle, $desfilepath, $srcfilepath, ftp_ascii)

i tried file upload "ftp_binary" option didn't work.

looking solution advice. in advance.

cheers

the error message pretty explanatory:

 bind() failed: address in use (98) 

this takes number of files out of question, has nothing transfer mode.

the reason buried deep internals of ftp protocol. ftp old protocol uses connection commands , new connection each file transfers. each tcp connection bound port; port number 16-bits unsigned integer. leads maximum 64k ports available.

the first 1024 port numbers (0..1023) reserved well-known protocols (80 http, 25 smtp, 22 ftp command connection , on). range 1024..49151 contains "registered" ports , port numbers between 49152 , 65535 contains "dynamic" ports. available os bind short-lived connections them. there 16k port numbers available in "dynamic" range.

some operating systems use larger range dynamic ports. noted on wikipedia page):

many linux kernels use port range 32768 61000.

there 28k ports available in range. take account there other programs running on computer during execution of script; many of them create tcp connections , connections use ports too. reduces number of port numbers available application.

i don't know happens inside php's implementation of ftp looks doesn't close data connections uses file transfers , after several thousands files, entire stock of free ports can used applications consumed , system call bind() fails.

the solution

since apparently there no way configure behaviour of ftp functions in php, can try close ftp connection , reopen it. hopefully, php release associated resources, including many ports used file transfers.

i didn't check this, don't know if works!

in case doesn't work can try use ssh/sftp functions provided php. ssh/scp protocol designed avoid many of flaws of older protocols (like ftp). can find basic usage example on documentation page of function ssh2_scp_send().