Problems with Simultaneous Connections

[habanero]

Hello everyone. Does anyone else have problems with running more than 1 simultaneous connection? I've been running the max of 10, and during sync operations the connections time out and the files that were syncing at the time of the inturruption must be restarted. If I try and sync files sizes of several MB, this constant retrying takes a long time. I've tried 2 different FTP servers (WS_FTP and ServU) with the same results.

Is it possible to have more than 1 simultaneous connection and get reliable syncs? With 10 connections, the inital folder scans go real fast, but then as I manually sync, the file transfers have problems. I'm doing some experiments to try and determine what the max number of connections are that produce reliable results. Anyone else experienced this?

Regards,

Habanero

[habanero]

Update: I upgraded Serv-U to the latest version and the problems described above have gone away for the most part. However, on large file transfers (40-50MB), the transfer occurs ok, but I get read time out errors when BC tries to finalize the file. I can interrupt BC while it's waiting for the default 40 sec read time out period, then touch the file manually and everything is ok. Otherwise, BC deletes the file and starts the transfer over. Has anyone else noticed problems transferring large files?

Habanero

[mstrong7]

I'm having the same problem for very large files (5GB +) but it always seems to give me the read timeout error with 3% or less left. I just upped the time out value but the files are so large it'll be a while before I can see if it makes any difference.

Usually I just open up Filezilla and pull the remainder of the file with Resume.

[Aaron]

mstrong7, this post looks newer than your other, but did your previous solution solve this issue as well, or is this one still present for you?:
http://www.scootersoftware.com/vbull...ead.php?t=4264

[Aaron]

Hello Habanero,

Do you have trouble with your transfers if you use only one connection? Or does one connection also sometimes timeout and need to be restarted as well?

Beyond Compare does not currently support RESUME, so if your transfer fails for any particular reason, we have to restart and resend that file.
If you use another FTP Client, such as Filezilla, do you notice that filezilla aborts and RESUMES every once in awhile?

[habanero]

Thanks for the reply Aaron. I tried transferring the file using WS_FTP Client and it also had problems at the end, but was able to resume and finish. It, like BC, gets the entire file transferred, but dies at the very end. Here is part of the log from WS_FTP:

Error reading response from server.
transferred 49564684 bytes in 1712.406 seconds, 226.129 Kbps ( 28.266 Kbps), transfer failed.
It appears that the connection is dead. Attempting reconnect...

Are there plans to support RESUME? Is there a setting on the server I could tweak to help?

Thanks,

Habanero

[habanero]

I think I solved the problem. It is an issue with our router's NAT timeout. The control port of the FTP server is not used during a long file transfer, and after 10 minutes, the router thinks the connection is dead and kills it. After the transfer, the client tries to communicate with the server on the control port and it times out. If the client doesn't support RESUME, it starts the transfer all over again. I added the following line to the registry on the client machine:

[HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Servic es\Tcpip\Parameters]
"KeepAliveTime"=dword:0000ea60

This makes WinXP send a keep alive packet every 60 seconds to keep the router from dropping the connection. I need to experiment with the time, as I don't think it needs to send every minute, I'm thinking every 10 minutes will work.

This worked great for the WS_FTP client, and I assume for BC as well. I am waiting until tonight to test so I don't hog bandwidth on my tests.

Habanero

[mstrong7]

Hey Aaron,

This is a problem I'm still seeing after my reg fix on our server. I'm going to try habanero's registry fix. It sounds like it might fix the issue for me.

Thanks

[Zoë]

Habanero,

Thanks for the tip. It might not make it into 3.0.14, but I'll try to add support for using TCP/IP keepalives soon. I'm also looking into adding support for automatic resume in the case of dropped connections too; that will be easier to add than the arbitrary resume that Filezilla supports.

[habanero]

Unfortunately, the keep alive time fix did not work for BC. With WS_FTP client, I can open a connection and use a packet sniffer and see windows xp sending the keep alive packet every 60 seconds for the FTP control connection. When I use BC and establish a connection, windows xp does not send its keep alive packet. I assume this is the problem. WS_FTP itself supports keep alives, but it only works when there isn't a data transmission, so it doesn't keep the control connection alive. That's why I was hoping the windows registry fix would solve the problem with BC the way it helped WS_FTP. So if you do add this support to BC, make sure it works differently than WS_FTP.

Craig, do you have any insight into why BC's FTP engine acts differently than WS_FTP? I will keep experimenting and report back any useful findings.

Habanero

[Aaron]

Hi habanero,

It looks like we'll need to do some work on our end to get this working. The reason we're different from WS_FTP is we use the Indy10 library to support our FTP engine.

If you have any other questions, or find any other useful tidbits, please feel free to post here, or email us at [email protected]. Please include a link to this post for quick reference.

[habanero]

Thanks Aaron. Here is a link that might help you, it looks like others have reported this problem. I'm not much of a programmer, but I am very computer knowledgeable. I will keep playing around with things and let you know if I discover anything useful. I can always use WS_FTP to transfer my large files. One thing I don't understand is why Windows XP doesn't send the keep alive packets whenever BC has a TCP connection open. Windows will send the keep alive packet every 60 seconds if any other app on my system has a TCP connection open. Answering this question seems to me the easiest way to solve this issue? But you guys are the experts.

http://www.codinggroups.com/borland-...rge-files.html

Habanero

[Zoë]

Which packet sniffer are you using, and did you have to do anyting special to see the keepalive packets? I've made a change that should enable it (WSAIoctl(SIO_KEEPALIVE_VALS)) but it doesn't seem to have helped. I'll have to fire up WS_FTP tomorrow and see if it's working similarly.

[habanero]

I'm using PIAFCTM (here at http://www.networkactiv.com/PIAFCTM.html). It's very simple, but I've had good luck with it. I have my ftp server on port 2650, so I just filter out all communication on other ports. I start a 50MB file transfer and while it's transferring, a packet is sent on the control channel 2650, with a response back from the server every 60 seconds. This solved the problem with WS_FTP, so I was sure it would work with BC. If you try the same experiment with BC, the keep alive packets never show up on PIAFCTM.

How hard is it to implement the RESUME command to resume file transfers if they get interrupted? It's not the best solution, but at least it wouldn't start over completely.

Hope this helps,

Habanero

[Zoë]

I've sent an email to the address you used to register in the forums. Let me know if you don't get it.