What could cause reduced throughput when syncing with BC?

[timg11]

I'm using BC 4.1.2 on Windows 7 to back up a folder to a share on a server.

The LAN is Gigabit Ethernet. When I update using BC, the maximum network throughput is about 20 Mb/s. When I use Windows Explorer to copy a file, the throughput is about 400 Mb/s.

I checked in Process Explorer and the IO Priority for BC is normal.

I asked for a throughput-throttling feature in BC a while ago, but I don't think it has been implemented.

Is there any other setting in BC that could limit throughput during a sync operation?

[Chris]

Beyond Compare doesn't support throttling throughput for network shares. It will always copy or compare at maximum throughput. Only FTP supports throttling transfer speed.

Do you have Content Comparison (CRC, binary, or rules-based) checked in Session Settings' Comparison tab? If you do, that might slow things down. If you're using the default size and modified date comparison, there shouldn't be anything else that will optimize speed.

What is the size of the file or files you're copying? Beyond Compare 4.0.1 and newer use unbuffered copies for files 1 GB and larger in an attempt to optimize performance.

If you look at task manager during the copy, do you see high CPU utilization for your antivirus software? Is it possible your antivirus software is running a more detailed scan on bcompare.exe's file operations than it does on explorer.exe?

[timg11]

Chris,

The files are over 1 Gbyte in size. There is no content comparison in the session. However the slow speed is seen during the actual file transfer, not during the comparison phase.

CPU utilization is normal in both cases.


It's almost like when a laptop has both Wi-Fi and Ethernet, and for some reason the system decides to use the Wi-Fi link instead of Ethernet. But this system has only the wired gigabit Ethernet port, so I'm not sure why BC would be slowed down.

[Chris]

Sorry, I don't know why the transfer is slower in BC on your system.

I did a quick test on our office network but didn't see a performance difference between BC 4.1.2 and Windows Explorer.

Test PC was a Windows 10 Pro system with 16 GB of ram and files on an internal SATA hard drive.
Destination server was a Windows Server 2012 system with hardware raid and 32 GB of ram.
Both machines were connected to the same gigabit LAN.

I created a 10 GB test file using "fsutil file createnew file.bin 10737418240".

In Windows 10 Task Manager, the Performance tab showed transfer rates between 920 and 950 Mbps when copying with BC and when copying with Windows Explorer.

The BC session used default Folder Compare settings (timestamp and size).

[timg11]

Chris,

I've had some time to do some more testing. This is very interesting, and does seem to involve something specific with BC.

To recap, I'm syncing large files from a Windows 7 desktop to a share on a Windows 2008r2 server.
When syncing or copying from BC4, the network throughput is 12-20 Mbps.
When using Windows Explorer (drag/drop or CTRL-C CTRL-V copy), the throughput is 500-700 Mbps.

The problem is limited to a specific share on the server. If I copy to a different shared drive, BC will copy at full speed.

The problem is not specific to a given workstation. I can run BC from another Windows 7 machine and see the same slow transfer to the specific server and share.

The problem is version-specific to BC. If I revert to BCompare-4.0.7.19761, the problem goes away. I see full speed copy and sync to all shares on the server. BCompare-4.1.1.20615 and BCompare-4.1.2.20720 exhibit the very slow copy to the specific shared drive.

Here's the Wireshark TCP Throughput analysis with the problematic version of BC:


It seems that BC is transferring data only in brief bursts.

I don't observe any unusual CPU usage or other system activity with BC (on either system that exhibits this problem)

Let me know what other data I can gather on this. I looked through the changelog post 4.0.7.19761, and I don't see anything that would seem to be related.

One other detail - I recently replaced the hard disk on the server that is used for the share that is exhibiting the problem. That is also when this problem with BC seems to have started. However, overall the drive seems to work fine, locally and over the network - except for the newer versions of BC. Is there anything I should look for on the server with respect to that new drive that could affect operation with BC? I can't be certain it is the drive change only, because I upgraded BC at the same time also.

[timg11]

The problem is version-specific to BC. If I revert to BCompare-4.0.7.19761, the problem goes away. I see full speed copy and sync to all shares on the server. BCompare-4.1.1.20615 and BCompare-4.1.2.20720 exhibit the very slow copy to the specific shared drive.

I'm really curious to learn what is different between 19761 and later versions.

You said that implementing a "throttled" or "background" copy functionality was on the wish list. It looks like the developers may have stumbled onto a way to do exactly that. Just determine what causes this behavior, and make it an option!

[timg11]

I'm still hoping someone from Team Scooter has some insights on this. To recap the unusual situation:

[FONT=Lucida Console]Copy Method                |          Target Drive          |       Result
---------------------------|--------------------------------|-------------------
Windows Explorer           |       WDC20EARS (2TB)          |      400 Mb/s
Windows Explorer           |       Any Other Drive          |      400 Mb/s
BC 4.0.7.19761 or earlier  |       WDC20EARS (2TB)          |      400 Mb/s
BC 4.0.7.19761 or earlier  |       Any Other Drive          |      400 Mb/s
BC 4.1.1.20615  or later   |       WDC20EARS (2TB)          |      20 Mb/s
BC 4.1.1.20615  or later   |       Any Other Drive          |      400 Mb/s[/FONT]

What changed with BC 4.1.1 that would affect throughput to specific drives? Is there some copying "mode" that was added in 4.1 that takes advantage of certain features expected to be present in Windows 2008 server? Maybe that feature on the server was disabled for that drive port when the previous drive failed, and was not re-enabled when the drive was replaced?

[Aaron]

Hello,

Sorry for the delay. We've been a little swamped and don't have a clear answer yet. We didn't make any changes in 4.0.7 to 4.1 which should effect this behavior. There was a change back in 4.0.3 which might have, but if you rolled back to 4.0.7 and it works then that is unrelated.

If you create a Portable Install of BC 4.1.2 and choose the 32bit version, does this have an impact on your tests?

Next, if you have any 3rd party firewalls or scanning software, it might help to temporarily disable them.

And you mentioned generally that this is a setup share. This is a Windows Network Shared folder (samba, using \\servername\folder\ syntax), correct?

[timg11]

Aaron,
I'm already using the 32 bit version of BC, since the client is 32 bit Windows.
No 3rd party firewall.
Yes, it is to a Windows Network Shared folder (samba, using \\servername\folder\ syntax).

I just updated to BCompare-4.1.3.20814. Here's some more interesting data:


This shows network throughput with a first instance of BC syncing to the server described above writing to a different drive.
Part way through I start a second instance of BC syncing to the WDC20EARS drive on the same server.
Throughput for both instances drops off to a trickle.


The image above shows the point where the second instance of BC syncing to the WDC20EARS drive is stopped.
The throughput from the first instance is restored.

[timg11]

After thinking about how copying to the WDC20EARS "Green" drive brought down the throughput for the entire server, I monitored the server during the two instance test shown above, and nothing unusual happens during the time where the throughput is throttled. There is no uptick in CPU use, errors in the log, or anything else out of the ordinary. My next step was to replace the WDC20EARS "Green" drive with a WD20EFRX "Red" drive.

Now I'm seeing full throughput with the latest version of BC.

There is still a mystery of why earlier versions of BC would provide full performance with this specific drive, and basic Windows explorer copies could also provide full performance. The issue is not simply the drive behavior alone, but some combination of BC behavior in the recent versions, as well as some aspect of the drive.

I'll keep the old WDC20EARS around in case I can collect any additional data off of it. I still think understanding the root cause might be a great opportunity to easily implement a low system impact "throttled" throughput mode in BC.

[Aaron]

Hello Tim,

Are there any firmware updates available for the Green drive? Does the issue only occur if a second transfer begins? Could there be an anti-virus or other software that could initiate a scan during the copy which is then starting as a separate transfer?

[timg11]

Aaron, I will check on the firmware version the next time I have the drive hooked up. Definitely no other programs or processes accessing the drive during the test.