Slow Performance with NAS Storage Unit

[d2b]

I have no idea where the problem is, but I doubt if it is due to Beyond Compare. Still, I have to ask this question:

I just built up a Buffalo TeraStation NAS, Model TS-TGL70C, with two 2TB Hitachi DeskStar (7200rpm, 32MB cache) hard drives configured as a RAID1 array. The Ethernet port on the Buffalo is rated as a 1000Mbps port.

The firmware version is 1.12.

I did a test run, transferring approximately 513GB of music files from a USB 2.0 drive on a Dell GX260 desktop through the Dell's 1000Mbps network port via a wired network with a single 1000Mbps switch also rated at 1000Mbps and then directly into the TeraStation's port.

I used the latest version of Beyond Compare (Scooter Software) to sync the NAS with the USB.

It took 25 hours to complete the file transfer! This is an average transfer rate of 45.6 Mbps, far short of what USB 2.0 and a 1000Mbps network and either hard drive setup should be capable of delivering.

Where's the bottleneck?

[Chris]

My guess is that the usb interface for the external hard drives is limiting the transfer speed.

What kind of transfer speed do you get if you copy a large number of files from an internal sata disk in the Dell to the NAS?

You can also try the external USB to NAS copy with Windows Explorer to see if the speed issue is specific to Beyond Compare.

[dickdunbar]

NAS Storage slowness

I use D-Link NAS storage, configured for Jbod on Gb ethernet.

Running from WinXP laptop, connected through Gb USRobotics switch,
it runs the way I expect it ... very fast.

Switch to a Win7 X64 desktop machine attached to the same switch.
transferring from very fast sata drives, I get performance similar to
that reported by d2b ... glacially slow, measured in days, not minutes.

BC 3.2.0 beta on all machines.

Transfer from XP to Win7 .... very fast. All switch connections are Gb,
no 10/100 connections.

I suspect Win7 is at fault, and it is likely some tcp setting.
Just haven't hooked up netmon to study it yet.

[Chris]

It might help to try the same copy using Windows Explorer to see if the problem is BC specific or if it is a more general problem.

[dickdunbar]

I'll try Windows Copy next. Running sysinternals ProcessMonitor,
BC3 is issuing hundreds of failing requests:

CreateFile C:\Windows\CSC\v2.0.6\namespace\NAS "NAME NOT FOUND"

The copy is from L: local esata disk to N: Network Mount \\NAS\Volume_1

Call Stack:

0 fltmgr.sys fltmgr.sys + 0x2027 0xfffff8800115a027 C:\Windows\system32\drivers\fltmgr.sys
1 fltmgr.sys fltmgr.sys + 0x48ca 0xfffff8800115c8ca C:\Windows\system32\drivers\fltmgr.sys
2 fltmgr.sys fltmgr.sys + 0x222a3 0xfffff8800117a2a3 C:\Windows\system32\drivers\fltmgr.sys
3 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
4 ntoskrnl.exe ntoskrnl.exe + 0x36a80f 0xfffff80002db680f C:\Windows\system32\ntoskrnl.exe
5 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
6 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
7 ntoskrnl.exe ntoskrnl.exe + 0x380498 0xfffff80002dcc498 C:\Windows\system32\ntoskrnl.exe
8 ntoskrnl.exe ntoskrnl.exe + 0x6f853 0xfffff80002abb853 C:\Windows\system32\ntoskrnl.exe
9 ntoskrnl.exe ntoskrnl.exe + 0x6bdf0 0xfffff80002ab7df0 C:\Windows\system32\ntoskrnl.exe
10 csc.sys csc.sys + 0x1e85d 0xfffff88003ab385d C:\Windows\system32\drivers\csc.sys
11 csc.sys csc.sys + 0x1e9c5 0xfffff88003ab39c5 C:\Windows\system32\drivers\csc.sys
12 csc.sys csc.sys + 0x1e35b 0xfffff88003ab335b C:\Windows\system32\drivers\csc.sys
13 ntoskrnl.exe ntoskrnl.exe + 0x68487 0xfffff80002ab4487 C:\Windows\system32\ntoskrnl.exe
14 ntoskrnl.exe ntoskrnl.exe + 0x68441 0xfffff80002ab4441 C:\Windows\system32\ntoskrnl.exe
15 ntoskrnl.exe ntoskrnl.exe + 0x7ff0a 0xfffff80002acbf0a C:\Windows\system32\ntoskrnl.exe
16 csc.sys csc.sys + 0x1e459 0xfffff88003ab3459 C:\Windows\system32\drivers\csc.sys
17 csc.sys csc.sys + 0x1ebbe 0xfffff88003ab3bbe C:\Windows\system32\drivers\csc.sys
18 csc.sys csc.sys + 0x16cd1 0xfffff88003aabcd1 C:\Windows\system32\drivers\csc.sys
19 csc.sys csc.sys + 0x16e84 0xfffff88003aabe84 C:\Windows\system32\drivers\csc.sys
20 csc.sys csc.sys + 0x5dd5e 0xfffff88003af2d5e C:\Windows\system32\drivers\csc.sys
21 csc.sys csc.sys + 0x17a36 0xfffff88003aaca36 C:\Windows\system32\drivers\csc.sys
22 csc.sys csc.sys + 0x58967 0xfffff88003aed967 C:\Windows\system32\drivers\csc.sys
23 csc.sys csc.sys + 0x12b4d 0xfffff88003aa7b4d C:\Windows\system32\drivers\csc.sys
24 csc.sys csc.sys + 0x420b3 0xfffff88003ad70b3 C:\Windows\system32\drivers\csc.sys
25 rdbss.sys rdbss.sys + 0x23a1c 0xfffff88003c23a1c C:\Windows\system32\DRIVERS\rdbss.sys
26 rdbss.sys rdbss.sys + 0x22f67 0xfffff88003c22f67 C:\Windows\system32\DRIVERS\rdbss.sys
27 rdbss.sys rdbss.sys + 0x2421f 0xfffff88003c2421f C:\Windows\system32\DRIVERS\rdbss.sys
28 rdbss.sys rdbss.sys + 0x3aa0 0xfffff88003c03aa0 C:\Windows\system32\DRIVERS\rdbss.sys
29 rdbss.sys rdbss.sys + 0x21614 0xfffff88003c21614 C:\Windows\system32\DRIVERS\rdbss.sys
30 mrxsmb.sys mrxsmb.sys + 0x202c4 0xfffff880035672c4 C:\Windows\system32\DRIVERS\mrxsmb.sys
31 dfsc.sys dfsc.sys + 0x9ce1 0xfffff88003b21ce1 C:\Windows\System32\Drivers\dfsc.sys
32 dfsc.sys dfsc.sys + 0xa692 0xfffff88003b22692 C:\Windows\System32\Drivers\dfsc.sys
33 dfsc.sys dfsc.sys + 0xdf6d 0xfffff88003b25f6d C:\Windows\System32\Drivers\dfsc.sys
34 dfsc.sys dfsc.sys + 0x199a2 0xfffff88003b319a2 C:\Windows\System32\Drivers\dfsc.sys
35 mup.sys mup.sys + 0x6c40 0xfffff8800196fc40 C:\Windows\System32\Drivers\mup.sys
36 mup.sys mup.sys + 0x6157 0xfffff8800196f157 C:\Windows\System32\Drivers\mup.sys
37 mup.sys mup.sys + 0x764b 0xfffff8800197064b C:\Windows\System32\Drivers\mup.sys
38 fltmgr.sys fltmgr.sys + 0x323f 0xfffff8800115b23f C:\Windows\system32\drivers\fltmgr.sys
39 fltmgr.sys fltmgr.sys + 0x222b9 0xfffff8800117a2b9 C:\Windows\system32\drivers\fltmgr.sys
40 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
41 ntoskrnl.exe ntoskrnl.exe + 0x36aa64 0xfffff80002db6a64 C:\Windows\system32\ntoskrnl.exe
42 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
43 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
44 ntoskrnl.exe ntoskrnl.exe + 0x319eab 0xfffff80002d65eab C:\Windows\system32\ntoskrnl.exe
45 fltmgr.sys fltmgr.sys + 0x25dca 0xfffff8800117ddca C:\Windows\system32\drivers\fltmgr.sys
46 fltmgr.sys fltmgr.sys + 0x2401e 0xfffff8800117c01e C:\Windows\system32\drivers\fltmgr.sys
47 fltmgr.sys fltmgr.sys + 0x25361 0xfffff8800117d361 C:\Windows\system32\drivers\fltmgr.sys
48 fltmgr.sys fltmgr.sys + 0x2513e 0xfffff8800117d13e C:\Windows\system32\drivers\fltmgr.sys
49 fltmgr.sys fltmgr.sys + 0x10b2d 0xfffff88001168b2d C:\Windows\system32\drivers\fltmgr.sys
50 fltmgr.sys fltmgr.sys + 0xabd6 0xfffff88001162bd6 C:\Windows\system32\drivers\fltmgr.sys
51 fltmgr.sys fltmgr.sys + 0x11ad4 0xfffff88001169ad4 C:\Windows\system32\drivers\fltmgr.sys
52 PROCMON20.SYS PROCMON20.SYS + 0x4a11 0xfffff88004817a11 C:\Windows\system32\Drivers\PROCMON20.SYS
53 fltmgr.sys fltmgr.sys + 0x2027 0xfffff8800115a027 C:\Windows\system32\drivers\fltmgr.sys
54 fltmgr.sys fltmgr.sys + 0x48ca 0xfffff8800115c8ca C:\Windows\system32\drivers\fltmgr.sys
55 fltmgr.sys fltmgr.sys + 0x222a3 0xfffff8800117a2a3 C:\Windows\system32\drivers\fltmgr.sys
56 ntoskrnl.exe ntoskrnl.exe + 0x374777 0xfffff80002dc0777 C:\Windows\system32\ntoskrnl.exe
57 ntoskrnl.exe ntoskrnl.exe + 0x36aa64 0xfffff80002db6a64 C:\Windows\system32\ntoskrnl.exe
58 ntoskrnl.exe ntoskrnl.exe + 0x36fb76 0xfffff80002dbbb76 C:\Windows\system32\ntoskrnl.exe
59 ntoskrnl.exe ntoskrnl.exe + 0x376887 0xfffff80002dc2887 C:\Windows\system32\ntoskrnl.exe
60 ntoskrnl.exe ntoskrnl.exe + 0x38f5b4 0xfffff80002ddb5b4 C:\Windows\system32\ntoskrnl.exe
61 ntoskrnl.exe ntoskrnl.exe + 0x6f853 0xfffff80002abb853 C:\Windows\system32\ntoskrnl.exe
62 ntdll.dll ntdll.dll + 0x5008a 0x76f4008a C:\Windows\SYSTEM32\ntdll.dll
63 wow64.dll wow64.dll + 0x162d7 0x738862d7 C:\Windows\SYSTEM32\wow64.dll
64 wow64.dll wow64.dll + 0xcf87 0x7387cf87 C:\Windows\SYSTEM32\wow64.dll
65 wow64cpu.dll wow64cpu.dll + 0x276d 0x7380276d C:\Windows\SYSTEM32\wow64cpu.dll
66 wow64.dll wow64.dll + 0xd07e 0x7387d07e C:\Windows\SYSTEM32\wow64.dll
67 wow64.dll wow64.dll + 0xc549 0x7387c549 C:\Windows\SYSTEM32\wow64.dll
68 ntdll.dll ntdll.dll + 0x7c367 0x76f6c367 C:\Windows\SYSTEM32\ntdll.dll
69 ntdll.dll ntdll.dll + 0x32e2e 0x76f22e2e C:\Windows\SYSTEM32\ntdll.dll
70 ntdll.dll ntdll.dll + 0x1fd06 0x770efd06 C:\Windows\SysWOW64\ntdll.dll
71 KERNELBASE.dll KERNELBASE.dll + 0x19702 0x74da9702 C:\Windows\syswow64\KERNELBASE.dll
72 BCompare.exe madTraceProcess + 0xd4980 0x54bad0 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
73 BCompare.exe madTraceProcess + 0x4e412c 0x95b27c C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
74 BCompare.exe madTraceProcess + 0x4e6c4a 0x95dd9a C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
75 BCompare.exe madTraceProcess + 0x4f4071 0x96b1c1 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
76 BCompare.exe BCompare.exe + 0x765c9 0x4765c9 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
77 BCompare.exe BCompare.exe + 0x2caef 0x42caef C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
78 BCompare.exe BCompare.exe + 0x5c4a 0x405c4a C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
79 BCompare.exe BCompare.exe + 0x764ab 0x4764ab C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
80 BCompare.exe BCompare.exe + 0x76518 0x476518 C:\Program Files (x86)\Beyond Compare 3\BCompare.exe
81 kernel32.dll kernel32.dll + 0x13677 0x765a3677 C:\Windows\syswow64\kernel32.dll
82 ntdll.dll ntdll.dll + 0x39d42 0x77109d42 C:\Windows\SysWOW64\ntdll.dll
83 ntdll.dll ntdll.dll + 0x39d15 0x77109d15 C:\Windows\SysWOW64\ntdll.dll

[dickdunbar]

I changed the BC3 invocation to "Run as Administrator" on a very large .iso file.

The copy is proceeding in 1MB chunks; the trace shows a single thread:

R - read
W - Write
C - CreateFile NAS

RWRWRWCC, RWRWRWRWRWRWC, RWRWRWCC, RWRWRWRWRWC, ...

[dickdunbar]

While I was writing this reply, the copy finished ... so it was running much quicker
as administrator.

[dickdunbar]

Run -> Cmd -> xcopy /J xxx.iso N:
/J Copies using unbuffered I/O. Recommended for very large files.

xcopy is reading/writing 32K chunks.

The trace pattern is something like:

RRRWWWRW, RWRWRW, RRRRRWWRWRW ....

xcopy activity starts with:
Read Offset 129,536 length 7K
Read Offset 25,952,256 lengtgh 32K
Followed by:

11 "Create File ..\NAS" requests (NOT FOUND)

Those \NAS requests are not continuous as they are in BC3.

xopy ran _MUCH_ faster than BC3 for this 559MB file.

4:29:41.2492832 PM xcopy.exe 3208 ReadFile
4:30:20.1974570 PM xcopy.exe 3208 CloseFile

Perhaps the large blocks are causing packet re-assembly at the tcp layer,
but there is also the nuisance of the Create File requests.

xcopy is using
ReadFile L:\xxx.iso (Local Disk)
WriteFile \\NAS\Volume_1\xxx.iso

[Zoë]

If xcopy has a "run unbuffered" command then it must include its own copy engine. BC relies on the Windows CopyFileEx routine. You might try TeraCopy and see if it is similarly speedy, since I know it has an optimized copy routine too.

Also, it looks like C:\Windows\CSC is "Client Side Cache" aka Offline Files. http://blogs.msdn.com/b/oldnewthing/.../27/54712.aspx

[dickdunbar]

Figured it out, and now I'm back at _faster_than_xcopy speeds.

xcopy figured out it N: was a mounted filesystem with a UNC of \\NAS\Volume_1
and apparently switched the way it was doing writes.

When I ditched the mounted filesystem, and used the UNC in the BC3 folder,
the speed came back.

So perhaps BC3 can do this switch transparently the way xcopy did.

I'll read up on "Client Side Cache". The trace data doesn't make any sense;
why would Win7 check CSC the way it does? Why not every write ... or like xcopy,
figure it out once and "do the right thing".

[dickdunbar]

"Offline Files" is a service running on Win7.

Since my machine is a desktop, I have no need for offline copies,
so I disabled the service:

Start -> Control Panel -> Administrative -> Services

Offline Files / Properties / Startup Type = Disabled

[dickdunbar]

I've about given up on Win7 x64 ... even with all the changes,
it is painfully slow to synchronize disks.

My guess at the moment is that it is each new file that is queried
that is causing the slowdown.

To actually complete the synchronization, I switched to XP.
- mounted the network drive and the Win64 machine drive shared
- WinXP performs much faster, even though I have to drag the
files across the network through the XP box, doubling the network load.

[Aaron]

Hello,

This may be driver related. Perhaps the Win7 drivers for your NAS device are not quite as good on Win7 as on XP? A few of our machines are Win7 in the office, and there are no big speed differences for remote comparisons in BC3.