Results 1 to 5 of 5
  1. #1
    Join Date
    Dec 2007
    Location
    U.S. East coast
    Posts
    295

    Default Removed "Bypass disk cache during binary comparisons" option.

    In the "Notable Changes" for Beyond Compare 4.2 Beta at http://www.scootersoftware.com/download.php?zz=beta42, is listed:

    "Removed "Bypass disk cache during binary comparisons" option. Due to changes in hardware, operating systems, and drivers, it no longer works as originally intended."

    Could you explain further? Even if this function is not fully supported by the host operating system, is there no advantage whatsoever in providing the setting? (My O/S is currently Ubuntu 12.04.5)
    Last edited by Dave_L; 28-Feb-2017 at 07:24 PM.

  2. #2
    Join Date
    Oct 2007
    Location
    Madison, WI
    Posts
    2,503

    Default

    Hi Dave,

    I'm happy to go into more details; I figured we'd have at least one person ask.

    The short answer is, No, I don't think there's any advantage to keeping the setting, and I actually think it's harmful to have it, because it gave people a false sense of security.

    The long answer:

    The setting was originally added to support verifying copies to floppy disks. The code hasn't really been touched since then, and it isn't possible to make it work for that purpose on other drive types.

    As background, on Windows it was implemented using the FILE_FLAG_NO_BUFFERING flag, on Linux it opened the file using O_DIRECT, and on macOS it called fcntl with F_NOCACHE. The one thing all of those have in common is that they're largely intended as an optimization for applications that do their own caching, not as a way to verify things.

    On Windows:

    1) Microsoft's official documentation on the flags we're using say that it isn't supported by the CD/DVD driver nor on network drives and does not affect caching done by the hard drive itself. That last has become a more significant as HDD onboard memory caches have grown, since SSD and hybrid drives have multiple layers of cache, and since SSDs have complex logic to handle bad sectors.

    https://support.microsoft.com/en-us/kb/99794

    2) Windows does not have a way to force writing a file directly to disk and then reading it back without accessing the disk cache. It used to for some cases, but bypassing the disk cache isn't supported by reads anymore, and doing it on writes requires passing a flag when opening the file, which we can't do since we're using Windows' CopyFile API. Drivers and/or disks could and did ignore the flags even when they were supported. (OSR is a consulting company for file system drivers; a Microsoft employee also chimed in).

    https://www.osronline.com/ShowThread.cfm?link=264166

    3) XCOPY's /V (verify) command just checks that the the source and destination are the same size. In MS-DOS it used to be able to issue a "Write then verify" command instead of a normal "Write" one, which would tell the driver to write the data and then immediately read it back and compare the results, but the Windows driver model no longer supports that.

    https://blogs.msdn.microsoft.com/old...21-00/?p=92911

    I'm more immersed in Windows developer culture than Linux/macOS, so it was easier for me to find (or already know about) the justifications for removing it on Windows, but the situation would be similar on the other OSes. The flag was advisory at best and was silently ignored if the file system didn't support it. In those "not supported" cases, BC was still much slower if the flag was on because we were using smaller read buffers. Since it was intended for already slow floppy drives, we didn't put effort into optimizing the reads, and since it was "working", there wasn't much incentive to improve it.

    If we wanted to keep it, we could redo everything to do optimized reads and theoretically improve I/O for large comparisons, but IMO, that's a new feature request, not a bug fix, and one that would have to compete with all of the other items in the wishlist.
    ZoŽ P Scooter Software

  3. #3
    Join Date
    Dec 2007
    Location
    U.S. East coast
    Posts
    295

    Default

    ZoŽ, thanks for the detailed answer.

  4. #4
    Join Date
    Jun 2009
    Posts
    11

    Default

    Until now I was relying on the 'bypass disk cache' option to ensure that a newly connected USB thumb drive is actually read and not data from any cache.
    What would now be the correct way to test multiple USB thumb drives (same hardware, same volume label etc.) which are expected to contain identical content against reference data from the hard drive?

  5. #5
    Join Date
    Oct 2007
    Location
    Madison, WI
    Posts
    11,376

    Default

    Hello,

    Are you seeing different behavior with the option disabled/removed? As Zoe mentioned above, the initial use of the option was for floppy disks, and it was not intended or supported by Microsoft for newer types of access. Even with the option enabled, it is possible to get cached results back.
    Aaron P Scooter Software

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •