No announcement yet.

Snapshot / making sure you know if a backup changes, etc.

  • Filter
  • Time
  • Show
Clear All
new posts

  • Snapshot / making sure you know if a backup changes, etc.

    I'm revisiting this concern I have.

    When making an image of a drive (using, say shadowprotect), you wind up with a 1 huge file. On a windows 10 pro NTFS drive, would anyone know how resiliant a file is till it's unreadable / unusable? Is this even a legitimate fear? That 1 bit of the gigabits that make up a huge file flips? would that be enough for a file to be unusable?

    a) is that conceivable? That over time, that 1 or more bits on an HD, SSD or maybe a dvd disk flips from 1 to 0 or the other way. On a file that you aren't using. write it to the disk, keep using that disk to write other images / files on the drive. come back 1 - 2 - 3 years later.... would you expect every bit to be as written previously? Or spurious noise, quality of the magnetic material, etc will make a bit unreadable or flipped? (I guess unreadable is another reason a file could get trashed?

    b) and what it takes to be unusable would depend on the app that wrote it / will read it? The app may have error correction better than other apps?

    I think an answer to at least knowing if the file changed is to save a snapshot of it?

    Trying that out now, I took a snapshot of a folder with 26 files, 2.6GB. All the boxes were checked (CRC, etc.). It took a while to create it and I have a file that's 975 bytes! Seems too small! : )

    I then mounted it on 1 side and the folder on the other. File compare (CRC) took a split second. That seems to be because the real files CRC was cached?

    Closing then opening the app, the file compare took longer.

    c) in the file compare window, there's CRC, binary and rules based comparison options. binary is the most detailed comparison (although now I see rules based INCLUDES binary compare?!)? But would take the longest?

    To compare based on CRC, BC has to create the CRC for files (not the snapshot?). Is that quicker than binary considering you have to read the entire file AND calculate the CRC?

    d) if I copy those files from external hard drive A to external hard drive B, and do a CRC compare between them, how would you think the time to complete would compare to doing a CRC of the external drive A data vs. the snapshot? And in theory, the snapshot of A should be usable for the copied data on B? sector size / drive format type, etc are irrellevant to CRC (and binary?) compares?

    THANKS! Stay safe!

  • #2
    A) It's possible, though in personal experience it's more about hardware failure than over-use. The OS itself has some error correction built-in, but it's never a bad idea to have multiple copies/backups, in case you find one that's bad you can restore from another.

    B) Also yes, resilient apps may have their own methods of trying to protect and read files, but I'm not familiar with shadowprotect.

    *BC Snapshots are not backups of the files. They are a moment in time representation of the folder structure and store the metadata (timestamp, crc optional, etc), but no file content. This way, you can see what has changed, but there's no way to restore from a Snapshot. It's a virtual representation.*

    C) Snapshots only support storing CRC codes, so you would want/need to use a CRC comparison. This generates the CRC for the live side, and uses the saved CRC embedded within the snapshot. If both sides are Live data, then binary can be a little faster, since it can bail out of the scan as soon as it hits a difference; the CRC generation requires scanning the entire file to generate the code, and then compares the code. Since one side is already generated if comparing against a snapshot, then you are only generating a single side in that case. This leads into D), a CRC against a snapshot would be faster, but that's also because you already did the work of generating the snapshot; if you don't already have it then binary would be faster if scanning real/live data on the external to another real/live data set on another external.

    The snapshot would let you know if you compare the live data back to the older snapshot what has changed. Is this a scenario you are looking to accomplish? Then, after finding the differences and acting on them, you can generate a new snapshot to use next time. But the key is, the snapshot itself has no file data, so you still need the real files/folders on the external drives.
    Aaron P Scooter Software


    • #3
      Aaron - thanks for all that info.

      Yes, the snapshots are only the metadata : ) being curious, I did double clicked on a pair of files - in the snapshot and live. And yeah, snapshot was... duh... blank : )

      And I was doing several compares and noting the times binary vs. crc of live data on both sides & . crc against a snapshot

      I learned about snapshots here in a post a while ago... what would you say is best practices with backups of a drive to another drive (so lots of data) and using BC to do that... (family movies, etc that don't change much at all.).

      Now you have the 2 drives of the same data. if you come back in a year, and do a compare of the files on the drives and it finds a difference, you have to wonder where the change occured. Yes there's a difference, but is it on the backup or original data?. I think that's when someone mentioned snapshots. You could do a snapshot and then in a year, compare both drives to the year old snapshot and hopefully (at least) 1 of the drives will match the snapshot. right? Same for several backups... just make a snapshot of the drive. Although.... the snapshot file could change?! : )

      all my fears and scenarios... and procrastinate actually doing something!.



      • #4

        Yes, if it's a corruption (so the timestamp is still equal) it would be trickier to know without opening the file and manually verifying (which assumes it corrupted in a way that then renders it unusable). The snapshot saving the CRC is a good way to verify which side corrupted, by comparing it to its past snapshot.
        Aaron P Scooter Software