Announcement

Collapse
No announcement yet.

Mirroring script deletes file with no reason

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • Mirroring script deletes file with no reason

    Hi,
    I'm new to the forum.
    After reading on this forum, I created a simple mirroring script to mirror a set of shared folders from a Windows Server 2012 R2 Essentials to a remote PC connected via VPN.
    On the remote PC, I automatically run this script every night using Task Scheduler:
    log verbose append:"C:\Users\***\Desktop\Mirror Log.txt"
    load "Mirror"
    option confirm:yes-to-all
    sync mirror:left->right
    The left side is the selected shared folders on the Windows Server 2012 R2 Essentials
    The right side is the mirror location on the remote PC.

    I've been using this script with BC4.1.3 for the past two weeks without a problem.
    Yesterday, the script completely erased one of the shared folders on the mirror side without reason (there were no change in that shared folder).
    I don't know if it's related, but I upgraded to BC4.1.4 before the script ran yesterday.
    Is there something to be done to the script after upgrading?
    It's also possible the vpn link went down during the comparison process.
    Is it possible BC4 decided to delete a folder because it couldn't see it due to the vpn not connected?

    Please help me explain this behavior.

  • #2
    Yes, the Sync Mirror will delete files on the right if they do not exist on the left. How would a failure to load appear on your VPN? Could this give us a folder structure with folders missing? Or would the folder appear but have an error, as if Access was Denied? This would depend a bit on the remote device and how it presents the error.

    The difference between a Sync Mirror and Update:
    Sync Update will copy any Newer or Orphan files from Source to Destination.
    Sync Mirror will copy any Newer, Older, Different and Orphan Files from Source to Destination. Will also delete any Orphan files in the Destination.
    Aaron P Scooter Software

    Comment


    • #3
      Thanks for the reply.

      I need Sync Mirror because I want an exact copy of the shared folders on the remote PC.

      While preparing my response to your reply, I checked the script log again and found this (I missed it in the first pass):
      2016-03-07 04:53:18 Unable to load \\Q\Shared Folders\Software: The network path was not found

      This is the folder that was completely removed from the mirror.
      Should I understand that BC considered this folder as "removed" because it couldn't find it due to a network error?
      If this is the case, how can I prevent BC from acting on this folder, or all folders when this kind of error is found?

      Thanks

      Comment


      • #4
        Hello, BC 4.1.4 and earlier have a bug which might trigger this behavior if the error occurs mid-load as a timing error. Usually, if a folder fails to load it would be marked as an error, it wouldn't result in deleting files in the destination. We have a fix included with the next release which will help, but may not solve for all scenarios. Are you able to reproduce this error?
        Aaron P Scooter Software

        Comment


        • #5
          Because of the quantity of data (over 4 TB to compare), it's kind of impractical to reproduce intentionally.

          So far, I've seen it only once during the comparison load. I've seen it more often at the beginning of the comparison load. When that happens, BC simply stops. The comparison load takes about 3 hours (over 4 TB to compare). Therefore, loosing the vpn in the middle of the process is bound to happen again. Is there a way, in the script or BC settings, to indicate what to do when the network path is not found?

          Comment


          • #6
            The intended behavior is to log the error in script and not perform sync actions on the folder with the error, so we wouldn't delete on the other side. BC 4 has detection code to try and find various issues that can pop up, and there is expanded detection code included in the next release (4.1.5). Let us know if this occurs again, or you find a reliable means of reproducing the error after the initial load.
            Aaron P Scooter Software

            Comment


            • #7
              As I mentioned earlier, I can't afford to loose 0.5 TB when the problem occurs. It's too much data to transfer via VPN. I'd rather wait for your next release. On the other hand, if you have a beta version with this problem fixed, I could give it a try if you want.

              Comment


              • #8
                Do you still have access to the full log when this issue occurred? We may need to see this issue from the start to better know if/how we've tackled it. You can post here or you can email us at support@scootersoftware.com with a link back to this forum thread if you need to submit it privately.
                Aaron P Scooter Software

                Comment


                • #9
                  Hi, the original log is pretty long. I edited the log to remove repetitive actions like successful copy and successful delete.
                  You'll see three different behaviors.
                  March 3rd = unable to load comparison from the start resulting in a script failure.
                  March 5th = successful completion of the synchronize process.
                  March 7th = unwanted delete of a shared folder because BC couldn't load it, but was able to access the other shared folders.

                  2016-03-02 02:00:02 >> log verbose append:"C:\Users\*\Desktop\Mirror Log.txt"
                  2016-03-02 02:00:02 >> load "Mirror"
                  2016-03-02 02:00:09 Unable to load \\Q\Shared Folders\: The network path was not found
                  2016-03-02 02:00:09 Load comparison: <-> E:\
                  2016-03-02 02:00:09 Fatal Scripting Error: Unable to load base folder
                  2016-03-02 02:00:09 Script completed in 7,46 seconds

                  2016-03-05 02:00:02 >> log verbose append:"C:\Users\*\Desktop\Mirror Log.txt"
                  2016-03-05 02:00:02 >> load "Mirror"
                  2016-03-05 04:40:45 Load comparison: \\Q\Shared Folders <-> E:\
                  2016-03-05 04:40:46 >> option confirm:yes-to-all
                  2016-03-05 04:40:46 >> sync mirror:left->right
                  2016-03-05 04:47:13 Copied \\Q\Shared Folders\Pictures\a.pdf to E:\Pictures
                  2016-03-05 04:47:15 Copied \\Q\Shared Folders\Pictures\b.pdf to E:\Pictures
                  2016-03-05 04:47:16 Copied \\Q\Shared Folders\Pictures\c.jpg to E:\Pictures
                  2016-03-05 04:47:16 Copied \\Q\Shared Folders\Pictures\d.pdf to E:\Pictures
                  2016-03-05 04:47:17 Copied \\Q\Shared Folders\Pictures\e.jpg to E:\Pictures
                  2016-03-05 04:47:17 more files copied
                  2016-03-05 04:49:08 Deleted E:\Pictures\f.bmp
                  2016-03-05 04:49:08 Deleted E:\Pictures\g.jpg
                  2016-03-05 04:49:08 Deleted E:\Pictures\h.jpg
                  2016-03-05 04:49:08 Deleted E:\Pictures\i.jpg
                  2016-03-05 04:49:08 Deleted E:\Pictures\j.jpg
                  2016-03-05 04:49:08 more files deleted
                  2016-03-05 04:49:10 Copied \\Q\Shared Folders\Pictures\k.jpg to E:\Pictures
                  2016-03-05 04:49:10 Copied \\Q\Shared Folders\Pictures\l.jpg to E:\Pictures
                  2016-03-05 04:49:10 more files copied
                  2016-03-05 06:35:09 Copied \\Q\Shared Folders\Videos\m.mp4 to E:\Videos
                  2016-03-05 06:35:09 Successfully synchronized 622 items. Completed in 1 hours, 54 minutes.
                  2016-03-05 06:35:10 Script completed in 4 hours, 35 minutes

                  2016-03-07 02:00:00 >> log verbose append:"C:\Users\*\Desktop\Mirror Log.txt"
                  2016-03-07 02:00:01 >> load "Mirror"
                  2016-03-07 04:53:18 Unable to load \\Q\Shared Folders\Software: The network path was not found
                  2016-03-07 06:07:22 Load comparison: \\Q\Shared Folders <-> E:\
                  2016-03-07 06:07:31 >> option confirm:yes-to-all
                  2016-03-07 06:07:36 >> sync mirror:left->right
                  2016-03-07 07:47:24 Deleted E:\Software\a.ini
                  2016-03-07 07:47:28 Deleted E:\Software\b.txt
                  2016-03-07 07:47:29 all E:\Software\ files and folders deleted
                  2016-03-07 08:40:55 Copied \\Q\Shared Folders\Games\a.exe to E:\Games
                  2016-03-07 08:48:44 more files copied
                  2016-03-07 10:48:01 Successfully synchronized 243 641 items. Completed in 4 hours, 40 minutes.
                  2016-03-07 10:48:06 Script completed in 8 hours, 48 minutes

                  Comment


                  • #10
                    Hello,

                    Thanks. The log is helpful, and this should be fixed in the next release of BC4.
                    Aaron P Scooter Software

                    Comment


                    • #11
                      Beyond Compare 4.1.5 is now available with a fix for this bug.
                      Chris K Scooter Software

                      Comment


                      • #12
                        Hi, I installed version 4.1.5 last week. I think the fix is working. The problem occurred this weekend and this is the log:

                        2016-03-27 02:00:33 >> log verbose append:"C:\Users\***\Desktop\Mirror Log.txt"
                        2016-03-27 02:00:34 >> load "Mirror"
                        2016-03-27 03:26:27 Unable to load \\Q\Shared Folders\Videos: The network path was not found
                        2016-03-27 03:28:14 Load comparison: \\Q\Shared Folders <-> E:\
                        2016-03-27 03:28:14 >> option confirm:yes-to-all
                        2016-03-27 03:28:14 >> sync mirror:left->right
                        2016-03-27 03:28:56 Unable to load \\Q\Shared Folders\Videos: The network path was not found
                        2016-03-27 03:32:00 Script completed in 1 hours, 31 minutes


                        From this log, can you confirm the problem is fixed?

                        Comment


                        • #13
                          The fix submitted helps prevent the deletes from happening if the targets fail to load. The load error "The network path was not found" is an issue with the network locations. What error are you running into now? Is it that some directories fail to load and the comparison continues after excluding them (expected), or that the Mirror is still deleting these folders in the destination (error)?

                          Note: if a base folder fails to load, this is a fatal error and the entire script would exit.
                          Aaron P Scooter Software

                          Comment


                          • #14
                            Hi. I'm sorry about my last reply. It wasn't very clear.

                            As I mentioned earlier, I use Beyond Compare to synchronize a set of shared folders from a Windows Server Essentials to a remote PC thru a VPN across the Internet. The VPN is not stable. It disconnects and reconnects randomly. When that happens during the synchronization process, I get The network path was not found errors for a folder or the entire set.

                            The excerpt from my log shows that on March 27th one folder was unable to load during the comparison and BC 4.1.5 excluded that folder from the synchronization process. As a result, the corresponding destination folder wasn't synchronized nor deleted on the remote PC. This is the expected behavior. The fix provided by version 4.1.5 is working well.

                            Thank you very much.

                            Comment

                            Working...
                            X