Announcement

Collapse
No announcement yet.

SSH Syncronization never works with files modified at 8:00pm

Collapse
X
 
  • Filter
  • Time
  • Show
Clear All
new posts

  • SSH Syncronization never works with files modified at 8:00pm

    This is a really odd issue. I have some understanding of what's going on, but not all.

    For some reason, every single file that has a modification time of 8:00pm, once copied over to my server I'm connecting to over SSH, is seen by BeyondCompare to be actually at midnight the next day on the destination side. I transfer it over, and then when I close BeyondCompare and reopen it again to run a sync, it shows up like below again and again, never fully syncing.

    Click image for larger version

Name:	2020-05-27 23_32_03-UnRaid 3_ HadSav - Folder Sync - Beyond Compare.png
Views:	124
Size:	123.9 KB
ID:	83506
    Because of this 4 hour offset, I figured it was because my current timezone with daylight savings (EDT) is -4 hours behind UTC.
    Below is the setting I am using for the Time Zone for the SSH server. I've changed it to Automatically Detect, and the same problem happens. If I change it to my actual time zone (UTC-5) then it gets all the times super wrong (on both source and destination), and is forcing me to transfer all the files all over again.

    Click image for larger version

Name:	2020-05-27 23_33_59-Profiles.png
Views:	73
Size:	10.8 KB
ID:	83507

    Is there some setting I'm missing or is this a bug? This only happens with SSH, so I wonder if there's something I need to change on the SSH server itself. Any thoughts?

  • #2
    Hello,

    One key for troubleshooting: if you make any changes to the Profile, close and restart the entire BC4 application, just to be sure nothing is cached when connecting. When you update the timestamp in the Profile's Server tab, this should impact the listing on that side, but shouldn't alter the file listing of the local side. If it does, how does D:\'s displayed Modified Timestamp compare to what Windows Explorer's Properties shows?

    Your first hunch is mine, too, but it would apply to all files, not just 8:00pm. From the screenshot, are these the leftover, only 8pm files? And if you have a file with any other timestamp, what does it do to that timestamp? Also plus 4 hours, but doesn't land on exactly midnight?

    For troubleshooting from the server side, it is important to have a distinction from the SSH server and the SFTP protocol. BC4 (and other SFTP clients) are connecting via SFTP, which is something the SSH server can run in addition to being an SSH server. SSH servers also support connecting via ssh (like Putty) on the Terminal, but this may provide different results as it isn't exactly the same connection method.
    Aaron P Scooter Software

    Comment


    • #3
      Hi Aaron,

      I actually had been closing and restarting BC4 before retrying, because I noticed that changes didn't take in the past.

      In Explorer, the timestamp shows 7pm (I'm guessing this is because I'm in EDT Daylight savings time. I'm thinking it would read 8pm in December when it's EST here. Peeking at the other files, you can see that the time is off by an hour for the other files too. For some reason BC4 is fine with all the other files with other timestamps, just 8pm AKA midnight UTC is causing the issue.

      Click image for larger version

Name:	2020-05-29 16_02_48-UnRaid 1_ MontOct - Folder Sync - Beyond Compare.png
Views:	55
Size:	16.2 KB
ID:	83523
      The below is what the files look like in D:\ and also what they look like in the SFTP share if I mount it with Samba.
      Click image for larger version

Name:	2020-05-29 16_00_17-D__Stock Audio_Sonniss_Sonniss.com - GDC 2019 - Game Audio Bundle Part 7of8_2496.png
Views:	73
Size:	9.3 KB
ID:	83522
      Regarding the SSH, yeah, I am using SFTP to connect. If I connect with putty and navigate to that folder, here is what I see.

      Click image for larger version

Name:	2020-05-29 16_09_24-root@Komodo_ _mnt_user_MontOct_Stock Audio_Sonniss_Sonniss.com - GDC 2019 - Game.png
Views:	55
Size:	15.8 KB
ID:	83524

      Does this help you figure out anything? Please let me know if there's anything else I could share that would help with troubleshooting. Thanks in advance!

      Comment


      • #4
        Hello,

        That's particularly odd that (some) timestamps are different between all three displays, in different ways. Putty itself would be an example of SSH (not SFTP) connection, but it is still a good test. You could install an SFTP client like Filezilla to test that protocol; if downloading, be sure to use the Alternate Downloads link to download the regular setup.exe and not the sponsored.exe.

        Should the server be UTC? Or should the profile be set to Eastern? If updating this setting, fully close BC4 and restart the application before testing a file transfer. It's good to double check existing files, but also check newly transferred files (fresh, new copies, not overwrites) and check the timestamp before and after a program restart.

        Is this a server we could get a test account to?

        Aaron P Scooter Software

        Comment


        • #5
          Hi Aaron,

          I'm using UnRaid, and my server time settings give methe option to choose my time zone. If I change it to UTC, everything will be off.Click image for larger version

Name:	2020-06-04 14_29_39-Komodo_DateTime.png
Views:	44
Size:	28.5 KB
ID:	83560

          I tried futzing around with some more settings, and I think I found a workaround. Originally, my "Comparison" settings were as listed below

          Click image for larger version

Name:	2020-06-04 14_27_27-Home - Beyond Compare.png
Views:	50
Size:	7.2 KB
ID:	83558

          However, when I changed it to the below settings, it forces a full binary comparison, and then those 8pm/midnight files don't show up. So it seems that the files are in fact exactly the same, but the time is just showing up oddly.

          Click image for larger version

Name:	2020-06-04 14_28_23-Home - Beyond Compare.png
Views:	44
Size:	6.1 KB
ID:	83559
          I'm not sure I feel comfortable opening up my server to the outside to allow remote connections. Is there anything that you can gather here about fixing the time situation? OR I guess it just makes sense for me to have it do a full binary check on those files, which slows the process down a bit, but not too much, I guess.

          Comment


          • #6
            Hello,

            Ok, so the server is set to Eastern. If you update the BC4 Tools menu -> Profiles dialog, specific profile -> Server tab, and set to Eastern, then restart BC4: what does this show? I'm re-reading the first post, and was this the initial profile setting?

            Are there any updates available for Unraid?
            Aaron P Scooter Software

            Comment


            • #7
              I just tried doing what you recommended. It shows the wrong time on the local side (off by an hour from what Windows Explorer shows) and also the wrong time on the SFTP side (off by 5 hours). It forces me to copy over every file again. So I ran the transfer with just one folder as a test, and it copied everything over, but then still shows the wrong times. No matter how many times I click "sync now", it keeps showing what you see below:

              Click image for larger version

Name:	2020-06-06 17_34_00.png
Views:	45
Size:	235.0 KB
ID:	83588

              Comment


              • #8
                I think setting back as UTC and using the Binary scan if quick scan shows differences is a good workaround. Binary would indicate the files are exactly the same.

                One other test would be to verify against Filezilla (SFTP-only client software) to check its behavior. Filezilla is free, but please carefully select the Alternate Download and avoid the sponsored download link (which installs a bunch of extra stuff). Or if you have another SFTP client (not Putty or WinSCP), that would be a good test, too.

                We do have some additional debug logging you can enable in the Options dialog, Tweaks tab, Log Panel section, but we aren't able to capture as much from an SFTP connection. If this does look useful, though, you can email into support@scootersoftware.com along with the captured log.
                Aaron P Scooter Software

                Comment


                • #9
                  Here's that Filezilla shows for that folder. Seems it's the same as what Windows Explorer shows.


                  Click image for larger version

Name:	2020-06-11 19_10_59-Komodo - sftp___root@192.168.0.106_57922 - FileZilla.png
Views:	40
Size:	18.3 KB
ID:	83622
                  And I've attached the log file with me syncing without the "skip if quick tests indicate files are the same". And it looks like everything syncs properly (even though if I quit and re-open BC4, those same stragglers still show up as needing to be transfered). In the same log file I ran the sync with "skip if quick tests indicate files are the same" checked and it goes through with no copying needed.
                  Attached Files

                  Comment

                  Working...
                  X