Folder/file display taking a very longt time ?

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • peterr
    Fanatic
    • Nov 2004
    • 142

    Folder/file display taking a very longt time ?

    I have noticed BC4 very slow lately. Today I opened a new session, compare files/folders, just on the /home/*****/Downoads path. This usually takes a few seconds to display, but it has already been over 7 minutes and no display as yet. CPU load for BC4 was high, up to 12% of total, 100% of one cpu.

    Here are some details - snapshot whilst BC4 is running ..

    Code:
    $ inxi -Fxz
    System:    Host: ********-Inspiron-3542 Kernel: 4.4.0-104-generic x86_64 (64 bit gcc: 5.4.0)
               Desktop: KDE Plasma 5.5.5 (Qt 5.5.1) Distro: Ubuntu 16.04 xenial
    Machine:   System: Dell (portable) product: Inspiron 3542
               Mobo: Dell model: 0WW73H v: A01 Bios: Dell v: A01 date: 03/28/2014
    CPU:       Dual core Intel Core i7-4510U (-HT-MCP-) cache: 4096 KB
               flags: (lm nx sse sse2 sse3 sse4_1 sse4_2 ssse3 vmx) bmips: 7980
               clock speeds: max: 3100 MHz 1: 808 MHz 2: 893 MHz 3: 809 MHz 4: 813 MHz
    Graphics:  Card-1: Intel Haswell-ULT Integrated Graphics Controller bus-ID: 00:02.0
               Card-2: NVIDIA GM108M [GeForce 840M] bus-ID: 08:00.0
               Display Server: X.Org 1.18.4 drivers: intel (unloaded: fbdev,vesa) Resolution: [email protected]
               GLX Renderer: Mesa DRI Intel Haswell Mobile GLX Version: 3.0 Mesa 17.2.4 Direct Rendering: Yes
    Audio:     Card-1 Intel 8 Series HD Audio Controller driver: snd_hda_intel bus-ID: 00:1b.0
               Card-2 Intel Haswell-ULT HD Audio Controller driver: snd_hda_intel bus-ID: 00:03.0
               Sound: Advanced Linux Sound Architecture v: k4.4.0-104-generic
    Network:   Card-1: Qualcomm Atheros QCA9565 / AR9565 Wireless Network Adapter driver: ath9k bus-ID: 06:00.0
               IF: wlp6s0 state: down mac: <filter>
               Card-2: Realtek RTL8101/2/6E PCI Express Fast/Gigabit Ethernet controller
               driver: r8169 v: 2.3LK-NAPI port: e000 bus-ID: 07:00.0
               IF: enp7s0 state: up speed: 100 Mbps duplex: full mac: <filter>
               Card-3: Atheros usb-ID: 001-007
               IF: null-if-id state: N/A speed: N/A duplex: N/A mac: N/A
    Drives:    HDD Total Size: 1000.2GB (46.1% used) ID-1: /dev/sda model: ST1000LM024_HN size: 1000.2GB
    Partition: ID-1: / size: 17G used: 7.9G (52%) fs: ext4 dev: /dev/sda1
               ID-2: /home size: 896G used: 417G (50%) fs: ext4 dev: /dev/sda6
               ID-3: swap-1 size: 5.00GB used: 0.00GB (0%) fs: swap dev: /dev/sda5
    RAID:      No RAID devices: /proc/mdstat, md_mod kernel module present
    Sensors:   System Temperatures: cpu: 47.0C mobo: N/A
               Fan Speeds (in rpm): cpu: N/A
    Info:      Processes: 192 Uptime: 14 min Memory: 1192.6/7887.7MB Init: systemd runlevel: 5 Gcc sys: 5.4.0
               Client: Shell (bash 4.3.481) inxi: 2.2.35
    There have been other speed issues with other applications the last few days though. Thought it was CPU temp, but 47.0C at present is fine. It's now been 12 minutes, still no display, but CPU for BC4 is down to 1% now. I just displayed the same folder with Dolphin, it took less than 1 second. Debug is on - it tells me nothing.

    ...Later - I pressed 'Stop' over 5 minutes ago, yet BC4 is still active. I have noticed this a lot in the past, and have had to kill the process from the system monitor.

    Much later: I started a BC4 seesion (File/folders) and specified the same path. Came back 1 hr and 11 minutes later and it still hasn't displayed the contents. Killed the process
    Last edited by peterr; 09-Jan-2018, 11:23 PM.
  • peterr
    Fanatic
    • Nov 2004
    • 142

    #2
    As you can see by the last 'edit' line in the post above, I was basically unable to use BC4 at all. That said, if I did specify a small path with no folders beneath it, it would display. But unable to navigate up from there.

    I did try a reinstall the other day, but no difference, so figured there may be something corrupted in the config files.

    Backed up the $home/.beyondcompare path and also did a backup from within BC4. Used synaptic to completely uninstall BC4, removed the $home/.beyondcompare path (as it was backup up), then synaptic to install. Went into BC4 and tried the /$home/Downloads path, and the display was instantaneous.

    So, the question is, what was corrupt in the settings/config files ??

    Also how do I restore the settings that were in use before, .. WITHOUT adding that possible (settings) corruption ??

    Comment

    • Dave_L
      Veteran
      • Dec 2007
      • 348

      #3
      Have you tried using BC to compare the good config files with the bad ones? That might provide a clue.

      Comment

      • peterr
        Fanatic
        • Nov 2004
        • 142

        #4
        Originally posted by Dave_L
        Have you tried using BC to compare the good config files with the bad ones? That might provide a clue.
        Yes I did try that. There is one XML file called sessions and basically the new one doesn't have much data in it at all, because that was the first session and the backup had a lot of data with sessions. The other file was BCStat.xml which again, comparing a new one of 1,660 bytes with only a few lines against a 8,978 byte file with many lines, I couldn't see anything to hint at a possible problem.

        The other file was from the export before the new installation, it is called BCSettings.bcpkg and unfortunately it is binary. Would be nice if it could be converted to (say) XML, then I could compare it against a 'new' export file.

        Comment

        • peterr
          Fanatic
          • Nov 2004
          • 142

          #5
          Just opened the file BCSettings.bcpkg with Ark and realised it is just a backup of those XML files. Did an export and will compare the old fileset to the new. These backups contain ..

          BCColors.xml
          BCCommands.xml
          BCFileFormats.xml
          BCPreferences.xml
          BCProfiles.xml
          BCSessions.xml
          BCState.xml
          BCSysInfo.txt

          The first 3 equal, and now I will look at the others. There is one (slight) possibility of a setting (in the old settings) that, as I viewed some of the XML files. It is to do with CRC, and now I'm wondering if BC4 was attempting to do a CRC on all those files ??

          Comment

          • Aaron
            Team Scooter
            • Oct 2007
            • 15941

            #6
            Hello,

            In the Session menu -> Session Settings, Comparison tab, if CRC comparison is enabled, it will try to scan/create all CRC codes.

            If the CRC column is visible (Name, Date, Size are default), then it will try to generate codes for all visible items as you scroll around. You can right click the column header to disable it.

            Are either of these enabled?
            Aaron P Scooter Software

            Comment

            • peterr
              Fanatic
              • Nov 2004
              • 142

              #7
              Originally posted by Aaron
              In the Session menu -> Session Settings, Comparison tab, if CRC comparison is enabled, it will try to scan/create all CRC codes.
              It says 'Rules based' - that is because I removed BC$ and all the config/settings and re-installed. So it is currently showing the default.

              Originally posted by Aaron
              If the CRC column is visible (Name, Date, Size are default), then it will try to generate codes for all visible items as you scroll around. You can right click the column header to disable it.
              The CRC column isn't visible, reason being the same as above.

              However, I still feel what happened is focused on CRC. I usually don't use CRC, because I know it takes so long, and binary compare works for me. But the other day, I was running "fdupes" to quickly find any duplicate files. But I like to have BC4 confirm it, so ran a CRC session. It took a while to display the CRC, as a lot of those duplicates were over 1 Gb.

              All I did to enable it though was to simply 'view' CRC and then visually check that the CRC's were the same. Any that were the same, I deleted one of them.

              So that type of session was okay, maybe 5 to 10 in total, and as it was a folder that had no other sub folders, there was no 'recursive' CRC type activity that would take a long time. Session was okay.

              Then as it takes so long for CRC, I removed the 'view CRC' from the column view, just so that any subsequent sessions were simply on either 'rules based' or binary.

              However, the next session was the one that just took for ever, no display, and still no display 1 hr , 11 minutes later.

              This may help in your investigation. When I checked the XML file with sessions. I will attempt to explain ..

              When I was using CRC as a column view, here is a cutdown of the session/state in the XML file

              <State>
              <ColAttributesVisible Value="False"/>
              <ColAttributesWidth Value="0"/>
              <ColCRCVisible Value="True"/>
              <ColCRCWidth Value="0"/>
              <ColExtVisible Value="False"/>
              <ColExtWidth Value="0"/>
              <ColModifiedVisible Value="True"/>
              <ColModifiedWidth Value="0"/>
              <ColPathWidth Value="0"/>
              <ColSizeVisible Value="True"/>
              <ColSizeWidth Value="0"/>
              <ColUnixGroupVisible Value="False"/>
              <ColUnixGroupWidth Value="0"/>
              <ColUnixOwnerVisible Value="False"/>
              <ColUnixOwnerWidth Value="0"/>
              <ColVcsStatusVisible Value="False"/>
              <ColVcsStatusWidth Value="0"/>
              <ColVcsVersionVisible Value="False"/>
              <ColVcsVersionWidth Value="0"/>
              <ColVersionVisible Value="False"/>
              <ColVersionWidth Value="0"/>
              <DirHandling Value="dirCompareStructure"/>
              <DisplayFilter Value="[showSame, showLeftAdded, showRightAdded, showLeftChanged, showRightChanged, showConflict]"/>
              <InsigIgnored Value="False"/>
              <SortCol Value="colCRC"/>
              <SortDown Value="False"/>
              <SortSide Value="dsRight"/>
              </State>
              CRC column is visible and the column sort value is CRC. This session went okay.

              Now the session that hung ..

              <State>
              <ColAttributesVisible Value="False"/>
              <ColAttributesWidth Value="0"/>
              <ColCRCVisible Value="False"/>
              <ColCRCWidth Value="0"/>
              <ColExtVisible Value="False"/>
              <ColExtWidth Value="0"/>
              <ColModifiedVisible Value="True"/>
              <ColModifiedWidth Value="0"/>
              <ColPathWidth Value="0"/>
              <ColSizeVisible Value="True"/>
              <ColSizeWidth Value="0"/>
              <ColUnixGroupVisible Value="False"/>
              <ColUnixGroupWidth Value="0"/>
              <ColUnixOwnerVisible Value="False"/>
              <ColUnixOwnerWidth Value="0"/>
              <ColVcsStatusVisible Value="False"/>
              <ColVcsStatusWidth Value="0"/>
              <ColVcsVersionVisible Value="False"/>
              <ColVcsVersionWidth Value="0"/>
              <ColVersionVisible Value="False"/>
              <ColVersionWidth Value="0"/>
              <DirHandling Value="dirCompareStructure"/>
              <DisplayFilter Value="[showSame, showLeftAdded, showRightAdded, showLeftChanged, showRightChanged, showConflict]"/>
              <InsigIgnored Value="False"/>
              <SortCol Value="colCRC"/>
              <SortDown Value="False"/>
              <SortSide Value="dsRight"/>
              </State>
              We have moved from CRC column visible (session works okay) to CRC column not visible, ...YET we still have CRC as the sort column. Was BC4 still trying to determine the CRC for all those files ?? In a recursive nature ? That folder has 358.1 Gb, 66,715 files and 6,993 sub folders.

              Comment

              • Aaron
                Team Scooter
                • Oct 2007
                • 15941

                #8
                Hello,

                Thanks! I think that's the key. We'll look into how you can get into this state. In the meantime, if you load the problem session and set to sort on a visible column (Name), then re-save the Session, this fixes it for these sessions, correct?
                Aaron P Scooter Software

                Comment

                • peterr
                  Fanatic
                  • Nov 2004
                  • 142

                  #9
                  Originally posted by Aaron
                  Hello,

                  Thanks! I think that's the key. We'll look into how you can get into this state. In the meantime, if you load the problem session and set to sort on a visible column (Name), then re-save the Session, this fixes it for these sessions, correct?
                  The problem session /$HOME/Downloads loads instantaneously now, because I removed everything and reinstalled. Fresh start. However, what you have described, should fix it. I remember I did actually sort of Name after removed 'view col CRC', but that did not fix it. The last function you have mentioned, 'saving' must be essential I presume.

                  I can probably send you all the files before the problem and 'current', but will have to go through them and reove any references to confidential info. However, I'm sure the scenario could be replicated by ..

                  Pick a folder that has many sub folders, select view CRC column, then see what happens ?? Exit BC4 and then try selecting the same folder, without 'view CRC column'. That was pretty much the scenario.

                  Comment

                  • Aaron
                    Team Scooter
                    • Oct 2007
                    • 15941

                    #10
                    Hello,

                    To confirm your steps, you have to also Sort by CRC first, correct? It's not just enabling CRC, but enable and sort, and then remove the column without setting another column to be sorted. If the sorted column is crc, this is the slow load scenario I've found. If another column is set to sort during the load and crc is hidden, it should be quick.
                    Aaron P Scooter Software

                    Comment

                    • peterr
                      Fanatic
                      • Nov 2004
                      • 142

                      #11
                      Originally posted by Aaron
                      To confirm your steps, you have to also Sort by CRC first, correct? It's not just enabling CRC, but enable and sort, and then remove the column without setting another column to be sorted. If the sorted column is crc, this is the slow load scenario I've found. If another column is set to sort during the load and crc is hidden, it should be quick.
                      The default columns to display are size and modified. If I add another column, in this case, CRC, and then remove CRC column view, I'm left with the defaults, size and modified.

                      To answer your question, yes I used sort by CRC as that is how I picked up the duplicates. Just a manual visual check. But then I unchecked CRC column view and was left with the defaults, size and modified.

                      I'm not sure if I then clicked either size or modified, because name/filename is always the default sort view. I'm not keen to try and replicate it as BC4 works fine now and I'm not _real_ willing to have it hang on me.

                      Comment

                      • peterr
                        Fanatic
                        • Nov 2004
                        • 142

                        #12
                        I sent an email to Scooter software, containing a cut down version of the sessions XML file, so hopefully you can replicate the problem.

                        Comment

                        • Aaron
                          Team Scooter
                          • Oct 2007
                          • 15941

                          #13
                          Hello,

                          Looks like it, with those settings for sorting by CRC but the column isn't visible is the issue.
                          Aaron P Scooter Software

                          Comment

                          • peterr
                            Fanatic
                            • Nov 2004
                            • 142

                            #14
                            Originally posted by Aaron
                            Looks like it, with those settings for sorting by CRC but the column isn't visible is the issue.
                            Have you been able to replicate the problem.

                            Comment

                            • Aaron
                              Team Scooter
                              • Oct 2007
                              • 15941

                              #15
                              Yes, we've repeated it. Have you workaround fixed the saved session by loading it and clicking on Name or another column to sort other than the (invisible) CRC column?
                              Aaron P Scooter Software

                              Comment

                              Working...