The FTP files ends up in the wrong remote directory!

Collapse
X
 
  • Time
  • Show
Clear All
new posts
  • Geoff McLane
    Journeyman
    • May 2020
    • 10

    The FTP files ends up in the wrong remote directory!

    My FTP signin is to the `root` of my GoDaddy user space, so for a **LONG** time, I was plagued with a mysterious error report -

    21/05/2020 20:08:57 Sent> PWD
    21/05/2020 20:08:58 Recv> 257 "/" is your current location
    21/05/2020 20:08:58 Sent> PASV
    21/05/2020 20:08:58 Recv> 227 Entering Passive Mode (50,62,103,1,195,231)
    21/05/2020 20:09:00 Sent> MLSD
    21/05/2020 20:09:00 Recv> 150 Accepted data connection
    21/05/2020 20:09:00 Recv> 226-Options: -a -l
    21/05/2020 20:09:00 226 159 matches total
    21/05/2020 20:09:06 Sent> CWD /fg
    21/05/2020 20:09:06 Recv> 250 OK. Current directory is /fg
    21/05/2020 20:09:06 Sent> PASV
    21/05/2020 20:09:06 Recv> 227 Entering Passive Mode (50,62,103,1,195,121)
    21/05/2020 20:09:06 Sent> MLSD
    21/05/2020 20:09:06 Recv> 150 Accepted data connection
    21/05/2020 20:09:06 Recv> 226-Options: -a -l
    21/05/2020 20:09:06 226 143 matches total
    21/05/2020 20:09:31 Sent> CWD /
    21/05/2020 20:09:31 Recv> 250 OK. Current directory is /
    21/05/2020 20:09:31 Unable to load profile:[email protected]/: Profile not found: [email protected]/

    Notice how it started out in root, `/`, and tried to change to `/fg`, but this is NOT a correct, existing directory.

    All my sites `html` files are stored in `/html` directory, so I sort of got the idea that I should be targeting `[email protected]/html` ...

    This sort of works, but say a src file from `[D:\HOMEPAGE\GA\]fg\testb.htm` ends up in `./html/html/testb.htm`, or `./html/html/fg/testb.htm`, both of which are wrong! It should be `./html/fg/testb.htm`...

    What am I doing wrong? Hope someone can help... thanks...
  • Aaron
    Team Scooter
    • Oct 2007
    • 16026

    #2
    Hello,

    It would help if we could see a more complete log, and the command you are issuing to send the file (what is selected, how it is aligned and visible, and the target side). You could email us with a more complete connection log and single file transfer, with a couple screenshots (before Copy and after Copy) at [email protected].

    Generally, if you select a file on one side, it will then copy with the visible sub folder structure to the other side; the top folder paths (base folders) determine the level of the comparison loaded. You can set a lower subfolder to be the new base folder, so that only the current visible structure is copied. You generally want the structure to match from side to side to make future comparisons easier.
    Aaron P Scooter Software

    Comment

    • Geoff McLane
      Journeyman
      • May 2020
      • 10

      #3
      Hi Aaron,

      Thank you for the feedback...

      My workflow is to have 2 local hdd directories of my online site - `D:\HOMEPAGE\GA` and `D:\HOMEPAGE\GAUP` - Then after editing/creating some files in `D:\HOMEPAGE\GA`, I load BC, `GA-GAUP`, only showing diff, and I can see what files I have changed, added... the site has nearly 100,000 files...

      So I end up with a left side showing that I have added say fg\testb.htm - the right side will show a blank - new file, or the date/size of the old file in the right...

      I will select this left testb.htm line, right click and select `Copy to Folder...`, select `ftp://[email protected]/html/`, ensure `Keep relative folder structure` checked, and `start`...

      If the ftp transfer is successful, then I will right click on the left file again, and select `Copy to Right`, and the difference will disappear...

      But as previously posted, they are **NOT** landing in the right place...

      This has been my work flow since 2006, with BC2. Worked perfectly for years, and years, and YEARS...

      Around 2015, BC2 would start the ftp connection, but eventually `freeze` after transmitting a command. May be this was something to do with GoDaddy changing the FTP client, or server, or whatever...

      So I decided to upgrade to BC4 Multi Pro - Order xxxxxxx- but started to get this strange error/failure -

      Unable to load profile:[email protected]/: Profile not found: [email protected]/

      My PC was getting old, and slow, like me!, so I lost interest in maintaining my site, for years... only made a smattering of changes, using FileZilla and/or Putty...

      Then a few months ago, splashed out and bought a new PC - Dell XPS 8930 - and this revived my interest in updating my site, more vigorously...

      So, ok, I tried another test just now - but instead of selecting `ftp://[email protected]/html/`, selected `ftp://[email protected]/`, note no 'html', but the file landed in `./html/testb.htm`

      Logging in through Putty, now find I have 3 copies of that file, 2 copies from previous `with-com/html` tests, **NONE** of which are in the right place, which is ./html/fg/testb.htm -

      [Iusr_10954296@p3nlhftpg258 ~]$ find . -name testb.htm -ls
      88051742 4 -rw----r-- 1 Iusr_10954296 inetuser 180 May 22 06:10 ./html/html/fg/testb.htm
      37339282 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 22 10:07 ./html/html/testb.htm
      2151327 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 27 11:01 ./html/testb.htm
      [Iusr_10954296@p3nlhftpg258 ~]$

      There is something **interesting** going on -

      If I open a Putty SSH session to `geoffair.com` I land in my root, and these files are in my site folder `html`...

      If I use FileZilla, I land in `html`, even thought the site manager says the default remote should be `/`, I land in 'html', and clicking on the `..`, refuses to backup to the real root...

      Now maybe in FileZilla this has something to do with the sync folder comparing??? but...

      Likewise BC4 somehow knew to add a `html` folder, even when it should not... what does it see/detect that causes this?

      If you do not see a quick solution, then I suppose another try would be to create another ftp login, which lands in `html`... will try to find time for this test... but this will take time...

      Yes, I can send you full connection logs - I have at least 4. And 4 `BCsupport` zips - but I do not think a screenshot would add anything... but will try to add some - still getting this new PC fully setup with tools... will advise when sent...

      This is getting **VERY** frustrating... I want to get back to my site updates...

      Kind regards,
      Geoff.
      Last edited by Aaron; 28-May-2020, 04:11 PM. Reason: Removing identifying information

      Comment

      • Geoff McLane
        Journeyman
        • May 2020
        • 10

        #4
        Have sent an email, to support, attaching a zip, geoff-01.zip, with 4 logs, 4 session zips, and 2 screenshots... hope this helps solving this problem... thanks.. geoff.

        Comment

        • Aaron
          Team Scooter
          • Oct 2007
          • 16026

          #5
          Hello,

          Ok, there are a few things happening here, but I think I can narrow down a couple things. On the Copy To Folder dialog, click the '?' button to note the differences between Relative directory structure and Base folder structure options in the Help file. I think you want the preserve Base folders option to preserve the structure up to the base folder no matter what is selected.

          For the profile error, did you add the / after the profile name? FTP syntax uses a slash between server and subfolder structure: ftp://user@server/folder/, but profiles use ?: profilerofile name?folder/subfolder/. If you use the Browse button, you can browse into the Profile using the Profile button to select the target folder, and that should auto-populate the syntax correctly.

          Does this combination of tips help get your Copy To Folder dialog up and running? And were there any remaining issues?
          Aaron P Scooter Software

          Comment

          • Geoff McLane
            Journeyman
            • May 2020
            • 10

            #6
            Hi Aaron,

            Thank you for the further tips... in all my years of using BC, I had never noticed the tiny '?' button. I forget whether I had been successfully using the 'relative' or 'base folder', but in reading that, I guess it was 'base folder'...

            And that error message 'Unable to load profile...' seems to have now gone away - now that a 'profile' has somehow been created, but I never select this...

            I thought I had done these tests before, but I have now conducted two more tests, on 29/05 and 30/05, touch'ing the file to differentiate on date, using 'base folder' each time -

            On 29: I selected `ftp://[email protected]/html/` - it landed in '/html/html/fg/testb.htm'??? Why does BC4 add the 'html' again... BAD! ... especially since this created a NEW site folder '/html/html/fg'... which I had noted via a dialog, in the ealier tests... there has always been a '/html/html' folder, but BC4 added 'fg'...

            On 30: I selected `ftp://[email protected]/` - it landed in the CORRECT place, './html/fg/testb.htm' - great! But how did BC4 'know' to add 'html' in front?

            Putty SSH view - only the first is correct...

            [Iusr_10954296@p3nlhftpg259 ~]$ find . -name testb.htm -ls
            9644439 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 30 08:37 ./html/fg/testb.htm = OK
            88051742 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 29 05:37 ./html/html/fg/testb.htm = BAD
            37339282 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 22 10:07 ./html/html/testb.htm = BAD
            2151327 4 -rw----r-- 1 Iusr_10954296 inetuser 191 May 27 11:01 ./html/testb.htm = BAD

            So to me it is still quite a mystery... and I can send my -05 and -06 logs if you are interested to explore this more... I would certainly like to 'understand' more... advise...

            And have tested on an update of 1 actual site file... and appeared ok...

            Over the next weeks I will get to doing more testing/updating... but maybe this is now solved... thanks...

            Comment

            • Aaron
              Team Scooter
              • Oct 2007
              • 16026

              #7
              Hello,

              Logs would help, or a Before and After full screen screenshot. For creating the HTML folder during a Copy To Folder command, what item was selected in the interface (Copy Filename to get the full file name path to the selected item), and what is the current base folder (select and copy the base folder text in the folder path at the top of the selected item's pane)?
              Aaron P Scooter Software

              Comment

              • Geoff McLane
                Journeyman
                • May 2020
                • 10

                #8
                Hi Aaron,

                Will get around to emailing the logs -05 = failed, and -06 = success - it's late today... probably tomorrow...

                The only difference was in -05 I selected copy to ftp://<user>@<domain>/html/, and in -06 selected just ftp://<user>@<domain>/, in the 'Copy selected files to this folder' drop-down...

                And in BOTH cases selected 'Keep base ...', in the Folder structure: buttons...

                The sc-01.jpg already sent, shows the before, and after, since there is no change after this 'Copy to folder...'

                You can see the base structure in the top text box, name 'D:\HOMEPAGE\GA' in bold, plus the greyed '\fg'

                The left pane shows the selected file in light blue - i.e. + testb.htm, where the '+' is actually a small solid square...

                The right pane shows nothing on this line, since I have not done a 'Copy to Right' yet...

                And it is NOT that BC4 created a 'html' folder - that has always existed. If I remember correctly, it asked about creating the 'fg' folder, in the 'html' folder... as I seem to remember it always does... if it is a new folder... which is good...

                The Putty SSH login shows the folders as -

                ./html/fg/ # and has more than 10 sub-directories...
                ./html/html/ # and had only 1 sub - images

                What did not exist before is './html/html/fg/' ...

                And again right clicking on the blued file, and selecting 'Copy Filename' puts, correctly, 'D:\HOMEPAGE\GA\fg\testb.htm' in the clipboard... so it is getting that right...

                The thing I still do NOT get is when the target is 'ftp://<user>@<domain>/', HOW does './html/, which is correct, get to be the target... per -06 log... i.e. './html/fg/testb.htm' - the correct place...

                While when the target is 'ftp://<user>@<domain>/hmtl/', BC4 still, somehow does another 'cd html', plus 'cd fg', before sending the file, so we get the incorrect './html/html/fg/testb.htm' result... per -05 log

                As stated have now done a few more tests, on other files, using 'ftp://<user>@<domain>/' and 'base folder...', and seems to be working fully correctly... will test more in the coming days... but seems like case closed...

                But this really does smell like a little BUG to me ;=))

                The addition of the known 'html/', which should be helpful to BC4, even if it /IS/ somehow getting the same information from the FTP interface... mysteries abound...

                Comment

                • Aaron
                  Team Scooter
                  • Oct 2007
                  • 16026

                  #9
                  Hello,

                  If "html" is a clickable item within the side you are selected (and not in the base folder path text above) that would explain why it ended up in the target side. You've run a lot of test cases, so it is a little difficult to follow which trial is which result; a fresh test and noting if HTML is within the path or is part of the clickable/loaded structure would be good. If it is clickable/expandable, then it would be included as part of the base folder structure. If you copy to folder target would then have this 'html/...' included in the structure copied into it. If it was an html/ target, it would have html/'html' placed into it. If the target was above html (server/), then html would be placed into it (including into an existing html/).
                  Aaron P Scooter Software

                  Comment

                  • Geoff McLane
                    Journeyman
                    • May 2020
                    • 10

                    #10
                    Hi Aaron,

                    Thank you for your further feedback... I hope all the following covers your questions...

                    The attach to email geoff-02.zip contains -

                    BcLogTrace-05.txt - this had /html/ tacked on the end of the ftp - failed
                    BcLogTrace-06.txt - this is just the root ftp://<user>@<domain> - success
                    BCSupport-05.zip - the support zip for 05 trial
                    BCSupport-06.zip - the support zip for 06 trial
                    05-06.diff - left 21 columns strip from the logs, and a diff -u
                    sc-03.jpg - but this is a WinMergeU screen shot which shows the diff clearer

                    Both logins have the same start, namely just after the pwd is accepted, the current directory is reported as the SAME -

                    Sent> PASS ********
                    Recv> 230 OK. Current restricted directory is /
                    Sent> FEAT
                    ... skipping the features, then again BOTH logs contain ...
                    Sent> PWD
                    Recv> 257 "/" is your current location
                    Sent> REST 1

                    Now I am starting the think that is a downright LIE! It is actually already in ./html/ !!!

                    Remember, 05 is with /html/ added to the ftp address - which seems reasonable, since the ftp login is to a 'root', so BC4 does a 'CWD html', which actually puts you in './html/html', but a PWD again LIES, and says you are in "/html" - NOT really TRUE - I think...

                    BC4 continue, and does a CWD /, recvs 250 OK current is /, UNTRUE in actual fact, then BC4 does CWD html/fg, 250 OK. current /html/fg - lie... and BC4 does PWD, gets 257 "/html/fg" - lies lies lies....

                    Now in 06, with NO /html/ addition, BC4 does a CWD fg, and PWD reports the lie, /fg is your current location. BC4 is happy, and sends the file, and it is put in the right place ./html/fg ...

                    Maybe something needs to be read into the first report "RESTRICTED directory is /"...

                    Now to go back to the real issue - Why did I start adding the /html/ to the FTP?

                    I just went back to my old PC, WIN7-PC, created a 'fg\testc.htm' for testing - in BC4, 4.2.6 (build 23150), expand fg foldeer, right click on the selected 'testc.htm'... Copy to folder... - use current [email protected], select 'keep base...' and start, and it end with the red error -

                    Unable to load profile:[email protected]/fg: Profile not found: [email protected]/fg

                    Attached to email geoff-03.zip, containing BCSupport-07.zip & BcLogTrace-07.txt... of that test... AND a 2018-03-31 BcLogTrace-01.txt, to show when this problem started... years ago... is not new, but persists in the current version...

                    As reported, now I have switched to this new PC, DELL03, BC4 4.3.4 (build 24657) ... Initially started to get the SAME error...

                    But somehow fiddling, fiddling, and fiddling, with the FTP setup, eventually got something working by adding the "/html/" target... AND somehow a 'profile' was also created, and the 'Unable to load...' error went away, but the file landed in wrong place... hence this post...

                    And going back to without the '/html/' got things going correctly...

                    So somehow, the whole problem stems from this missing 'profile:'... and as stated no idea what exactly got this created...

                    Hope all this helps, at least for others... thanks...

                    Geoff.

                    Comment

                    • Aaron
                      Team Scooter
                      • Oct 2007
                      • 16026

                      #11
                      Hello,

                      BC4 FTP Profiles do not save initial directories. The profile can only contain the server name (and other login information). To load into a remote directory, that would be saved as part of the Folder Compare folder path. For Profile syntax, this uses a "?" separator:
                      profilerofile name?html/fg/
                      if using directly typed ftp syntax, then a "/" separator is used:
                      ftp://user@server/html/fg

                      If you try to connect to profilerofile name/html/fg/ then this would fail, or including the path as any part of the Profile definition in the Profiles dialog would also fail.

                      Note, connecting to a server using the typed syntax does generate a Profile for the Profile part of the connection: Profilerofile name for "ftp://user@server/", but does not include the directory information.

                      It's difficult to tell from the logs alone exactly what you are seeing on screen, since you mention something might not be representing itself. Instead of several trials, could we get a screenshot of the current comparison view Before the copy command, showing the path loaded at the top and the selected item? I can note that the RECV commands are what we are receiving back from the server as logs, but since you are attempting to load into different directories in different trials, I can't comment yet on if the log is actually incorrect, depending on what the screenshot shows.
                      Aaron P Scooter Software

                      Comment

                      • Geoff McLane
                        Journeyman
                        • May 2020
                        • 10

                        #12
                        link http://geoffair.com/tmp/sc-01.jpg to screen shot included in geoff-01.zip
                        Last edited by Geoff McLane; 03-Jun-2020, 08:22 AM.

                        Comment

                        • Aaron
                          Team Scooter
                          • Oct 2007
                          • 16026

                          #13
                          Hi,

                          Sorry, what is the sc-01.jpg screenshot currently demonstrating? I see that both base folder paths are local D:\ paths, and there is no Copy To Folder dialog present so I cannot see the target you want to copy testb.htm to.

                          Assuming you are working with two local (no FTP yet), and only the Copy To Folder path is the FTP, it is expected to include "gf\testb.htm". so the target should be either
                          profilerofile name?folders/html/
                          or
                          ftp://user@server/folders/html/
                          in order to copy in as
                          ftp://user@server/folders/html/fg/testb.htm

                          If you try this test (and if we could get a screenshot of the Copy To Folder dialog and the After screenshot of the Folder Compare showing where the file ended up) I can review the specific syntax to see if I can see anything out of place or behaving oddly.
                          Aaron P Scooter Software

                          Comment

                          • Aaron
                            Team Scooter
                            • Oct 2007
                            • 16026

                            #14
                            I should add, this assumes the FTP does land in the root when connecting with the profile, and that it needs a subfolder added at all. To help verify that, connect to the FTP in the Folder Compare using the above syntax or without the /folders/html/, and note where your user lands.
                            Aaron P Scooter Software

                            Comment

                            • Geoff McLane
                              Journeyman
                              • May 2020
                              • 10

                              #15
                              Hi Aaron,

                              Thank you for the further feedback, but it seems this conversation is going backwards ;=))

                              I have already described my workflow, so won't repeat that in full... please re-read...

                              But simply 1234 - 1. modify local 'GA'... 2. load BC to see what I modified... 3. select and copy to folder, via 'FTP'... 4. If all ok, copy to right, local 'GABU' - been doing that for years and 'YEARS', since 2006...

                              Circa 2015 BC2 started to fail during the FTP transfer, so bought BC4, but initially could not get the FTP step 3. working... but had lost interest in my site... still used BC4 now and then to show me my changes, and used other FTP agents to do the FTP to the remote site...

                              Have clearly documented the trouble I am/was having with BC4 - see the last lines BcLogTrace-07.txt AND a 2018-03-31 BcLogTrace-01.txt, in geoff-03.zip - ends with error "Unable to load profile:<name>@<domain>..."

                              SOMEHOW, and I still do NOT KNOW HOW???, I was able to create this missing 'profile' in my Dell03 machine, and BC4 now works fine in there...

                              BUT I still can NOT get BC4 in my Win7-PC (it is actually Win10) working ... I thought by choosing the browse button, [...], and expanding the '> ftp://<name>@<site>', but while that works, and list the remote folders, BUT no 'profile' is created...

                              So I still do not know what I did in the Dell03:BC4 machine that got it setup???

                              RE: FTP, in general

                              I now 'understand' that a FTP connection, is a little like the 'http' protocol, in that it does NOT ever land in the real root of my shared server space... it always lands in the 'html' folder... like a 'http' GET does...

                              So I should NEVER have to add the 'html' folder... that was a big red herring, that caused no end of pain...

                              I can see that NOW with the above '> ftp://<name>@<site>' browsing... I can see it in FileZilla... I can see it when using the 'wput' app... these are ALWAYS only in the 'html' folder, and can NOT be backed up...

                              Only when using say a Putty SSH login, am I in my 'root', which has a 'html' subdirectory, among other things...

                              And please do not get confused with the fact that my 'http' site also has a 'html' folder... so 'html/html' exists, like 'html/fg', etc...

                              And yes, while I had not provided a screenshot with the 'Copy to Folder...' dialog open, I have always desribed what I was selecting... but in case of any doubt, this link - http://geoffair.com/tmp/sc-04.jpg - should make it clear...

                              And, of course, the list is the BCState.xml, in the BCSupport-06.zip, in geoff-02.zip...

                              <TOpCopyToState>
                              <DirStructure Value="ctKeepBase"/>
                              <MRUDestination>
                              <_0 Value="ftp://[email protected]/"/>
                              <_1 Value="ftp://[email protected]/html/"/>
                              <_2 Value="ftp://[email protected]/html//"/>
                              <_3 Value="profile:[email protected]"/>
                              </MRUDestination>
                              </TOpCopyToState>

                              And that leads to another small question - how to get rid of, delete, the two enties ending in 'html/' and 'html//'? The 'red herrings' - Can I just, carefully, modify BCState.xml in '%APPDATA%\Scooter Software\Beyond Compare 4'?

                              So, that just only leaves open the question 'How do I create a profile' in my Win7-PC?'

                              I feel sure other BC4 users must have the same problem... it is certainly NOT self evident... ;=))

                              Thanks, Geoff.

                              Comment

                              Working...