No announcement yet.

Feature request: Parallel operations

  • Filter
  • Time
  • Show
Clear All
new posts

  • Feature request: Parallel operations

    I'm managing a website that is accessed through SFTP over a DSL internet connection. PHP applications tend to become more complex, composed of many packages that consist of many subdirectories. Scanning the whole tree takes its time.

    My other access to that site is with a self-written deployment tool, based on SSH.NET, an SSH/SFTP library for .NET ( Today I added parallel operations to it. I had heard of SFTP being capable of multiplexing requests over a single stream like HTTP 2. It was an easy task due to the power of .NET. And the effect was impressive. Instead of like 10 seconds scanning the remote system, it's now done in around 2 seconds. The trace log shows all subdirectories so much faster, using up to 8 threads in my case.

    Uploading and deleting many small files is also a lot faster when doing it in parallel. That's all simply because over an internet connection, both sides are waiting for packets to arrive and return most of the time. Small packets that don't fill up the pipe.

    Would it be possible to implement the same in Beyond Compare? Expanding all subdirectories and copying or deleting many files should become much faster, up to an order of magnitude.

    Of course, the still existing bug with parallel transfers over SFTP must be finally fixed then. The bug leads to a crash of BC when I try to transfer (copy or open) a remote file while another scan or transfer is still in progress. This bug exists for years now and has never been fixed. I still run into it from time to time when I'm not patient enough.

  • #2

    Thanks for the suggestion. Adding parallel queue handling is something on our wishlist, although our use of different remote connection libraries and arbitrary left and right loaded locations make implementation non-trivial. I'll add your references/resources to our notes on the subject for our developers to review.
    Aaron P Scooter Software


    • #3
      Thanks for considering it.