BC3 slow... libqtc rebuild?

[mikko]

I have read a few other posts about slow startup times of BC3 under recent Linux versions. Running a rather custom brew of Fedora I'm also affected. BC3 needs 5sec to start and compare anything, no matter how small the files are
I'm using the latest version and tried nearly everything I could figure to find the problem of this slowness.

One idea still to explore is replacing the included qt3 library with the system one which is a recent built and likely better compatible with the surrounding environment. If I do that, however, BC3 crashes in some fontconfig functions. The remaining option is to make sure libqtc is in sync with the system qt3 library and I've tried finding the code to build it. Problem is, it does not look to be even close to what libqtc used to be in the KDE bindings packages on the net (e.g. kdebindings-3.2.3, latest to still include it). Could you share the build procedure for it and what type of qt3 setup was used (3.3.8?) ?

Thank you.

/Mikko

[Aaron]

Hello,

We did replace a lot of the underlying helper libraries for Linux with BC4 and it adds native 64bit support. If upgrading an option to see if that helps? I suspect BC 4.1.2 might run better. You wouldn't need to pay for the upgrade immediately to test; you can install the trial of BC4. But first, run the Tools menu -> Export wizard to create a back up of your current settings. To reinstall BC3 later, if needed, you would need to uninstall BC4 fully, then reinstall BC3 and then Tools menu -> Import your old settings.

It is unlikely we would make any extensive changes to BC3's core dependencies, as these could introduce bugs for people using it on supported or older platforms that they haven't upgraded due to limitations in their workplace or IT policy. The major version break allows us to shake things up a bit more and make changes that might only be supported under a more modern OS.

[mikko]

Hello Aaron,

Thanks for the suggestion. Indeed, BC4 was the 1st option I had in mind and I've tested both the 4.1.1 (64bit) and even the newly released 4.1.2 with same result, they need 15sec to start (older one was a bit faster at ~14). Good part about them is that once started the next comparisons are done at reasonable speed ~1sec for a very small file. For BC3 it's always taking a 5 sec delay to start a session. BC3 does however report times correctly while BC4 gives a rather "mean" assessment of 0.1sec after the wait of 15sec that I mentioned. I wouldn't mind as much the upgrade costs but considering BC3 is paid for, has all the functionality I need, is considerably smaller than the 30MB BC4 and used to work fine not long ago, going the BC4 route looks more like a new problem than a solution.

Would have been so nice to get a recompile of BC3 at 64bit as a future update... Lacking that, is it possible to get help in rebuilding/generating the libqtc support library?

Regards,
/Mikko

[Chris]

We will not release a 64-bit version of Beyond Compare 3. The development tools used to create Beyond Compare 3 only supported 32-bit. Beyond Compare 4 required changing the Linux development tools we use in order to release a 64-bit version. Although we still provide technical support for Beyond Compare 3, no further software updates to Beyond Compare 3 are planned.

We also will not produce a custom libqtc. Given limited resources, there are too many Linux distributions for us to fully support all of them. If you want full technical support on Linux, run a Linux distribution that is on our compatibility list and use the default desktop environment. Only limited technical support is provided for Linux distributions that aren't on the compatibility list or that use non-default desktop environments.

Linux Distribution Compatibility List:
http://www.scootersoftware.com/download.php?zz=kb_linux

If you want to upgrade your license to Beyond Compare 4, fill out the upgrade form on our website:
http://www.scootersoftware.com/shop....p_upgrade_cart

Discounts for license upgrades are described in our upgrade policy:
http://www.scootersoftware.com/shop....=upgradepolicy

[mikko]

Well, I figured my hopes for a lightweight 64bit BC3 were delusory... Going the BC4 route makes the problem even worse, 3x slower startup time for 4x bigger binary that does not add any benefit for my use cases.

With libqtc I wasn't asking for a custom built of it but maybe some help to build my own (if from public sources) or opening its source code to allow for better testing the reasons of the delay. Speaking of this delay, so far all I could see is that BC3's main thread is delayed for 5 sec waiting for a secondary thread which is locked at a timed futex call.

1.840148 futex(0x8d289f4, FUTEX_WAIT_PRIVATE, 1, {4, 999983620}) = -1 ETIMEDOUT (Connection timed out)

it happens at 1.84 sec from startup and lasts until 6.84s when the diff is finally shown.
CPU is nearly idle all of this time...

A similar issue happens (sometimes) when exiting. A Ctrl-Q press will result in another 5 sec delay until exit.

/M

p.s.
Using an official supported distribution is also impossible as I need an up-to-date system setup ~ Fedora 23. I imagined there are other users in my situation who could also benefit if the root cause of this problem is identified.

[Aaron]

Hello,

If you run
strace bcompare

Is the delay long enough to note where in the strace it is delayed? You can post the strace here, or email us at [email protected] with a link back to this forum thread for our reference.

[Aaron]

Something else that might be useful is debugging using gdb. To do se:

export LD_LIBRARY_PATH=/usr/lib/beyondcompare
gdb /usr/lib/beyondcompare/BCompare
run

(wait for beyond compare to Delay, then hit Ctrl+C)/(Or for other users: if crashing, wait for it to crash)

thread apply all bt

This should display a stack trace for all running threads after a delay/crash.

Please send us whatever output gdb generates and we'll keep trying to figure out what is causing the crash on your system.

[mikko]

I attached a trace which I run using a user account with minimal configuration overheads. I set the environment variables and called the main executable directly. On my system there-s a hard link to it in the installation directory named 'bcmp' (the program behaves similarly no matter I call it from the original script or as above).

The (s)trace is attached as well as the files to be compared (a.txt a11.txt)

Command used was:

strace -0ffo tr/u bcmp a*

There's not much overhead compared with a direct run. The moment the diff was shown I pressed Ctrl-Q. It took another ~5sec to terminate.

I use a slightly modified strace to produce the 0-based timing info on 1st column.

Regards,
/M

p.s.
A bit of good news is that after upgrading glibc to 2.22 (and perhaps some other changes) BC4 move much quickly, starting now in ~2sec on a 3.4GHz machine. Not sure where the original 15sec delay came from but no time to investigate that as well. Hopefully BC3 gets up to speed as I hope it to be more responsive than version 4 when properly working.

p.p.s
I had to rename the upload to a .bcpkg extension as the original tar.xz was not accepted, please change back to expand.

[mikko]

After noticing that the recent traces have different futex arguments (and strace looks to have a bug in printing the correct timeout values) I finally found the root problem of the delay: glibc 21 and 22. If using version 20 there's nearly no delay in startup. Still unclear to me why this happens, likely a pthreads/mutex thing... With the code at hand, you have better chances of finding out.
/M

[mikko]

A couple of new traces, maybe of help with your verification. With same execution environment and arguments as before I included a trace with glibc-2.20 which completes in 2.6sec as well as traces of the main thread with stack dumps at futex calls when glibc-2.20, 21 and 22 are used (files u20,u21 and u22). No real difference from 21 to 22 but clearly missing calls compared with version 20.

While with glibc 2.20 startup is now fine I noticed a new problem, maybe again something to do with my setup only. However, perhaps you can replicate it on your environment. The issue is that once a session is started for comparing 2 files then only one more session can be successfully "appended" to the BC2 UI. When trying to do a 3rd comparison, instead of creating it as before in a new tab, BC3 just crashes... (BC4 works fine for this although compared with BC3 which instantly adds the new session, BC4 takes it's time with constant delay of ~1.5sec)

Regards,
/M

[mikko]

After experimenting a bit more with BC4 I found the reason of its slow startup and a stack trace is below. Basically, it has to do with computing font metrics and uploading text glyphs to the X server which is quite slow. This only happens if you start BC4 without asking for a diff. When starting with a diff request (two files as arguments on command line) for some reason all this process is skipped and the diff is shown in less than 2 sec.

writev(6, [{"\212\24O\0A\0@\3\1\0\0\0V\0\0\0\t\0\10\0\1\0\10\ 0\6\0\0\0\0\0\0\0"..., 16240}, {"\0\0\0\0\0\0\0\0005\23\3\23\305\232g\232\377\373 \345\373\364\37
7\377\377\340\324\337\324\340\375\365\375"..., 440}, {"", 0}], 3) = 16680
> /usr/lib64/libc-2.22.so(__writev_nocancel+0x24) [0xf86ed]
> /usr/lib64/libxcb.so.1.1.0(_xcb_conn_wait+0x34b) [0xa57b]
> /usr/lib64/libxcb.so.1.1.0(_xcb_out_send+0x51) [0xa961]
> /usr/lib64/libxcb.so.1.1.0(xcb_writev+0x45) [0xa9e5]
> /usr/lib64/libX11.so.6.3.0(_XSend+0x16e) [0x40c7e]
> /usr/lib64/libXrender.so.1.3.0(XRenderAddGlyphs+0x1f8) [0x2fd8]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QFontEngineX11FT::uploadGlyphToS erver(QFontEngineFT::QGlyphSet*, unsigned int, QFontEngineFT::Glyph*, _XGlyphInfo*, int) const+0x54)
[0x4ea5c4]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QFontEngineFT::loadGlyph(QFontEn gineFT::QGlyphSet*, unsigned int, QFixed, QFontEngine::GlyphFormat, bool) const+0x989) [0x4f39c9]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QFontEngineFT::recalcAdvances(QG lyphLayout*, QFlags<QTextEngine::ShaperFlag>) const+0xc3) [0x4f4293]
> /usr/lib64/qt4/libQtGui.so.4.8.7(hb_getAdvances(HB_Font_*, unsigned int const*, unsigned int, int*, int)+0x111) [0x4231c1]
> /usr/lib64/qt4/libQtCore.so.4.8.7(HB_HeuristicPosition+0x36) [0xf9026]
> /usr/lib64/qt4/libQtCore.so.4.8.7(HB_OpenTypePosition+0x2a8) [0xfaea8]
> /usr/lib64/qt4/libQtCore.so.4.8.7(HB_ShapeItem+0x23) [0xffab3]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QTextEngine::shapeTextWithHarfbu zz(int) const+0x7c7) [0x453ec7]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QTextEngine::shapeText(int) const+0x72) [0x454812]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QTextEngine::shape(int) const+0x95) [0x454b45]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QTextEngine::width(int, int) const+0x29f) [0x45894f]
> /usr/lib64/qt4/libQtGui.so.4.8.7(QFontMetrics::width(QString const&, int, int) const+0x1ef) [0x43158f]
> /usr/lib64/qt4/libQt4Pas.so.5.2.5(QFontMetrics_width+0x6d) [0x14b28d]

[Aaron]

Thanks for this info dump. I'll pass it on to our Linux developer and see what feedback he has.

[mikko]

Thanks, hopefully he finds a cure for this problem. Meanwhile, I found a BC4 issue when comparing large files and reported that in another thread.
/M

[Aaron]

Hello,

Try running qtconfig from your command line and switching to a different or default theme. You may also need to reboot to troubleshoot and make certain the theme is applied.

If qtconfig is not installed, use apt-get install qtconfig-qt4 to install it.

[mikko]

I did that, tried them all with various fonts. The UI style does change, no need for reboot but BC4 behaves the same, much faster to start and do a diff than start the empty home view...

/M