Performance issues with open entry editor

[AEgit]

JabRef 4.0-dev--snapshot--2017-08-31--master--54e0994ba
Windows 10 10.0 amd64
Java 1.8.0_141

This isn't really a bug, it's more an observation I made that I wanted to share, as it might affect some users.

Steps to reproduce:

1. Open an item in the entry editor
2. Keep the entry editor open and select another item
3. The new item is opened in the entry editor
4. Do this multiple times
5. The Windows Task Manager shows a massive increase in the used CPU power (up to 95% on a Corei7 4+4HT system)
6. Indeed, the responsiveness of the entry editor does not feel as quick as with 3.8.2 - it is still ok on my machine, I guess, but people with a slower CPU might experience bigger performance issues

Note, that this might be also related to the size of the databse (>6k entries).
Note furthermore, that this issue does not appear, if the entry editor is closed.

[lenhard]

The entry editor also seems to be the main source of our memory problems, see #3113.

[lenhard]

@AEgit Could you do me a favour and try out the version available here http://builds.jabref.org/entry-editor-performance/

This is a tiny improvement and I would be surprised if it has a big effect performance-wise. Its purpose is more to pave the way for an actual performance improvement that is still to come. But do you maybe notice something, e.g. a little improvement in the responsiveness of the entry editor?

Moreover, it creates a visual effect: When selecting a new entry, then the tabs in the entry editor sort of "fly in". I am interested in your opinion about this effect. Do you maybe like it, is it no problem, or do you find it totally annoying? If the latter is the case, then this shouldn't go into 4.0 anymore.

[AEgit]

JabRef 4.0-dev--snapshot--2017-09-02--entry-editor-performance--548537f12
Windows 10 10.0 amd64
Java 1.8.0_141

@lenhard This version does seem to be slightly faster (although, as you say, the improvement is probably rather small). It still does use, however, a lot of CPU performance.

Regarding the visual effect: It is probably better without it - if you quickly change between different items it can happen that the tabs don't load properly or that it takes a while for them to appear (see appended image).
Dok1.pdf

[koppor]

Maybe, we have to use YourKit (see #3113 (comment)) to profile where the CPU is used?

[Siedlerchr]

I just saw that the PreviewPanel is not yet translated to Javafx, maybe that could help to avoid these issues, too?

[lenhard]

@Siedlerchr Or it will increase them ;) The performance problems are with the entry editor, not with the preview panel. It seems that JavaFX didn't turn things for the better here...

[AEgit]

I was about to say that, but didn't want to sound sarcastic ;)
Would be interesting to know whether JavaFX generally has a much higher performance fingerprint or whether it is just the implementation that can be improved (?).

[lenhard]

Yes absolutely. Maybe there are still some inception problems with JavaFX. Swing is the old gui framework that hasn't been maintained for years and will not receive any future extensions. JavaFX is the new central gui framework for Java. But it just became a core part during the release and maintenance of Java 8.

That's a reason to hope that things will get better on their own with a growing maturity of the framework and its implementation in the JDK. One very major milestone for this is the release of Java 9 which is due in two weeks.

Having said that, I am absolutely sure that there are things to improve in the implementation in JabRef. We're right now half swing and half JavaFX and that certainly doesn't make for a better performance. Unfortunately, the migration from one framework to another is a very long-time thing.

[lenhard]

@AEgit There have been some performance improvements in the entry editor during the last days that should already be noticeable from a user perspective. You could try out the most recent build.

On top of that, more performance improvements are in the pipeline. Right now it seems that we really stand a chance of getting back to a low-footprint JabRef :-)

[AEgit]

@lenhard : Cheers, I had another go with JabRef 4.1-dev--snapshot--2017-10-21--master--f71c5a8c7

I can confirm, that it appears faster than the previous builds. I think 3.8.2 still has a better performance, so I'll continue sticking with that version for my work, but once all the minor issues with version 4 are solved, I'll switch over.
I'm still not sold on the "fly-in" effect for the tabs, but at least I haven't encountered the issue with tabs that do not load properly anymore (bear in mind, however, that I did not test this a lot in the new version). The performance issue with the new groups filter is also still around: #2852

But overall, performance has definitely improved. Thank you very much for your hard work!

[halirutan]

@AEgit We have another pull requestion in the pipeline (#3333) that will improve things further. Once we have attacked all the high-resource hot spots, we can look at the details.

I'm not sure I like the fly-in effect either. With some customizations, I could bring it down to keeping all visible tabs and the fly-in effect (that is probably adjustable by the style of the tab pane) is reduced to this. I'm still not sure I like it, but the only other option is to keep all tabs visible and just disable the ones that are not used

Nevertheless, the main point is that we could bring down the memory consumption from about 2GB to 200MB which get more often garbage collected.

[AEgit]

@halirutan Thank you very much! I'll try the new builds from time to time and let you know, if I spot any issues.

Regarding the fly-in effect: What about just keeping the old behaviour of JabRef 3.2.8, i. e. no specific effect at all (I guess that's the alternative option you are talking about)?

[LinusDietz]

These effects are not explicitly coded by us. They are artifacts of the new UI technology.

[AEgit]

I see, I misunderstood things then. Based on this comment #3175 (comment), I thought the effect had been added on purpose, as it hadn't always been part of the developmental builds of version 4. Thanks for clarifying this.

[tobiasdiez]

With #3333 the tab animation will be disabled.

[halirutan]

@lynyus @AEgit @tobiasdiez

These effects are not explicitly coded by us. They are artifacts of the new UI technology.

But I'm sure they are not artifacts either. With @tobiasdiez first PR, that added new tabs at the wrong position, I didn't see the movement. In fact, there is a

com.sun.javafx.scene.control.skin.TabPaneSkin#stopCurrentAnimation

which is used when he invoked setAll. But as I mentioned in the PR, the skin is a black box and the only way I see is to copy the code and implement a custom skin.

[tobiasdiez]

It is also possible to disable these animations using css, see cd6a2f8

[tobiasdiez]

So now that #3333 is merged, I think we can close this issue. The performance is now on an acceptable level. If you experience any problems, you know the drill: just reopen this issue (or create a new issue if its not really related to the performance of the entry editor).

[AEgit]

I can confirm, the performance compared to previous versions has massively improved in:
JabRef 4.1-dev--snapshot--2017-10-23--master--451b6f3de
Windows 10 10.0 amd64
Java 1.8.0_151

Things I have noticed compared to JabRef 3.8.2:

1. When the database is loaded for the first time, the JabRef developmental version 4.1 will take more CPU power than the old JabRef 3.8.2 (the CPU usage is much higher as seen in the task manager). JabRef will be in this state for a couple of seconds. Afterwards the CPU power consumption drops and is approximately the same.
2. When switching between different items with open entry editor the new JabRef developmental version 4.1 consumes LESS CPU power than 3.8.2. So after the initial start the new version is actually more efficient in using the available resources when it comes to CPU power. Good job!
3. The memory footprint of the new version is sometimes (see below) much larger than in 3.8.2. The same database opened in JabRef 3.8.2 consumes about 1.2 GB of memory, while it consumes ~4.4 GB in the new developmental version 4.1. The memory consumption of the new version is therefore nearly 4x that of the old version. While this seems massive, it is actually not a big issue for me, as I'm using a system with 16GB of RAM.

Now, why am I saying "sometimes": It appears that the same (!) database opened in the new developmental version can consume between 1.4 to 4.4 GB of RAM (without doing much within JabRef). Initially I thought, that this different memory consumption might be related to how I treat the database at the immediate startup, i. e. whether I open the entry editor right at the start or whether I wait for a few seconds, until the database has "cooled off" (= no longer shows the high CPU consumption of step 1 reported above). But it does not seem to be related to that. So far I haven't found a way to reliably reproduce this behaviour: Sometimes the same database consumes 1.4-2 GB of RAM and sometimes it goes up to 4-4.4 GB. N. B.: I'm performing more or less the same actions on the database, i. e. opening the database and then potentially skip through a couple of items.
Now, personally I'm really happy with the new version - the big issue for me was the CPU consumption. The higher RAM consumption (in certain circumstances) is not really an issue for me, as I have enough RAM to cope with it. It could be a problem for other people, I guess (?). On the other hand, if reducing the RAM consumption would mean an increased CPU load (e.g., by compressing data) I would not try to optimize into that direction. The CPU load directly affects how the responsiveness of JabRef is perceived, while the RAM consumption is just something that happens in the background. As long as not too much RAM is consumed, the average user won't even notice it.
I'm just wondering why the RAM consumption of the same database can be so different at the startup of JabRef. Should I open an issue for this (as I said, for me things currently are fine)?

Having said all of this I would really like to thank the developers for this massive improvement in performance!

EDIT: The RAM "issue" might be related to #2533 (albeit the description given there differs from what I experienced, as the RAM consumption is already high at the startup of JabREF).