Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade lucence to 9.x with proper JPMS #8362

Merged
merged 27 commits into from
Apr 10, 2022
Merged

Upgrade lucence to 9.x with proper JPMS #8362

merged 27 commits into from
Apr 10, 2022

Conversation

Siedlerchr
Copy link
Member

@Siedlerchr Siedlerchr commented Dec 26, 2021

No more hacking needed. I could compile it and jlink worked locally.
Need to test if the binaries also work.

Fixes koppor#522

Changelog https://lucene.apache.org/core/9_0_0/changes/Changes.html

  • Change in CHANGELOG.md described in a way that is understandable for the average user (if applicable)
  • Tests created for changes (if applicable)
  • Manually tested changed features in running JabRef (always required)
  • Screenshots added in PR description (for UI changes)
  • Checked documentation: Is the information available and up to date? If not, I created an issue at https://github.com/JabRef/user-documentation/issues or, even better, I submitted a pull request to the documentation repository.

@Siedlerchr Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Dec 26, 2021
@Siedlerchr
Copy link
Member Author

Need testing by someone on windows + linux as well with some pdfs.
The segment error is now gone, I had the same prolem

@calixtus
Copy link
Member

Im currently in on a short trip, will test on Wednesday.

@ThiloteE
Copy link
Member

How would the test look like?

@koppor
Copy link
Member

koppor commented Dec 27, 2021

How would the test look like?

Have a bib file with linked PDFs. Do a full text search. The search should find something in the text of the PDFs.

(Documentation of that feature is still a TODO 😅)

@Siedlerchr
Copy link
Member Author

You need the binaries from here https://builds.jabref.org/pull/8362/merge/

@ThiloteE
Copy link
Member

Sorry, fulltext search fails on

JabRef 5.5-PullRequest8362.15--2021-12-27--a98ebec
Windows 10 10.0 amd64
Java 16.0.2
JavaFX 17.0.1+1

Error Log
java.lang.NoClassDefFoundError: Could not initialize class org.apache.lucene.codecs.Codec$Holder
	at org.jabref.merged.module@5.5.30015/org.apache.lucene.codecs.Codec.getDefault(Unknown Source)
	at org.jabref.merged.module@5.5.30015/org.apache.lucene.index.LiveIndexWriterConfig.<init>(Unknown Source)
	at org.jabref.merged.module@5.5.30015/org.apache.lucene.index.IndexWriterConfig.<init>(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.logic.pdf.search.retrieval.PdfSearcher.search(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.model.search.rules.ContainBasedSearchRule.getFulltextResults(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.model.search.rules.ContainBasedSearchRule.applyRule(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.logic.search.SearchQuery.isMatch(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.maintable.MainTableDataModel.lambda$isMatchedBySearch$3(Unknown Source)
	at java.base/java.util.Optional.map(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.maintable.MainTableDataModel.isMatchedBySearch(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.maintable.MainTableDataModel.isMatched(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.maintable.MainTableDataModel.lambda$new$1(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.collections.transformation.FilteredList.refilter(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.collections.transformation.FilteredList$1.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectPropertyBase$Listener.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.binding.ObjectBinding.invalidate(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.binding.BindingHelperObserver.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectPropertyBase.set(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.beans.property.ObjectProperty.setValue(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.StateManager.setSearchQuery(Unknown Source)
	at org.jabref@5.5.30015/org.jabref.gui.search.GlobalSearchBar.performSearch(Unknown Source)
	at org.jabref.merged.module@5.5.30015/org.reactfx.util.FxTimer.lambda$restart$0(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.scenario.animation.shared.TimelineClipCore.visitKeyFrame(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.scenario.animation.shared.TimelineClipCore.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.animation.Timeline.doPlayTo(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.animation.AnimationAccessorImpl.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.scenario.animation.shared.SingleLoopClipEnvelope.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.animation.Animation.doTimePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.animation.Animation$1.lambda$timePulse$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at org.jabref.merged.module@5.5.30015/javafx.animation.Animation$1.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.scenario.animation.AbstractPrimaryTimer.timePulseImpl(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.scenario.animation.AbstractPrimaryTimer$MainLoop.run(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$11(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
	at org.jabref.merged.module@5.5.30015/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at org.jabref.merged.module@5.5.30015/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

@Siedlerchr
Copy link
Member Author

Thanks, probably missing an additional required from the module info

@ThiloteE
Copy link
Member

I still got the error with 814eb0d

Error Log
java.lang.NoClassDefFoundError: Could not initialize class org.apache.lucene.codecs.Codec$Holder
	at org.jabref.merged.module@5.5.30017/org.apache.lucene.codecs.Codec.getDefault(Unknown Source)
	at org.jabref.merged.module@5.5.30017/org.apache.lucene.index.LiveIndexWriterConfig.<init>(Unknown Source)
	at org.jabref.merged.module@5.5.30017/org.apache.lucene.index.IndexWriterConfig.<init>(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.logic.pdf.search.retrieval.PdfSearcher.search(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.model.search.rules.ContainBasedSearchRule.getFulltextResults(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.model.search.rules.ContainBasedSearchRule.applyRule(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.logic.search.SearchQuery.isMatch(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.maintable.MainTableDataModel.lambda$isMatchedBySearch$3(Unknown Source)
	at java.base/java.util.Optional.map(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.maintable.MainTableDataModel.isMatchedBySearch(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.maintable.MainTableDataModel.isMatched(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.maintable.MainTableDataModel.lambda$new$1(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.collections.transformation.FilteredList.refilter(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.collections.transformation.FilteredList$1.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectPropertyBase$Listener.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.binding.ObjectBinding.invalidate(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.binding.BindingHelperObserver.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectPropertyBase.set(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.beans.property.ObjectProperty.setValue(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.StateManager.setSearchQuery(Unknown Source)
	at org.jabref@5.5.30017/org.jabref.gui.search.GlobalSearchBar.performSearch(Unknown Source)
	at org.jabref.merged.module@5.5.30017/org.reactfx.util.FxTimer.lambda$restart$0(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.scenario.animation.shared.TimelineClipCore.visitKeyFrame(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.scenario.animation.shared.TimelineClipCore.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.animation.Timeline.doPlayTo(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.animation.AnimationAccessorImpl.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.scenario.animation.shared.SingleLoopClipEnvelope.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.animation.Animation.doTimePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.animation.Animation$1.lambda$timePulse$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at org.jabref.merged.module@5.5.30017/javafx.animation.Animation$1.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.scenario.animation.AbstractPrimaryTimer.timePulseImpl(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.scenario.animation.AbstractPrimaryTimer$MainLoop.run(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$11(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
	at org.jabref.merged.module@5.5.30017/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at org.jabref.merged.module@5.5.30017/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)

* upstream/main:
  Bump jackson-datatype-jsr310 from 2.13.0 to 2.13.1 (#8366)
  Bump xmpbox from 2.0.24 to 2.0.25 (#8364)
  Bump mockito-core from 4.1.0 to 4.2.0 (#8367)
  Bump h2-mvstore from 2.0.202 to 2.0.204 (#8365)
  Update Gradle Wrapper from 7.3.2 to 7.3.3. (#8363)
@Siedlerchr
Copy link
Member Author

Siedlerchr commented Jan 1, 2022

java.util.ServiceConfigurationError: org.apache.lucene.codecs.Codec: module org.jabref.merged.module does not declare `uses`
	at java.base/java.util.ServiceLoader.fail(Unknown Source)
	at java.base/java.util.ServiceLoader.checkCaller(Unknown Source)
	at java.base/java.util.ServiceLoader.<init>(Unknown Source)
	at java.base/java.util.ServiceLoader.load(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.util.NamedSPILoader.reload(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.util.NamedSPILoader.<init>(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.util.NamedSPILoader.<init>(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.codecs.Codec$Holder.<clinit>(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.codecs.Codec.getDefault(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.index.LiveIndexWriterConfig.<init>(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.apache.lucene.index.IndexWriterConfig.<init>(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.logic.pdf.search.retrieval.PdfSearcher.search(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.model.search.rules.ContainBasedSearchRule.getFulltextResults(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.model.search.rules.ContainBasedSearchRule.applyRule(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.logic.search.SearchQuery.isMatch(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.maintable.MainTableDataModel.lambda$isMatchedBySearch$3(Unknown Source)
	at java.base/java.util.Optional.map(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.maintable.MainTableDataModel.isMatchedBySearch(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.maintable.MainTableDataModel.isMatched(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.maintable.MainTableDataModel.lambda$new$1(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.collections.transformation.FilteredList.refilter(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.collections.transformation.FilteredList$1.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectPropertyBase$Listener.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.binding.ObjectBinding.invalidate(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.binding.BindingHelperObserver.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectPropertyBase.set(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.beans.property.ObjectProperty.setValue(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.StateManager.setSearchQuery(Unknown Source)
	at org.jabref@5.5.30023/org.jabref.gui.search.GlobalSearchBar.performSearch(Unknown Source)
	at org.jabref.merged.module@5.5.30023/org.reactfx.util.FxTimer.lambda$restart$0(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.scenario.animation.shared.TimelineClipCore.visitKeyFrame(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.scenario.animation.shared.TimelineClipCore.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.animation.Timeline.doPlayTo(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.animation.AnimationAccessorImpl.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.scenario.animation.shared.SingleLoopClipEnvelope.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.animation.Animation.doTimePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.animation.Animation$1.lambda$timePulse$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at org.jabref.merged.module@5.5.30023/javafx.animation.Animation$1.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.scenario.animation.AbstractPrimaryTimer.timePulseImpl(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.scenario.animation.AbstractPrimaryTimer$MainLoop.run(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$11(Unknown Source)
	at org.jabref.merged.module@5.5.30023/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)

@Siedlerchr
Copy link
Member Author

Now testedd again, getting a different error:


java.util.ServiceConfigurationError: org.apache.lucene.codecs.Codec: Provider org.apache.lucene.backward_codecs.lucene70.Lucene70Codec could not be instantiated
	at java.base/java.util.ServiceLoader.fail(Unknown Source)
	at java.base/java.util.ServiceLoader$ProviderImpl.newInstance(Unknown Source)
	at java.base/java.util.ServiceLoader$ProviderImpl.get(Unknown Source)
	at java.base/java.util.ServiceLoader$3.next(Unknown Source)
	at org.jabref.merged.module@5.5.30024/org.apache.lucene.util.NamedSPILoader.reload(Unknown Source)

* upstream/main:
  Observable Preferences J (Preview) and minor refactor (#8370)
  quickfix (#8383)
  Live reloading when switching themes (#7336)
  FIX NPE on Merge Dialog (#8380)
  Bump org.eclipse.jgit from 5.13.0.202109080827-r to 6.0.0.202111291000-r (#8378)
  Bump jackson-dataformat-yaml from 2.13.0 to 2.13.1 (#8379)
  Bump tika-core from 2.2.0 to 2.2.1 (#8377)
  Update turabian-fullnote-bibliography-no-ibid.csl
  Squashed 'buildres/csl/csl-locales/' changes from d5ee85de8e..c38205618f
  Squashed 'buildres/csl/csl-styles/' changes from 60bf7d5..f78c707
check if index exists
@Siedlerchr
Copy link
Member Author

It seems to work now, I tested the local jpackage build and this worked now. Would be nice if someone could test with a couple of PDFs

@calixtus calixtus requested a review from btut January 4, 2022 13:04
@ThiloteE
Copy link
Member

ThiloteE commented Jan 7, 2022

Sorry, i still immediately and for every typing attempt, get an error on

JabRef 5.5-PullRequest8362.39--2022-01-04--c6c03ba
Windows 10 10.0 amd64
Java 16.0.2
JavaFX 17.0.1+1

Have not even tried testing the Linux version yet. Will do, when the windows one works.

Error log
java.lang.NoClassDefFoundError: Could not initialize class org.apache.lucene.codecs.Codec$Holder
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.codecs.Codec.forName(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.SegmentInfos.readCodec(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.SegmentInfos.parseSegmentInfos(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.SegmentInfos.readCommit(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.SegmentInfos.readCommit(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.StandardDirectoryReader$1.doBody(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.StandardDirectoryReader$1.doBody(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.SegmentInfos$FindSegmentsFile.run(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.StandardDirectoryReader.open(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.StandardDirectoryReader.open(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.apache.lucene.index.DirectoryReader.open(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.logic.pdf.search.retrieval.PdfSearcher.search(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.model.search.rules.ContainBasedSearchRule.getFulltextResults(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.model.search.rules.ContainBasedSearchRule.applyRule(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.logic.search.SearchQuery.isMatch(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.maintable.MainTableDataModel.lambda$isMatchedBySearch$3(Unknown Source)
	at java.base/java.util.Optional.map(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.maintable.MainTableDataModel.isMatchedBySearch(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.maintable.MainTableDataModel.isMatched(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.maintable.MainTableDataModel.lambda$new$1(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.collections.transformation.FilteredList.refilter(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.collections.transformation.FilteredList$1.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectPropertyBase$Listener.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.binding.ExpressionHelper$SingleInvalidation.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.binding.ObjectBinding.invalidate(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.binding.BindingHelperObserver.invalidated(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.binding.ExpressionHelper$Generic.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.binding.ExpressionHelper.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectPropertyBase.fireValueChangedEvent(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectPropertyBase.markInvalid(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectPropertyBase.set(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.beans.property.ObjectProperty.setValue(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.StateManager.setSearchQuery(Unknown Source)
	at org.jabref@5.5.30039/org.jabref.gui.search.GlobalSearchBar.performSearch(Unknown Source)
	at org.jabref.merged.module@5.5.30039/org.reactfx.util.FxTimer.lambda$restart$0(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.scenario.animation.shared.TimelineClipCore.visitKeyFrame(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.scenario.animation.shared.TimelineClipCore.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.animation.Timeline.doPlayTo(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.animation.AnimationAccessorImpl.playTo(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.scenario.animation.shared.SingleLoopClipEnvelope.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.animation.Animation.doTimePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.animation.Animation$1.lambda$timePulse$0(Unknown Source)
	at java.base/java.security.AccessController.doPrivileged(Unknown Source)
	at org.jabref.merged.module@5.5.30039/javafx.animation.Animation$1.timePulse(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.scenario.animation.AbstractPrimaryTimer.timePulseImpl(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.scenario.animation.AbstractPrimaryTimer$MainLoop.run(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.tk.quantum.QuantumToolkit.pulse(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.tk.quantum.QuantumToolkit.pulseFromQueue(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.javafx.tk.quantum.QuantumToolkit.lambda$runToolkit$11(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.glass.ui.InvokeLaterDispatcher$Future.run(Unknown Source)
	at org.jabref.merged.module@5.5.30039/com.sun.glass.ui.win.WinApplication._runLoop(Native Method)
	at org.jabref.merged.module@5.5.30039/com.sun.glass.ui.win.WinApplication.lambda$runLoop$3(Unknown Source)
	at java.base/java.lang.Thread.run(Unknown Source)
	Suppressed: org.apache.lucene.index.CorruptIndexException: checksum passed (6b29d453). possibly transient resource issue, or a Lucene or JVM bug (resource=BufferedChecksumIndexInput(NIOFSIndexInput(path="C:\Users\Thilo\AppData\Local\org.jabref\JabRef\0.5a\1752446388\segments_3")))
		at org.jabref.merged.module@5.5.30039/org.apache.lucene.codecs.CodecUtil.checkFooter(Unknown Source)
		... 53 more

@btut
Copy link
Contributor

btut commented Jan 7, 2022

Hi! Sorry for the late reply - I was on vacation.
I was very excited when I read this , but unfortunately it does not work for me. Tested on Arch, it builds, runs, indexes, but I do not get any search results.
During the indexing, I noticed I get way more Lucene warnings regarding encoding than I did before. Also, I get a warning that I did not close PDF documents. Maybe a warning regarding the encoding was escalated to an exception in Lucene 9 which is not properly handeled by JabRef? Will have a look.

Update: turns out I was unlucky with the first search term. Full-text search works fine now.

@Siedlerchr Siedlerchr removed the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Jan 7, 2022
@Siedlerchr
Copy link
Member Author

Siedlerchr commented Jan 7, 2022

Seems like I also had an old version with jpackage. Now I still get the same coded not found error fuu
I'v enabled debug mode for remote debugging
https://devdocs.jabref.org/advanced-reading/jpackage#debugging-on-windows
I did a manual deletion of the index folder (Application Support or AppData on Windows JabRef\0.5a) and that seems to work

It may have something to do that lucence writes the codecname in the segments file and it was previously differennt?

@Siedlerchr Siedlerchr closed this Jan 8, 2022
@calixtus calixtus reopened this Jan 16, 2022
@calixtus calixtus closed this Jan 16, 2022
@Siedlerchr
Copy link
Member Author

Reminder to try with 9.1

@Siedlerchr Siedlerchr reopened this Mar 30, 2022
* upstream/main: (150 commits)
  fix unit test
  Add check for developer's documentation
  Merge GitBook view
  Fix zbMath fetcher (#8623)
  GitBook: [#56] No subject
  Add an extra dialog to ask the user whether they want to open the saved file folder when the export the entries (#8567)
  Bump checkstyle from 10.0 to 10.1 (#8620)
  Bump peter-evans/create-pull-request from 3 to 4 (#8619)
  Bump pascalgn/automerge-action from 0.14.3 to 0.15.2 (#8618)
  Bump flexmark from 0.62.2 to 0.64.0 (#8621)
  Bump classgraph from 4.8.141 to 4.8.143 (#8622)
  update test getPart (#8610)
  Add ControlHelper truncateString tests comments (#8612)
  Allow using custom SSL certificates (#8583)
  Fix protectedTerms not stored due to weaklistener (#8609)
  Fix changelog and version parsing (#8578)
  Creating more unit tests for NumericFieldComparatorTest (#8604)
  Fix merge entries dialog exceeding screen size (#8599)
  StringUtilTest new test for method GetPart (#8594)
  Use unkown entry type
  ...

# Conflicts:
#	build.gradle
#	src/main/java/org/jabref/logic/pdf/search/retrieval/PdfSearcher.java
@Siedlerchr Siedlerchr closed this Mar 30, 2022
@Siedlerchr Siedlerchr reopened this Mar 30, 2022
@Siedlerchr Siedlerchr added the status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers label Mar 30, 2022
@Siedlerchr
Copy link
Member Author

There seems to be something wrong with the DocumentViewer. Can someone verify this in main?



[ERROR] COSObject - Can't dereference COSObject{3678, 0} <java.io.IOException: org.apache.pdfbox.io.RandomAccessReadBufferedFile already closed>java.io.IOException: org.apache.pdfbox.io.RandomAccessReadBufferedFile already closed
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.io.RandomAccessReadBufferedFile.checkClosed(RandomAccessReadBufferedFile.java:245)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.io.RandomAccessReadBufferedFile.getPosition(RandomAccessReadBufferedFile.java:99)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.pdfparser.COSParser.dereferenceCOSObject(COSParser.java:584)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.cos.COSObject.getObject(COSObject.java:115)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.cos.COSArray.getObject(COSArray.java:208)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.pdmodel.PDPage.getContentsForRandomAccess(PDPage.java:198)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.pdfparser.PDFStreamParser.<init>(PDFStreamParser.java:60)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.contentstream.PDFStreamEngine.processStreamOperators(PDFStreamEngine.java:485)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.contentstream.PDFStreamEngine.processStream(PDFStreamEngine.java:469)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.contentstream.PDFStreamEngine.processPage(PDFStreamEngine.java:142)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.rendering.PageDrawer.drawPage(PageDrawer.java:264)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.rendering.PDFRenderer.renderImage(PDFRenderer.java:338)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.rendering.PDFRenderer.renderImage(PDFRenderer.java:259)
	at pdfbox@3.0.0-RC1/org.apache.pdfbox.rendering.PDFRenderer.renderImageWithDPI(PDFRenderer.java:245)
	at org.jabref/org.jabref.gui.documentviewer.PdfDocumentPageViewModel.render(PdfDocumentPageViewModel.java:53)
	at org.jabref/org.jabref.gui.documentviewer.DocumentViewerControl$DocumentViewerPage.renderPage(DocumentViewerControl.java:214)
	at org.jabref/org.jabref.gui.documentviewer.DocumentViewerControl$DocumentViewerPage.lambda$0(DocumentViewerControl.java:160)
	at org.jabref/org.jabref.gui.util.BackgroundTask$1.call(BackgroundTask.java:59)
	at org.jabref/org.jabref.gui.util.DefaultTaskExecutor$1.call(DefaultTaskExecutor.java:162)
	at javafx.graphics@18/javafx.concurrent.Task$TaskCallable.call(Task.java:1426)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
	at java.base/java.lang.Thread.run(Thread.java:833)

@Siedlerchr
Copy link
Member Author

Bug is noted in #8628

@Siedlerchr
Copy link
Member Author

Fuu! Method too large on windows. Need to investigate if we can reduce this somehow by applying some uses directives.

Fix exception in document viewer if doc is not pdf
@Siedlerchr
Copy link
Member Author

Because we have a new version of Lucence I switched the index directory
@btut Do you know if theres an option to convert it to a newer format?

@koppor
Copy link
Member

koppor commented Apr 4, 2022

Does Lucene complain when it finds an old format of the index? Would we need a cleanup then? -- Do we need different directories depending on JabRef's version? Similar to IntelliJ creating a settinsg directory for each version? Feels strange...

@Siedlerchr
Copy link
Member Author

Lucence has backwards compatibility codecs. Need to investigate that if it works

* upstream/main:
  Remove obsolete comments
  Improve key generation (#8641)
  Refine search code (#8636)
  Fix import entry by id does not generate citation key (#8361)
  Update Gradle Wrapper from 7.4.1 to 7.4.2. (#8637)
  Bump hmarr/auto-approve-action from 2.1.0 to 2.2.0 (#8638)
  GitBook: [#57] test
  Citation keygen: Return vonPart if lastName is empty (#8634)
  Ensure SSL truststore is present at startup (#8631)
  Squashed 'buildres/csl/csl-styles/' changes from 6a7b708..21e2177 (#8632)
  Add more tests for FieldChange Class (#8614)
@Siedlerchr
Copy link
Member Author

It is forward compatible, e.g. it can use the old index, but older versions cannot read it then once it was created in a newer version.
java.lang.IllegalArgumentException: indexCreatedVersionMajor is in the future: 9

@Siedlerchr Siedlerchr merged commit 93177fa into main Apr 10, 2022
@Siedlerchr Siedlerchr deleted the lucenceUpgrade branch April 10, 2022 20:02
Siedlerchr added a commit that referenced this pull request Apr 11, 2022
…om.github.tomtung-latex2unicode_2.12-0.3.0

* upstream/main:
  Bump h2-mvstore from 2.1.210 to 2.1.212 in /buildSrc (#8665)
  Bump actions/setup-java from 2 to 3 (#8660)
  Bump hmarr/auto-approve-action from 2.2.0 to 2.2.1 (#8661)
  Bump unoloader from 7.3.0 to 7.3.2 (#8662)
  Bump antlr from 3.5.2 to 3.5.3 (#8663)
  Upgrade lucence to 9.x with proper JPMS (#8362)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
status: ready-for-review Pull Requests that are ready to be reviewed by the maintainers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Exceptions during file search
5 participants