Obtain fulltext from sciencedirect

[Julfried]

JabRef version

5.15 (latest release)

Operating system

Windows

Details on version and operating system

22H2

Checked with the latest development build (copy version output from About dialog)

• I made a backup of my libraries before testing the latest development version.
• I have tested the latest development version and the problem persists

Steps to reproduce the behaviour

When trying to obtain the fulltext from sciencedirect, it does not find it. For example taking this DOI: https://doi.org/10.1016/j.ndteint.2023.102990

I am able to add it to the library, but the fulltext does not resolve, although it is clearly available.

Appendix

Logs:
2024-11-07 16:22:10 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Deleted 1 entry(ies)
2024-11-07 16:22:43 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Looking for full text document...
2024-11-07 16:22:43 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050:
2024-11-07 16:22:44 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050:
2024-11-07 16:22:45 [JabRef CachedThreadPool] org.jabref.logic.importer.fetcher.SemanticScholar.findFullText()
INFO: Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
2024-11-07 16:22:45 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: No full text document found for entry Cosarinsky2024.

[Julfried]

I looked into the detailed debug logs, this appears to be a cookie handling issue:

1. The fetcher attempts to access the paper through multiple services:
   pubs.acs.org
   linkinghub.elsevier.com
   sciencedirect.com
   semanticscholar.org

2. Each attempt fails with the same error:
   ERROR: Invalid cookie for [URL]:

3. This happens despite:

• The paper being open access (verified by manual download)
• Testing on different networks (both institutional and home network)

Full debug logs for reference:
2024-11-09 22:27:06 [main] org.jabref.logic.protectedterms.ProtectedTermsLoader.update()
WARN: New protected terms resource '/protectedterms/computer_science.terms' is available and enabled by default.
2024-11-09 22:27:07 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Not updating theme because it hasn't changed
2024-11-09 22:27:07 [JavaFX Application Thread] org.jabref.gui.theme.ThemeManager.updateThemeSettings()
INFO: Theme set to Theme{type=DEFAULT, name=''} with base css StyleSheet{jrt:/org.jabref/org/jabref/gui/Base.css}
2024-11-09 22:27:08 [JavaFX Application Thread] org.jabref.gui.StateManager.setActiveDatabase()
INFO: No open database detected
2024-11-09 22:27:10 [pool-2-thread-1] org.jabref.gui.JabRefDialogService.notify()
INFO: Opening: 'C:\WORKSPACE\AT-Master\Bibliography - Master.bib'
2024-11-09 22:27:10 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-11-09 22:27:11 [JavaFX Application Thread] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
WARN: Resource "" not found.
2024-11-09 22:27:12 [JavaFX Application Thread] org.apache.lucene.internal.vectorization.VectorizationProvider.lookup()
WARN: Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.
2024-11-09 22:27:41 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: Looking for full text document...
2024-11-09 22:27:42 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990:
2024-11-09 22:27:42 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050:
2024-11-09 22:27:43 [JabRef CachedThreadPool] sun.util.logging.internal.LoggingProviderImpl$JULWrapper.log()
ERROR: Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050:
2024-11-09 22:27:43 [JabRef CachedThreadPool] org.jabref.logic.importer.fetcher.SemanticScholar.findFullText()
INFO: Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
2024-11-09 22:27:43 [JavaFX Application Thread] org.jabref.gui.JabRefDialogService.notify()
INFO: No full text document found for entry Automaticestimationsurface_Cosarinsky_2024.

[priyanshu16095]

Got a similar log, just like #12357.

SpringerLink API request failed
kong.unirest.core.UnirestException: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at unirest.java.core@4.4.5/kong.unirest.core.DefaultInterceptor.onFail(DefaultInterceptor.java:43)
	at unirest.java.core@4.4.5/kong.unirest.core.CompoundInterceptor.lambda$onFail$2(CompoundInterceptor.java:54)
	at java.base/java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:215)
	at java.base/java.util.Collections$2.tryAdvance(Collections.java:5075)
	at java.base/java.util.stream.ReferencePipeline.forEachWithCancel(ReferencePipeline.java:147)
	at java.base/java.util.stream.AbstractPipeline.copyIntoWithCancel(AbstractPipeline.java:588)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:574)
	at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:560)
	at java.base/java.util.stream.FindOps$FindOp.evaluateSequential(FindOps.java:150)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:265)
	at java.base/java.util.stream.ReferencePipeline.findFirst(ReferencePipeline.java:687)
	at unirest.java.core@4.4.5/kong.unirest.core.CompoundInterceptor.onFail(CompoundInterceptor.java:56)
	at unirest.java.core@4.4.5/kong.unirest.core.java.JavaClient.request(JavaClient.java:78)
	at unirest.java.core@4.4.5/kong.unirest.core.BaseRequest.request(BaseRequest.java:357)
	at unirest.java.core@4.4.5/kong.unirest.core.BaseRequest.asJson(BaseRequest.java:241)
	at org.jabref@100.0.0/org.jabref.logic.importer.fetcher.SpringerLink.findFullText(SpringerLink.java:58)
	at org.jabref@100.0.0/org.jabref.logic.importer.FulltextFetchers.lambda$getCallable$6(FulltextFetchers.java:102)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:317)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1575)
Caused by: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.net.http/jdk.internal.net.http.HttpClientImpl.send(HttpClientImpl.java:961)
	at java.net.http/jdk.internal.net.http.HttpClientFacade.send(HttpClientFacade.java:133)
	at unirest.java.core@4.4.5/kong.unirest.core.java.JavaClient.request(JavaClient.java:70)
	... 8 more
Caused by: javax.net.ssl.SSLHandshakeException: (certificate_unknown) PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:130)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:383)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:326)
	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:321)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:651)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.onCertificate(CertificateMessage.java:471)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.consume(CertificateMessage.java:367)
	at java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:393)
	at java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:476)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1274)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask$DelegatedAction.run(SSLEngineImpl.java:1260)
	at java.base/java.security.AccessController.doPrivileged(AccessController.java:714)
	at java.base/sun.security.ssl.SSLEngineImpl$DelegatedTask.run(SSLEngineImpl.java:1205)
	at java.base/java.util.ArrayList.forEach(ArrayList.java:1597)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.lambda$executeTasks$3(SSLFlowDelegate.java:1148)
	at java.net.http/jdk.internal.net.http.HttpClientImpl$DelegatingExecutor.execute(HttpClientImpl.java:178)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.executeTasks(SSLFlowDelegate.java:1143)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate.doHandshake(SSLFlowDelegate.java:1109)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader.processData(SSLFlowDelegate.java:508)
	at java.net.http/jdk.internal.net.http.common.SSLFlowDelegate$Reader$ReaderDownstreamPusher.run(SSLFlowDelegate.java:283)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$LockingRestartableTask.run(SequentialScheduler.java:182)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$CompleteRestartableTask.run(SequentialScheduler.java:149)
	at java.net.http/jdk.internal.net.http.common.SequentialScheduler$SchedulableTask.run(SequentialScheduler.java:207)
	... 3 more
Caused by: sun.security.validator.ValidatorException: PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:388)
	at java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:271)
	at java.base/sun.security.validator.Validator.validate(Validator.java:256)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:241)
	at java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:113)
	at org.jabref@100.0.0/org.jabref.logic.net.ssl.TrustStoreManager$1.checkServerTrusted(TrustStoreManager.java:225)
	at java.base/sun.security.ssl.AbstractTrustManagerWrapper.checkServerTrusted(SSLContextImpl.java:1444)
	at java.base/sun.security.ssl.CertificateMessage$T12CertificateConsumer.checkServerCerts(CertificateMessage.java:629)
	... 21 more
Caused by: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:148)
	at java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:129)
	at java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297)
	at java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:383)
	... 28 more

[Julfried]

@priyanshu16095 I am unsure regarding your logs. For me if I try to lookup the full text of the DOI I mentioned in the issue I still get the error "Invalid cookie". I am unsure if this is related to the issue you mentioned. For reference, here are the recent logs for jabref 6.0 alpha:

Detail information:

Postgres server started, connection port: 26450
Not updating theme because it hasn't changed
Theme set to Theme{type=DEFAULT, name=''} with base css StyleSheet{jrt:/org.jabref/org/jabref/gui/Base.css}
No open database detected
Opening: 'C:\WORKSPACE\AT-Master\Bibliography - Master.bib'
Resource "" not found.
Resource "" not found.
Java vector incubator module is not readable. For optimal vector performance, pass '--add-modules jdk.incubator.vector' to enable Vector API.
Looking for full text document...
Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990: 
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
No full text document found for entry Cosarinsky2024.
Looking for full text document...
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
Invalid cookie for https://www.researchgate.net/publication/377043615_Automatic_estimation_of_surface_and_probe_location_for_3D_imaging_with_bidimensional_arrays: 
No full text document found for entry Cosarinsky2024.

[priyanshu16095]

Yeah, this time I got the same log.

Could not find file: /ieee.csl
There is no journal-list.mv. We use a default journal list
Postgres server started, connection port: 52905
Looking for full text document...
Invalid cookie for https://pubs.acs.org/doi/abs/10.1016/j.ndteint.2023.102990: 
Invalid cookie for https://linkinghub.elsevier.com/retrieve/pii/S0963869523002050: 
Invalid cookie for https://www.sciencedirect.com/science/article/pii/S0963869523002050: 
Invalid cookie for https://www.researchgate.net/search.Search.html?type=publication&query=Automatic%20estimation%20of%20surface%20and%20probe%20location%20for%203D%20imaging%20with%20bidimensional%20arrays: 
Invalid cookie for https://www.researchgate.net/search.Search.html?type=publication&query=Automatic%20estimation%20of%20surface%20and%20probe%20location%20for%203D%20imaging%20with%20bidimensional%20arrays: 
Fulltext PDF found @ SemanticScholar. Link: https://doi.org/10.1016/j.ndteint.2023.102990
No full text document found for entry Cosarinsky_2024.

Recording.2025-02-07.mp4

@Julfried Am I doing it right this time?

[Julfried]

Yes, thats exactly the issue that I am facing. Would be very helpful if this gets fixed since sciencedirect is quiete common in many technical fields

[priyanshu16095]

/assign-me

[github-actions]

👋 Hey @priyanshu16095, thank you for your interest in this issue! 🎉

We're excited to have you on board. Start by exploring our Contributing guidelines, and don't forget to check out our workspace setup guidelines to get started smoothly.

In case you encounter failing tests during development, please check our developer FAQs!

Having any questions or issues? Feel free to ask here on GitHub. Need help setting up your local workspace? Join the conversation on JabRef's Gitter chat. And don't hesitate to open a (draft) pull request early on to show the direction it is heading towards. This way, you will receive valuable feedback.

Happy coding! 🚀

⏳ Please note, you will be automatically unassigned if the issue isn't closed within 90 days (by 08 May 2025). A maintainer can also add the "📌 Pinned"" label to prevent automatic unassignment.

[priyanshu16095]

@Julfried Sorry, I have to unassign myself from this for now as I have started working on another issue.
Once I complete that task, I will try to fix this.
For now, it is open to work on.

[priyanshu16095]

/unassign-me